Manager - Data Engineering
Bengaluru, Karnataka, India, India
⚠️ We'll shut down after Aug 1st - try foo🦍 for all jobs in tech ⚠️
Tesco
Welcome to Tesco online. Discover great value groceries, plus clothing, recipes, bank, and mobile services. Browse on Tesco.com now!Build solutions for the real-world problems in workforce management for retail. You will work with a team of highly skilled developers
and product managers throughout the entire software development life cycle of the products we own. In this role you will be responsible
for designing, building, and maintaining our big data pipelines. Your primary focus will be on developing data pipelines using available
tec hnologies.
In this job, I’m accountable for:
Following our Business Code of Conduct and always acting with integrity and due diligence and have these specific risk responsibilities:
-Represent Talent Acquisition in all forums/ seminars pertaining to process, compliance and audit
-Perform other miscellaneous duties as required by management
-Driving CI culture, implementing CI projects and innovation for withing the team
-Design and implement scalable and reliable data processing pipelines using Spark/Scala/Python &Hadoop ecosystem.
-Develop and maintain ETL processes to load data into our big data platform.
-Optimize Spark jobs and queries to improve performance and reduce processing time.
-Working with product teams to communicate and translate needs into technical requirements.
-Design and develop monitoring tools and processes to ensure data quality and availability.
-Collaborate with other teams to integrate data processing pipelines into larger systems.
-Delivering high quality code and solutions, bringing solutions into production.
-Performing code reviews to optimise technical performance of data pipelines.
-Continually look for how we can evolve and improve our technology, processes, and practices.
-Leading group discussions on system design and architecture.
-Manage and coach individuals, providing regular feedback and career development support aligned with business goals.
-Allocate and oversee team workload effectively, ensuring timely and high-quality outputs.
-Define and streamline team workflows, ensuring consistent adherence to SLAs and data governance practices.
-Monitor and report key performance indicators (KPIs) to drive continuous improvement in delivery efficiency and system uptime.
-Oversee resource allocation and prioritization, aligning team capacity with project and business demands.
Key people and teams I work with in and outside of Tesco:
People, budgets and other resources I am accountable for
in my job:
TBS & Tesco Senior Management
TBS Reporting Team
Tesco UK / ROI/ Central Europe
Any other accountabilities by the business
Business stakeholders
Operational skills relevant for this job:
Experience relevant for this job:
Skills: ETL, YARN,Spark, Hive,Hadoop,PySpark/Python
• 7+ years of experience inbuilding and maintaining big data
(anyone) Linux/Unix/Shell environments(anyone), Query
platforms using Spark/Scala.
optimisation
• Strong knowledge of distributed computing principles and big
Good to have: Kafka, restAPI/reporting tools.
data technologies such as Hadoop, Spark, Streaming etc.
• Experience with ETL processes and data modelling.
• Problem-solving and troubleshooting skills.
• Working knowledge on Oozie/Airflow.
• Experience in writing unit test cases, shell scripting.
• Ability to work independently and as part of a team in a
fast-paced environment. You will need Refer to responsibilities About us Tesco in Bengaluru is a multi-disciplinary team serving our customers, communities, and planet a little better every day across markets. Our goal is to create a sustainable competitive advantage for Tesco by standardising processes, delivering cost savings, enabling agility through technological solutions, and empowering our colleagues to do even more for our customers. With cross-functional expertise, a wide network of teams, and strong governance, we reduce complexity, thereby offering high-quality services for our customers. Tesco in Bengaluru, established in 2004 to enable standardisation and build centralised capabilities and competencies, makes the experience better for our millions of customers worldwide and simpler for over 3,30,000 colleagues. Tesco Business Solutions: Established in 2017, Tesco Business Solutions (TBS) has evolved from a single entity traditional shared services in Bengaluru, India (from 2004) to a global, purpose-driven solutions-focused organisation. TBS is committed to driving scale at speed and delivering value to the Tesco Group through the power of decision science. With over 4,400 highly skilled colleagues globally, TBS supports markets and business units across four locations in the UK, India, Hungary, and the Republic of Ireland. The organisation underpins everything that the Tesco Group does, bringing innovation, a solutions mindset, and agility to its operations and support functions, building winning partnerships across the business. TBS's focus is on adding value and creating impactful outcomes that shape the future of the business. TBS creates a sustainable competitive advantage for the Tesco Group by becoming the partner of choice for talent, transformation, and value creation.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow Architecture Big Data Data governance Data pipelines Data quality Engineering ETL Hadoop Kafka KPIs Linux Oozie Pipelines PySpark Python Scala SDLC Shell scripting Spark Streaming
Perks/benefits: Career development Competitive pay Health care Medical leave Salary bonus Wellness
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.