Data engineering lead
IND - Bengaluru, India
INR 3000K-4000K (estimate) Senior-level Full Time
Tasks
- Build batch micro batch and streaming data pipelines
- Collaborate with stakeholders across analytics data science AI and cloud teams
- Create technical roadmaps and measurable execution plans
- Drive Delta Parquet partitioning and Z ordering optimizations
- Enforce data engineering standards and governance controls
- Implement observability error handling and retry mechanisms
- Lead data quality reconciliation and lineage
- Lead scalable data architecture
- Manage production data accuracy timeliness and reliability
- Mentor and grow data engineering team
- Optimize AWS cloud cost and infrastructure utilization
- Optimize Spark and Databricks performance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS DMS | AWS Lambda | AWS Step Functions | Amazon Aurora | Amazon Aurora RDS | Amazon CloudWatch | Amazon EventBridge | Amazon S3 | Apache Spark | Aurora RDS | CI/CD | Data Governance | Data Lineage | Data Modeling | Data Products | Data Quality | Data fabric | Data mesh | Databricks | Databricks Unity | Databricks Unity Catalog | Delta Lake | Dimensional data | Dimensional data modeling | ETL | Git | GitHub Actions | Parquet | PySpark | Python | SQL | Spark Structured Streaming | Step Functions | Structured Streaming | Test automation | Unity Catalog
Education
Roles
Data Engineering | Data Engineering Lead | Engineering Lead | Lead
Related jobs
-
Azure Data | Azure Data Factory | Azure Databricks | Azure Purview | Azure SynapseSenior-level Full TimeBangalore, Karnataka, India8h ago
-
Lead Data Engineer - Databricks, AWS INR 2200K-3200KAWS Glue | AWS Lake Formation | AWS S3 | Amazon Athena | Amazon EMRSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Strategic Analytics - Vice President INR 3300K-4000KAI machine learning | Apache Spark | Data Governance | Data Product Management | Data QualityExecutive-level Full TimeBengaluru, Karnataka, India12h ago
-
Staff Engineer, Artificial Intelligence INR 2000K-3500KAPI Design | AWS | Apache Spark | Data Governance | Data PipelinesSenior-level Full TimeIndia - Bangalore - 5th floor, …18h ago
-
Specialist, Global Workforce Analytics INR 800K-1200KAdvanced Report Writer | Calculated Fields | Custom Report | Custom Report Fields | Data AnalysisMid-level Full TimePune, India18h ago
-
Lead Specialist - Data Analytics & Insight INR 3000K-4000KAWS | Azure | Cloud Computing | Data Modeling | Data QualityHybrid work | Inclusive culture | Professional development | Supportive leadershipSenior-level Full TimeMumbai - Hiranandani, India R18h ago
-
Data Engineering Manager INR 700K-1000KApache Airflow | Apache Flink | Apache Hadoop | Apache Hive | Apache IcebergMid-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India18h ago
-
Senior-level Full TimeTower 02, Manyata Embassy Business Park, …18h ago
-
AWS | Amazon Web Services | Apache Spark | Azure | Azure DataFlexibility programmes | Inclusive benefits | MentorshipSenior-level Full TimeBengaluru Millenia, India18h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Bangalore INR 1500K-2000KAzure | CI/CD | Databricks | Delta Lake | DockerFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportMid-level Full TimeBengaluru Millenia, India18h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Bangalore INR 1500K-2000KAzure | CI/CD | Databricks | Delta Lake | DockerFlexibility programmes | Inclusive benefits | MentorshipMid-level Full TimeBengaluru Millenia, India18h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Bangalore INR 1500K-2000KAzure | CI/CD | Databricks | Delta Lake | DockerFlexible work arrangements | Inclusive benefits | MentorshipMid-level Full TimeBengaluru Millenia, India18h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Bangalore INR 1500K-2000KAzure | CI/CD | Databricks | Delta Lake | DockerFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportMid-level Full TimeBengaluru Millenia, India18h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Bangalore INR 1500K-2000KAzure | CI/CD | Databricks | Delta Lake | DockerFlexibility programmes | Inclusive benefits | MentorshipMid-level Full TimeBengaluru Millenia, India18h ago
-
Data Architect - IWPB INR 1000K-3000KAPI Design | Data Governance | Data Lake | Data Lineage | Data ManagementSenior-level Full TimeIndia - Bengaluru1d ago
-
Gen AI INR 2500K-4500KAWS | Agent systems | Airflow | Asynchronous programming | Attention MechanismsSenior-level Full TimeIN-KA-Bangalore1d ago
-
AI Engineering Lead INR 2475K-4500KAWS | AWS Bedrock | Agentic Workflows | Azure | Azure Prompt FlowCompany-provided equipment | English lessons | Flexible working options | Learning opportunities | Mentorship programsSenior-level Full TimeHyderabad, TS, India1d ago
-
Senior Data Scientist INR 1500K-2000KData Analysis | Data Modeling | Data Science | Machine Learning | Project ManagementSenior-level Full TimeIndia1d ago
-
Procurement Analytics Lead INR 1250K-1800KBenchmarking | Category Taxonomy | Dynamic Arrays) | ERP | ExcelGlobal team collaboration | Remote-first work environmentSenior-level Full TimeDelhi1d ago
-
Data Analyst INR 2040K-3356KA/B | A/B Testing | B testing | Business Analytics | Data VisualizationSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Lead Data Scientist INR 2520K-3380KApache Hadoop | Artificial Intelligence | Azure Kubernetes | Azure Kubernetes Service | Azure Machine LearningEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeVadodara, GJ, India1d ago
-
Lead Software Engineer - Apache Spark, AWS INR 2000K-4500KAWS | Agile methodology | Amazon Kafka | Apache Airflow | Apache SparkSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Lead - IT Data Engineering and Analytics INR 2000K-3900KAAS | ADLS Gen2 | AI Foundry | AI Platform | Active DirectoryArchitecture ownership | Hybrid work | Mentorship | Team leadershipSenior-level Full TimeBangalore, India1d ago
-
Staff - IT Data Engineering and Analytics INR 2040K-3380KAgile | C# | C++ | Cassandra | Cloud ComputingSenior-level Full TimePune, India1d ago
-
Lead Data Engineer INR 2520K-3880KADLS | AWS | Airflow | Azure | CassandraBonuses for approved content | Broadband reimbursement | Medical insurance | Paid certifications | Professional development allowanceSenior-level Full TimeIndia - Remote R1d ago