Lead Data Engineer
Tasks
- Apply CI CD integration and code reviews
- Build ML inference pipelines
- Collaborate with engineering, data science, and product teams
- Contribute across software development lifecycle
- Deploy and ensure production readiness
- Design build and operate Databricks data pipelines
- Document and maintain production data solutions
- Drive Databricks performance and cost optimization
- Handle cluster and job management
- Implement security access controls and governance
- Implement test automation
- Manage Databricks workspace configuration and optimization
- Own end-to-end data pipeline lifecycle
- Perform data ingestion, cleansing, and transformation
- Translate business requirements into production pipelines
- Troubleshoot and resolve production incidents
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Agile | Airflow | Amazon S3 | Apache Spark | CI/CD | Databricks | Docker | ETL | Elasticsearch | Git | Grafana | Java | Kafka | Kinesis | Kubernetes | ML inference | Prometheus | Python | Redis | SQL | Scala | Scrum | Terraform
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Senior Data Integration Engineer I INR 3264K-4401KAWS IAM | Airflow | Amazon VPC | Apache Flink | Apache SparkSenior-level Full TimeBengaluru6h ago
-
Analytics Engineer INR 1500K-3000KDBT | Data Governance | Data Modeling | Git | MetaplaneCareer developmentMid-level Full TimeIndia6h ago
-
Data Engineer INR 1340K-1680KAWS | Azure | ClickHouse | DBT | Data ArchitectureDonation matching | Flexible PTO | Flexible schedule | Health insurance | Lunch stipendMid-level Full TimeBengaluru, Karnataka, India7h ago
-
IN_Associate_AI Engineer_GCC_Advisory_Hyderabad INR 2500K-4800KAPI Development | Anthropic API | Async Programming | Autogen | CI/CDFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeHyderabad - Salarpuria, India12h ago
-
Data Engineer (SQL + Python/Java) INR 700K-1500KAWS EMR | Agile Scrum | Amazon Web Services | Apache Spark | Continuous DeliveryHybrid work flexibility | Remote work supportMid-level Full TimeGurgaon - Cyber Park, India12h ago
-
Lead Data Scientist INR 2520K-3500KA/B | A/B Testing | AWS | Agentic AI | Applied AIEmployee assistance program | Medical insurance | Paid time off | Parental leave | Professional developmentSenior-level Full TimeGurgaon - Cyber Park, India12h ago
-
IN_Sr Associate_ AWS Devops With MLOPs Developer_Application Technology_Advisory_ PAN INDIA (early joiners) INR 3000K-4000KAWS | AWS Lambda | Amazon CloudFormation | Amazon EKS | Amazon S3Flexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India12h ago
-
Data Engineer - S/4 Analytics INR 2000K-2400KABAP RAP | AMDP | Access Control | Agile | Analytical ModelsFlexible working hours | Paid parental leave | Paid shift flexibility | Paid training and development programs | Remote work optionMid-level Full TimeBangalore RMZ-ECO WORLD, India12h ago
-
Vice President - Principal Data Engineer INR 3584K-5500KAPI | AWS | Agile | Anomaly Detection | AzureComprehensive healthcare | Flexible time off | Hybrid work model | Retirement plan | Support for working parentsSenior-level Full TimeM11 - Commers III, 43, 44 … R12h ago
-
Mid-level Full TimeBangalore, KA, IN12h ago
-
Senior .NET AI Engineer- 3188 (Remote) INR 2500K-4500K.NET | Azure | C# | Distributed Systems | Event DrivenFlexible working hours | L and D programs | Remote work | Rewards and recognition | Work-life balanceSenior-level Full TimeChennai, India R22h ago
-
Senior-level Full TimeBangalore, India Office (BANGALORE)23h ago
-
ETL Tester INR 1500K-2500KData Mapping | Data Quality | Data Reconciliation | Data Transformation | Data WarehouseSenior-level Full TimeChennai, India23h ago
-
AI Engineer - Bengaluru INR 2000K-3000KDocker | JavaScript | Kubernetes | LLM | Language ProcessingMid-level Full TimeBengaluru, Karnātaka, India23h ago
-
Mid-level Full TimeIN-KA-Bangalore23h ago
-
AI Engineer with RAG / MCP INR 1500K-2500KDocument processing | LLMs | Microservices | Prompt engineering | PythonMid-level Full TimeIN-AP-Hyderabad23h ago
-
Senior-level Full TimeIN-TN-Chennai23h ago
-
Data Engineer INR 1500K-2500KApache Hive | Apache Kafka | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeIN-KA-Bangalore23h ago
-
Apache Beam | Apache Flink | Apache Kafka | Apache Spark | Google DataflowMid-level Full TimePune, Maharashtra, India23h ago
-
Applied AI Solution Architect INR 3000K-4500KAI/ML | API Development | AWS Bedrock | AWS SageMaker | Agent memoryContinuous improvement culture | Ownership | Remote workSenior-level Full TimeIndia-Remote R1d ago
-
Data Bricks Infra Engineer INR 1500K-2500KAccess Control | Caching | Data Governance | Databricks | Databricks SQLWork from office 5 days per weekSenior-level Full TimeGurugram, Haryana, India1d ago
-
Software Engineer INR 2000K-3300KAKS | Angular | App Service | Azure | Azure AppEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India1d ago
-
Senior-level Full TimeBangalore, Karnataka, India1d ago
-
Associate Software Engineer - Scala and spark INR 1500K-2000KAgile | Apache Spark | CI/CD | Databricks | GitHubEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India1d ago
-
Amazon Web Services | Apache Airflow | Apache Kafka | Audit Readiness | CI/CDCareer growth | Continuous learning opportunities | Health insurance | Hybrid work | Remote workSenior-level Full TimeIndia1d ago