Lead Data Engineer
Tasks
- Build CI/CD pipelines
- Build lakehouse architecture
- Conduct code reviews
- Configure Databricks workspaces and clusters
- Design ETL/ELT pipelines
- Design data models
- Ensure data security and privacy
- Implement data governance and lineage
- Implement data quality checks
- Integrate cloud storage and messaging
- Lead and mentor data engineers
- Optimize Spark jobs
- Participate in agile ceremonies
- Set up monitoring and alerting
- Translate business requirements into data solutions
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Kafka | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Event | Azure Event Hubs | Azure Key Vault | CI/CD | Data Factory | Data Lake Storage Gen2 | Databricks | Datadog | Delta Lake | Delta Live Tables | Delta Live) | Event Hubs | Git | GitHub Actions | Grafana | Key Vault | Photon | Prometheus | PySpark | Python | SQL | Spark SQL | Terraform | Unity Catalog | Z-Order
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Sr SDE (Java + AI) INR 3000K-5000KAI orchestration | AWS | Alerting | Automated testing | AzureDesign review leadership | MentorshipSenior-level Full TimePune City, India3h ago
-
Embedded Software Development Engineer (8-10 Years) INR 3200K-4110KAUTOSAR | Agile | Assembly | Automated testing | Automotive SPICESenior-level Full TimeBengaluru, India9h ago
-
Data Scientist - GenAi-AgenticAi INR 2500K-5000KAI orchestration | AWS | Agentic AI | Crew AI | Data MiningSenior-level Full TimeIndia10h ago
-
Staff Software Engineer - Data Platform INR 3000K-4000KAWS | Azure | Bazel | BigQuery | BuildkiteCell phone stipend | Free food | Holidays | In-office collaboration | Medical insuranceSenior-level Full TimeBengaluru, Karnataka, India, APAC11h ago
-
Senior-level Full TimeMumbai, MH, IN, 40007214h ago
-
Senior AI Engineer – Data Science & Generative AI INR 2000K-5000KAPI Development | AWS | Agentic AI | Asynchronous programming | AutogenSenior-level Full TimeBengaluru, KA, IN14h ago
-
Lead Data Analyst & AI - Product Delivery INR 3000K-5000KData Modeling | Data Quality | PowerBI | Python | SQLSenior-level Full TimeJohnson Controls India COEE117h ago
-
Lab and Center Data Engineer INR 938K-1408KAnsible | Bash | Enterprise Linux | Infrastructure as Code | KVMMid-level Full TimePune, Maharashtra, India20h ago
-
Senior-level Full TimeHyderabad, IND-TG, India23h ago
-
Planview Administration 169576 INR 1500K-2500KData Governance | Data Modeling | Data Pipelines | Data Validation | ELTSenior-level Full TimeHyderabad, IND-TG, India23h ago
-
Senior-level Full TimeChennai, Tamil Nādu, India23h ago
-
AI / ML Developer / Engineer INR 2000K-3000KAI Agent | AI agent orchestration | Agent Orchestration | Artifact Registry | Backup and Disaster RecoveryMid-level Full TimeIndia1d ago
-
GCP Backend Developer / Engineer INR 3000K-5000KAgile | BigQuery | CI/CD | Cloud Functions | Cloud RunSenior-level Full TimeIndia1d ago
-
Software Engineer II INR 2771K-4000KAlgorithms | Amazon Web Services | Cloud platform | Data Structures | Distributed SystemsComprehensive health and wellbeing support | Learning and development opportunitiesEntry-level Full TimeGurugram, HR, India1d ago
-
Sr.DataOPS Developer INR 2000K-3000KAWS Greengrass | Airflow | Apache Flink | Apache Kafka | Apache Kafka StreamsSenior-level Full TimeBangalore, India1d ago
-
AI Engineer - India INR 1800K-3000KAzure APIM | Cloud platform | Cypher | Embeddings | FastAPICareer pathing | Flexi benefit plan | Focus time Fridays | Food at work | Group medical coverageMid-level Full TimeChennai R1d ago
-
Data Scientist - GenAi/Agentic-Ai INR 2000K-5000KAI orchestration | AWS | Advanced Analytics | Crew AI | Data MiningMentoring program | Training and coachingSenior-level Full TimeIndia1d ago
-
DevOps Track Sr.Engineer INR 3000K-4200KAccess Control | Access Control Lists | Azure | Azure Dashboards | Azure DataSenior-level Full TimeIndia1d ago
-
Data Scientist INR 3000K-5000KAWS | Airflow | Amazon SageMaker | Apache Spark | Azure Machine LearningSenior-level Full TimeIN-MH-Pune1d ago
-
Net Core Azure Cosmos DB Gen AI INR 1800K-3300K.NET | API Management | ASP.Net Core | App Service | Application InsightsMid-level Full TimeIN-KA-Bangalore1d ago
-
Senior-level Full TimeIN-AP-Hyderabad1d ago
-
Data Platform Engineer- Azure Databricks INR 2000K-3900KAccess Management | Ansible | Apache Ranger | Azure Data | Azure Data LakeSenior-level Full TimeNoida, Uttar Pradesh, UTTAR PRADESH, India1d ago
-
Senior Applied AI/ML Engineer INR 3500K-5000KAgentic Systems | Autonomous Agents | Bias Mitigation | Data Analysis | Deep learningSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Forward Deployed Engineer, Google Cloud Business Platform INR 2500K-5000KArtificial Intelligence | Content Security | Content Security Policy | Data Storage | Distributed ComputingSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Assistant Manager- Python Gen AI with GCP- Bangalore INR 1500K-2500KADK | AI Agents | API Development | Agile | Cloud platformWork from client office 5 days per weekMid-level Full TimeBangalore, Karnataka, India1d ago