Lead Data Engineer
Tasks
- Build data ingestion and transformation pipelines
- Build reusable data frameworks libraries and templates
- Conduct code and architecture reviews
- Design and operate cloud-native data platforms
- Design batch and real-time data pipelines
- Develop ETL/ELT frameworks
- Drive CI CD automated testing and observability
- Drive cost-optimization strategies
- Enforce best practices and engineering excellence
- Ensure data quality lineage observability governance
- Ensure high availability fault tolerance disaster recovery
- Implement infrastructure-as-code
- Implement secure data access controls
- Implement stream processing solutions
- Lead development mentoring and design guidance
- Optimize data jobs performance scalability cost
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS CloudFormation | Airflow | Amazon EMR | Amazon Redshift | Amazon S3 | Apache Flink | Apache Spark | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | BigQuery | CI/CD | Cloud Networking | Cloud Storage | Data Governance | Data Lake Storage | Data Lineage | Data Warehousing | Databricks | Dimensional Modeling | Docker | Dynatrace | ELT | ETL | GDPR | Gatling | GitHub Actions | Go | Google Cloud | Google Cloud Storage | IAM | Infrastructure as Code | JMeter | JUnit | Java | Jenkins | Kafka | Kubernetes | Observability | PCI DSS | Prometheus | Python | SQL | Snowflake | Snowflake Schema | Splunk | Star Schema | Structured Streaming | Terraform | “as-code”
Education
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Mid-level Full TimeHyderabad, IND-TG, India4h ago
-
Associate Data Engineer INR 1500K-2500KAgile | Apache NiFi | Azure DevOps | C plus plus | Database systemsMid-level Full TimePune, India7h ago
-
Senior-level Full TimeBANGALORE, Karnataka, Inde7h ago
-
Advanced Engineer-AI/ML INR 3264K-4800KAPI Integration | ASR | AUTOSAR | AWS | Audio ProcessingEmployee discounts | Recognition rewards program | Training opportunities | Tuition reimbursement | Wellness benefitsSenior-level Full TimeBangalore - Karnataka, India - Kalyani …7h ago
-
Senior Software Engineer - Data License Analytics, Pune INR 3000K-4720KAWS S3 | Agent to Agent | Agent to Agent A2A | Apache Airflow | Apache IcebergSenior-level Full TimePune7h ago
-
Spark, Scala, Hadoop & Quantexa/Senior Consultant Specialist INR 2000K-2800KApache Airflow | Apache Spark | CI/CD | DevOps | ElasticsearchFlexible working hours | On-call support | Shift workSenior-level Full TimeBangalore, Karnataka, India R8h ago
-
Mid-level Full TimeIN-KA-Bangalore8h ago
-
AI Lead and Automation Tester - Document Intelligence INR 2500K-5000KAI Workflow Orchestration | AI workflow | Automation Testing | Clinical Trial Compliance | Clinical trialSenior-level Full TimeIN-TN-Chennai8h ago
-
Senior Consultant INR 2000K-4500KAPI Testing | Adversarial Testing | Amazon SageMaker | Azure Machine Learning | Bias detectionSenior-level Full TimeIN-AP-Hyderabad8h ago
-
Assistant Director - Insights And Analytics INR 3000K-5199KCode reuse | Data Governance | Data Quality | Data Wrangling | Machine LearningExecutive-level Full TimeBangalore, Karnataka, India R8h ago
-
Senior-level Full TimeBangalore - Karnataka - India8h ago
-
Java Full Stack Engineer - Professional INR 1200K-3000KAngular | Apache Airflow | Apache Spark | Azure Kubernetes | Azure Kubernetes ServiceSenior-level Full TimeIndia9h ago
-
Senior-level Full Timebengaluru, India9h ago
-
Senior-level Full Timebengaluru, India9h ago
-
Lead Data Platform Engineer INR 2040K-3900KAPI Integration | AWS | AWS CloudWatch | AWS DataZone | AWS GlueEnhanced medical benefits | Family well-being benefits | Hybrid work | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India9h ago
-
Anomaly Detection | Autoencoders | Bayesian optimization | Classification | ClusteringOnsite workEntry-level Full TimeHyderabad, India10h ago
-
AI Agent Development Kit | API Gateway | Agent Development Kit | Agentic AI | Audit LoggingSenior-level Full TimeMumbai, Maharashtra, India10h ago
-
AI/ML Infrastructure Software Engineer INR 3000K-5000KC++ | JAX | Keras | Machine Learning | Machine Learning FairnessSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Data Engineer INR 1300K-2000KAPI Integration | Data Modeling | Data Transformation | Data pipeline | Power BIMid-level Full TimeHyderabad, Telangana, India10h ago
-
Senior Software Engineer INR 3000K-4000KALB | API Gateway | AWS ECS | AWS Glue | AWS LambdaFamily well-being benefits | Health benefits | Hybrid work | Paid time offSenior-level Full TimeHyderabad, India10h ago
-
Senior QA Engineer (Data Platform Testing) INR 2500K-3000KAPI Gateway | API Testing | AWS Lambda | CI/CD | CloudWatchSenior-level Full TimeHyderabad, India10h ago
-
Consultant - Cloud Data Engineer INR 1500K-2500KApache Airflow | Apache Kafka | Apache Spark | Azure | Azure DataMid-level Full TimeGurgaon, Haryana, India10h ago
-
Generative AI Lead INR 3264K-5876KAI Governance | APIs | Cloud Architecture | Cloud infrastructure | Data LakeHybrid workSenior-level Full TimeHyderabad, India11h ago
-
Mid-level Full TimePune, Maharashtra, India11h ago
-
Senior-level Full TimeHyderabad, India11h ago