Lead Data Engineer
Tasks
- Build ETL/ELT frameworks
- Build data ingestion and transformation pipelines
- Conduct code reviews architecture reviews
- Design scalable batch data pipelines
- Design scalable real-time data pipelines
- Develop reusable data frameworks libraries pipeline templates
- Drive CI CD automated testing observability adoption
- Drive cost-optimization strategies
- Enhance developer tooling and platform capabilities
- Ensure data quality lineage observability governance
- Ensure high availability fault tolerance disaster recovery
- Implement infrastructure-as-code
- Implement secure data access controls
- Implement stream processing solutions
- Operate cloud native data platforms
- Optimize data jobs performance scalability cost
- Provide technical leadership mentorship design guidance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS CloudFormation | Airflow | Amazon Redshift | Amazon S3 | Apache Flink | Apache Spark | Avro | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | BigQuery | CI/CD | Cloud Networking | Cloud Storage | Data Lake Storage | Data Modeling | Databricks | Dimensional Modeling | Disaster Recovery | Docker | Dynatrace | ELT | EMR | ETL | GCP | GDPR | Gatling | GitHub Actions | Go | Google Cloud | Google Cloud Storage | IAM | JMeter | JUnit | Java | Jenkins | Kafka | Kubernetes | Monitoring | ORC | Observability | PCI DSS | Parquet | Prometheus | Python | SQL | Snowflake | Snowflake Schema | Splunk | Star Schema | Terraform
Education
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Planview Administration 169576 INR 1500K-2700KData Cleansing | Data Modeling | Data Validation | Data pipeline | Data pipeline orchestrationSenior-level Full TimeHyderabad, IND-TG, India5h ago
-
Storage 170374 INR 2000K-4000KAWS | Azure | Backup solutions | Command Line | Data Protection24x7 operational support | Cross-functional collaborationSenior-level Full TimeHyderabad, IND-TG, India6h ago
-
Mid-level Full TimeHyderabad, IND-TG, India6h ago
-
Senior Agentic AI Engineer ID80140 INR 2000K-5000KAnthropic API | CI/CD | Infrastructure as Code | Langchain | LanggraphFlexible hours | Learning budget | Mentorship | Performance reviews | Remote workSenior-level Full TimeBengaluru, India10h ago
-
Lead AI Engineer ID80141 INR 2500K-5000KAWS S3 | Amazon Web Services | Azure | CI/CD | Cloud SecurityAnnual learning budget | Collaborative culture | Flexible hours | Internal TechTalks | MentorshipSenior-level Full TimeBengaluru, India10h ago
-
Senior-level Full TimeBangalore - Karnataka - India Chennai …10h ago
-
Mid-level Full TimeBangalore - Karnataka - India10h ago
-
Staff Data Engineer INR 1500K-2500KAPIs | AWS | Agentic Workflows | Automated testing | Cloud PlatformsSenior-level Full TimeHyderabad, Telangana , India R12h ago
-
Data Engineer, YouTube Business Organization INR 1000K-2000KApache Flume | Apache Spark | Data Governance | Data Lakes | Data ModelingMid-level Full TimeBengaluru, Karnataka, India12h ago
-
Senior Software Engineer, Generative AI, Gaming INR 2500K-5000K3D mathematics | C++ | Computer Graphics | Computer Vision | Data ProcessingSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Sr Advanced Data Engineer INR 1500K-3000KAWS | Apache Spark | Azure | Data Cleansing | Data ModelingSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Data Engineering Lead INR 2000K-4580KAWS | Apache Airflow | Apache Spark | CI/CD | DBTBike to work | Career progression | Educational assistance | Employee assistance programme | Employee discountSenior-level Full TimeMumbai, MH, India R12h ago
-
Senior Artificial Intelligence/Machine Learning Engineer INR 2500K-5000KAWS | Agile | Azure | Embedding Models | Hugging FaceCompany-paid certifications | Financial consultations | Internal events | Language courses | Legal consultationsSenior-level Full TimeIndia12h ago
-
AI Senior Software Engineer INR 2000K-4500KAI Foundry | AI Search | AI gateway | AWS Bedrock | AngularSenior-level Full TimeBengaluru, India13h ago
-
Software Engineer II - Python INR 1500K-2500KAWS | Automated testing | Cloud | Code Refactoring | Data AnalysisMid-level Full TimeBengaluru, Karnataka, India13h ago
-
Senior Data Engineer INR 3000K-4000KAWS | Airflow | Apache Spark | CI/CD | DBTBike to work | Career mentorship | Educational assistance | Employee assistance programme | Employee discountSenior-level Full TimeMumbai, MH, India R13h ago
-
Data Engineer 2 INR 1500K-2500KAgile Development | Cassandra | Cloud Computing | Data Catalog | Data GovernanceSenior-level Full TimePune, Maharashtra, India14h ago
-
Data Engineer Manager - AWS Databricks INR 1500K-2600KAWS | Apache Spark | Automation | CI/CD | Data GovernanceSenior-level Full TimeHyderabad, TS, India14h ago
-
Data Engineer INR 3000K-4000KAmazon S3 | Databricks | ELT | ETL | PythonBike to work | Educational assistance | Employee assistance program | Employee discount | Enhanced parental leaveSenior-level Full TimeBengaluru, KA, India R14h ago
-
Senior Software Engineer INR 1163K-3000KAutomated testing | Batch Processing | CI/CD | DAX | Data GovernanceFlexible work arrangement | Hybrid work modelSenior-level Full TimeIndia, Telangana, Hyderabad R15h ago
-
Senior / Lead Data Engineer - Azure Databricks INR 1500K-3000KAzure Databricks | Azure DevOps | CI/CD | Cloud Security | Conformed DimensionsSenior-level Full TimeHyderabad, TS, India15h ago
-
Lead Software Engineer - Java/Python, Data Engineering INR 3000K-4200KAPI Design | AWS | Airflow | Apache Iceberg | AzureSenior-level Full TimeHyderabad, Telangana, India16h ago
-
Lead Software Engineer - Java/Python, Data Engineering INR 3000K-4200KAPI Design | AWS | AWS Bedrock | Airflow | Apache IcebergSenior-level Full TimeHyderabad, Telangana, India16h ago
-
Lead Software Engineer - Java/Python, Data Engineering INR 3000K-4200KAPI Design | AWS | AWS Bedrock | Apache Airflow | Apache IcebergSenior-level Full TimeHyderabad, Telangana, India16h ago
-
Sr Lead Software Engineer - Java/Python, Data Engineering INR 2500K-4300KAPI Design | AWS | Airflow | Apache Iceberg | Apache KafkaSenior-level Full TimeHyderabad, Telangana, India16h ago