Lead Data Engineer
Tasks
- Build and operate cloud-native data platforms
- Build reusable data frameworks libraries and templates
- Conduct code and architecture reviews
- Design batch and real-time data pipelines
- Develop ETL/ELT frameworks
- Drive CI CD automated testing and observability
- Drive cost-optimization strategies
- Ensure data quality lineage observability and governance
- Ensure high availability fault tolerance and disaster recovery
- Implement infrastructure-as-code
- Implement secure data access controls
- Implement stream processing for near real time use cases
- Ingest and transform data for data lakes and data warehouses
- Optimize data jobs for performance scalability and cost
- Provide technical leadership mentorship and design guidance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Amazon EMR | Amazon Redshift | Amazon S3 | Apache Airflow | Apache Flink | Apache Spark | Avro | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | BigQuery | CI/CD | Cloud Storage | Cloud platform | CloudFormation | Data Lake | Data Lake Storage | Data Warehouse | Databricks | Dimensional Modeling | Docker | Dynatrace | ELT | ETL | GDPR | Gatling | GitHub Actions | Go | Google Cloud | Google Cloud Platform | Google Cloud Storage | IAM | Infrastructure as Code | JMeter | JUnit | Java | Jenkins | Kafka | Kubernetes | ORC | PCI DSS | Parquet | Prometheus | Python | Snowflake | Snowflake Schema | Splunk | Star Schema | Structured Streaming | Terraform | “as-code”
Education
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Data Platform Operations Engineer (Databricks, DBT, Snowflake, AWS, Aurora) - Mumbai, MH - AS A INR 500K-1500KAWS | Amazon Aurora | Amazon S3 | Backup and Recovery | CDCMid-level Full TimeMumbai, MH7h ago
-
Amazon Aurora | Amazon RDS | Amazon S3 | Amazon Web Services | CI/CDMid-level Full TimeDahisar West, Mumbai City, Maharashtra8h ago
-
Data and Digital Initiatives INR 3000K-4000KAI | API Integration | Cloud Computing | Credit Risk | DashboardingFlexible working | Inclusive work environment | Opportunities to grow | Professional developmentSenior-level Full TimeMumbai, Maharashtra, India9h ago
-
AI Engineer with RAG / MCP INR 1500K-3000KDocument processing | LLM | Microservices | Model Context Protocol | Prompt engineeringMid-level Full TimeIN-AP-Hyderabad10h ago
-
SDET ETL Testing INR 1500K-2500KAWS | AWS Command Line Interface | AWS Data | AWS Data Catalog | AWS GlueSenior-level Full TimeIN-HR-Gurgaon10h ago
-
AWS Data Engineer- Gen AI INR 1500K-2500KAWS CDK | AWS Glue | AWS Step Functions | Amazon RDS | Amazon RedshiftSenior-level Full TimeIN-TN-Chennai10h ago
-
AWS Data Engineer- Airflow INR 1500K-2000KAWS Glue | AWS Lambda | Amazon Redshift | Amazon S3 | Apache HadoopMid-level Full TimeIN-AP-Hyderabad10h ago
-
Junior GenAI Developer INR 400K-480KAgentic Workflows | Embeddings | Git | Hybrid search | LLM APIHealth insurance | Language courses | Mentorship | Professional development opportunities | Relocation programEntry-level Full TimeAll Cities, India R11h ago
-
Lead Data Analyst – Business & Process INR 1800K-2700KAnalytics Platforms | Budgeting | Business Intelligence | Dashboarding | Data AnalysisFamily support | Learning and development opportunities | Life Coverage | Maternity benefits | Medical coverageExecutive-level Full TimeHyderabad, TS, India12h ago
-
MLOps Engineer INR 1200K-1680KAWS | Bedrock | CI/CD | Data Versioning | DockerEmployee Share Savings Plan | Hybrid work | Learning and mentoring | Paid volunteer day | Personal coachingMid-level Full TimeBengaluru12h ago
-
Senior-level Full TimePune, Maharashtra, India13h ago
-
Senior-level Full TimePune, Maharashtra, India13h ago
-
Staff Machine Learning Engineer INR 4000K-5000KC# | C++ | Deep learning | Fine Tuning | HadoopDental insurance | Equity awards | Health insurance | Hybrid work | Vision insuranceSenior-level Full TimeBengaluru, Karnataka, India R13h ago
-
Mid-level Full TimeIndia13h ago
-
Lead Software Engineer - Data Engineering INR 3500K-5000KAWS | Airflow | Authentication | Authorization | CI/CDSenior-level Full TimeBengaluru, Karnataka, India13h ago
-
Senior-level Full TimeMumbai, Maharashtra, India14h ago
-
Junior GenAI Developer INR 400K-480KAgent workflows | Embeddings | Evaluation metrics | Git | Hybrid searchCertification programs | Global team | Health insurance | Inclusive culture | Internship opportunitiesEntry-level Full TimeAll Cities, India R15h ago
-
Senior/Lead Data Scientist INR 3000K-5000KAPIs | Agents | Azure | Azure DevOps | CI/CDCareer development | Continuous learningSenior-level Full TimeBangalore, Karnataka, IN15h ago
-
Engineer - Digital & AI Solutions INR 302K-346KAWS | Agile | Azure | Big Data | Cloud platformCharity Programmes | Discounted gym memberships | Environmental initiatives | Flexible working | Hybrid workingEntry-level Full TimeHyderabad, Telangana, India15h ago
-
AI/ML Engineer_MS INR 2500K-5000KAI Foundry | AI Search | AWS Bedrock | Amazon QuickSight | App ServiceSenior-level Full Timetelengana, India16h ago
-
CLOUD AMAZE - DW Architect (Practice) INR 1500K-2500KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Full TimeIndia17h ago
-
Senior INR 3000K-5000KAI Foundry | Azure AI | Azure AI Foundry | Azure OpenAI | Azure OpenAI ServiceSenior-level Full TimeGurgaon, Haryana, India17h ago
-
Engineering Lead,Microservices,Java Springboot /Sr. Associate Director, Technology Management INR 1300K-2200KApache Airflow | Apache Beam | Apache Flink | BigQuery | ClickHouseMid-level Full TimePune, Maharashtra, India R17h ago
-
Senior-level Full TimeBengaluru, KA, India17h ago
-
Data Analyst Technical Lead (Alteryx) INR 3000K-4000KAPI Integration | Alteryx Designer | Alteryx Server | Data Cleansing | Data ReconciliationUK shiftSenior-level Full TimeNoida, UP, India17h ago