Lead Data Engineer - ETL
Gurugram, Haryana, India
INR 2000K-2500K (estimate) Senior-level Full Time
Tasks
- Build and maintain Operational Data Store ODS
- Collaborate on CI/CD pipelines with DevOps
- Design and implement ETL/ELT pipelines
- Develop denormalized API ready data models
- Enforce data governance data quality and metadata management
- Implement batch and streaming ingestion with Kinesis or Lambda
- Implement idempotent processing CDC merge upsert and reconciliation
- Ingest MMS and POS data into AWS platforms
- Integrate ODS with microservices via APIs and event streams
- Optimize data processing workflows with Apache Spark
- Provide technical leadership and mentorship through peer reviews
- Store and query data using Amazon S3 and analytics services
- Tune pipeline performance for scalability and cost
Perks/Benefits
- N/A
Skills/Tech-stack
AWS CodePipeline | AWS Glue | AWS Lambda | Amazon EMR | Amazon Kinesis | Amazon Redshift | Amazon S3 | Apache Kafka | Apache Spark | CDC | CI/CD | Couchbase | Data Governance | Data Quality | Data Store | DynamoDB | ELT | ETL | GitLab | Metadata Management | MongoDB | NoSQL | Operational data | Operational data store | PySpark
Education
N/A
Roles
Data Engineer | Data Engineering | Data Engineering Lead | Engineer | Engineering Lead | Lead
Related jobs
-
Senior Data Integration Engineer I INR 3264K-4401KAWS IAM | Airflow | Amazon VPC | Apache Flink | Apache SparkSenior-level Full TimeBengaluru4h ago
-
Analytics Engineer INR 1500K-3000KDBT | Data Governance | Data Modeling | Git | MetaplaneCareer developmentMid-level Full TimeIndia4h ago
-
Data Engineer INR 1340K-1680KAWS | Azure | ClickHouse | DBT | Data ArchitectureDonation matching | Flexible PTO | Flexible schedule | Health insurance | Lunch stipendMid-level Full TimeBengaluru, Karnataka, India5h ago
-
IN_Associate_AI Engineer_GCC_Advisory_Hyderabad INR 2500K-4800KAPI Development | Anthropic API | Async Programming | Autogen | CI/CDFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeHyderabad - Salarpuria, India10h ago
-
Data Engineer (SQL + Python/Java) INR 700K-1500KAWS EMR | Agile Scrum | Amazon Web Services | Apache Spark | Continuous DeliveryHybrid work flexibility | Remote work supportMid-level Full TimeGurgaon - Cyber Park, India10h ago
-
IN_Sr Associate_ AWS Devops With MLOPs Developer_Application Technology_Advisory_ PAN INDIA (early joiners) INR 3000K-4000KAWS | AWS Lambda | Amazon CloudFormation | Amazon EKS | Amazon S3Flexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India10h ago
-
Solutions Architect - Data Engineering INR 3000K-3880KAWS | AWS Data | AWS Data Migration | AWS Data Migration Services | AWS Managed AirflowFlexible distributed work | Ownership and accountability | Professional mentorship | Remote-first cultureSenior-level Full TimeIndia - Remote R19h ago
-
Senior .NET AI Engineer- 3188 (Remote) INR 2500K-4500K.NET | Azure | C# | Distributed Systems | Event DrivenFlexible working hours | L and D programs | Remote work | Rewards and recognition | Work-life balanceSenior-level Full TimeChennai, India R20h ago
-
Senior-level Full TimeBangalore, India Office (BANGALORE)20h ago
-
ETL Tester INR 1500K-2500KData Mapping | Data Quality | Data Reconciliation | Data Transformation | Data WarehouseSenior-level Full TimeChennai, India20h ago
-
Data Engineer INR 1500K-2500KApache Hive | Apache Kafka | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeIN-KA-Bangalore21h ago
-
Apache Beam | Apache Flink | Apache Kafka | Apache Spark | Google DataflowMid-level Full TimePune, Maharashtra, India21h ago
-
Data Bricks Infra Engineer INR 1500K-2500KAccess Control | Caching | Data Governance | Databricks | Databricks SQLWork from office 5 days per weekSenior-level Full TimeGurugram, Haryana, India23h ago
-
Software Engineer INR 2000K-3300KAKS | Angular | App Service | Azure | Azure AppEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India23h ago
-
Mid-level Full TimeBangalore, Karnataka, India23h ago
-
Senior-level Full TimeBangalore, Karnataka, India23h ago
-
Associate Software Engineer - Scala and spark INR 1500K-2000KAgile | Apache Spark | CI/CD | Databricks | GitHubEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India23h ago
-
Amazon Web Services | Apache Airflow | Apache Kafka | Audit Readiness | CI/CDCareer growth | Continuous learning opportunities | Health insurance | Hybrid work | Remote workSenior-level Full TimeIndia23h ago
-
Semantic Data Engineer INR 2520K-3500KAmazon Redshift | Apache Airflow | Apache Superset | BigQuery | Cube.devSenior-level Full TimeNoida,Uttar Pradesh,India23h ago
-
Senior-level Full TimeIndia - Bengaluru; India - Pune23h ago
-
Sr. Data Engineer INR 1500K-2500KACID | Amazon Redshift | Amazon S3 | Apache Kafka | Azure DataWork from office 5 days per weekSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Principal Software Engineer - AI Platform INR 2515K-3500KAgile | Apache Airflow | CI/CD | Cloud Native | Data AnalysisFlexible medical benefits | Life insurance | Retirement options | Volunteer programsSenior-level Full TimeIndia1d ago
-
Senior Software Engineer (Java/Python) INR 2200K-4000KAPI Gateway | AWS CLI | AWS Lambda | Amazon CloudWatch | Amazon DynamoDBSenior-level Full TimeHyderabad, India1d ago
-
Senior Data Engineer - GCP INR 1500K-2400KAI Embeddings | APIs | Airflow | Apache Airflow | Apache SparkSenior-level Full TimeHyderabad1d ago
-
Lead Data Engineer – Python & GCP INR 1500K-2500KAirflow | Apache Airflow | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeHyderabad1d ago