Lead Data Engineer - ETL
Gurugram, Haryana, India
INR 2000K-2500K (estimate) Senior-level Full Time
Tasks
- Build and maintain Operational Data Store ODS
- Collaborate on CI/CD pipelines with DevOps
- Design and implement ETL/ELT pipelines
- Develop denormalized API ready data models
- Enforce data governance data quality and metadata management
- Implement batch and streaming ingestion with Kinesis or Lambda
- Implement idempotent processing CDC merge upsert and reconciliation
- Ingest MMS and POS data into AWS platforms
- Integrate ODS with microservices via APIs and event streams
- Optimize data processing workflows with Apache Spark
- Provide technical leadership and mentorship through peer reviews
- Store and query data using Amazon S3 and analytics services
- Tune pipeline performance for scalability and cost
Perks/Benefits
- N/A
Skills/Tech-stack
AWS CodePipeline | AWS Glue | AWS Lambda | Amazon EMR | Amazon Kinesis | Amazon Redshift | Amazon S3 | Apache Kafka | Apache Spark | CDC | CI/CD | Couchbase | Data Governance | Data Quality | Data Store | DynamoDB | ELT | ETL | GitLab | Metadata Management | MongoDB | NoSQL | Operational data | Operational data store | PySpark
Education
N/A
Roles
Data Engineer | Data Engineering | Data Engineering Lead | Engineer | Engineering Lead | Lead
Related jobs
-
Engineeing Manager, Machine Learning INR 1068K-1890KA/B | A/B Testing | Apache Airflow | Apache Spark | B testingCommuter benefits | Financial wellness support | Healthcare dental and vision | Hybrid work schedule | Mental health supportMid-level Full TimeBengaluru, India4h ago
-
Staff Software Engineer - Database Engineering INR 2475K-3100KAmazon Aurora | Ansible | Backups | Cassandra | CitusSenior-level Full TimeBangalore, India R11h ago
-
Staff Software Engineer - Data Platform INR 2500K-3500KAmazon AWS | CI/CD | Caching | Data integration | Database DesignHybrid work | Onsite in Bengaluru as neededSenior-level Full TimeBangalore, India11h ago
-
Senior-level Full TimeGurugram, Haryana, India14h ago
-
Senior AI Engineer INR 2755K-4590KAgent Orchestration | Anthropic | Azure AI | CI/CD | Enterprise SearchAsynchronous work | Continuous learning culture | Mentorship opportunities | Remote workSenior-level Full TimeRemote, Bangalore R17h ago
-
Cyber security with Gen AI - Bengaluru INR 2000K-3500KAPI Integration | Bash | CI/CD | GitHub | GoCareer growth | Diversity and inclusion | Inclusive cultureSenior-level Full TimeIndia - Bengaluru18h ago
-
Senior-level Full TimeGurugram, Haryana, India; Hyderabad; Noida, Uttar …19h ago
-
Data Engineer - Snowflake/DBT INR 1500K-2500KCI/CD | DBT | Data Governance | Data Modeling | Data SecurityMid-level Full TimeBangalore, India19h ago
-
Senior Data Engineer - Snowflake/DBT INR 1500K-2040KCI/CD | DBT | Data Modeling | Data Warehousing | DatabricksSenior-level Full TimeBangalore, India19h ago
-
Senior Data Engineer INR 3000K-4000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon S3 | CloudWatchCab facility | Flexible working hours | Free medical insurance | Maternity leave | Meal facilitySenior-level Full TimeGurgaon (SEZ1)20h ago
-
Software Engineer - AI Products INR 2000K-5000KAgent Orchestration | Agentic AI | CI/CD | Dialogue Management | DockerSenior-level Full TimeNew Delhi, DL, India21h ago
-
AI Quality Engineer INR 1800K-3000KApplication Insights | Azure | Behavioral Testing | CI/CD | DataverseSenior-level Full TimeBengaluru, KA, India21h ago
-
Senior-level Full TimeBengaluru, KA, India21h ago
-
Cloud Data Engineer I, Professional Services, Google Cloud INR 1200K-1500KApache Hive | Apache Pig | Apache Spark | Big Data | C plus plusMid-level Full TimeBengaluru, Karnataka, India21h ago
-
Mid-level Full TimeBengaluru, KA, India22h ago
-
Data Lead INR 1500K-2600KAI Data | AI data pipelines | Azure Data | Azure Data Factory | Business glossaryHybrid workSenior-level Full TimeKolkata, WEST BENGAL, India22h ago
-
A/B | A/B Testing | API Design | AWS | AirflowSenior-level Full TimeBengaluru, Karnataka, India22h ago
-
Data engineering Analytics INR 1890K-2400KAgile | Cloud platform | GCP | Google Cloud | Google Cloud PlatformSenior-level Full TimeGurugram, HR, India22h ago
-
Lead Software Engineer – Java, DevOps, Cloud, AI Solutions INR 2200K-3500KAWS | AWS CloudWatch | Alerting | Bash | CI/CDSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
DE&A - Core - Advanced Visualization & Cloud Reporting - Tableau INR 1500K-3000KAWS | Access Control | Active Directory | Amazon EBS | Amazon EC2Mid-level Full TimePune, Maharashtra, India1d ago
-
Senior-level Full TimeIndia1d ago
-
Senior-level Full TimeIndia1d ago
-
Assistant Manager - Cloud Engineering INR 1500K-2000KAWS Lake Formation | Apache Airflow | Apache Kafka | Apache Spark | Azure CosmosClient relationship management | Consulting experience | Professional developmentMid-level Full TimePune, Maharashtra, India1d ago
-
AI Data Engineer III INR 3000K-3880KAWS | Agile | Apache Airflow | CI/CD | DBTContinuous learning | Generous leave policies | Health insurance | Hybrid work | Wellness programsSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
AWS Glue | AWS IAM | AWS Lambda | Amazon EC2 | Amazon EMRSenior-level Full TimeHyderabad, Telangana, India1d ago