Senior Data Engineer
Tasks
- Build event ingestion and processing pipelines
- Build tooling for historical corrections and dataset reprocessing
- Define event infrastructure requirements and interfaces
- Define event schemas and compatibility rules
- Design workflows for event dataset replay and rebuild
- Enable downstream model rebuild without manual intervention
- Ensure deterministic explainable identity resolution
- Ensure event streams are deduplicated and ordered
- Ensure pipelines are reliable scalable and observable
- Handle identity merges splits and corrections
- Implement automated validation and contract enforcement
- Implement identity stitching logic for anonymous and authenticated users
- Maintain event versioning and compatibility guarantees
- Propagate deletion and suppression requests through pipelines
- Provide guidance for consistent product event instrumentation
Perks/Benefits
Skills/Tech-stack
Airflow | Apache Kafka | Apache Spark | Data Governance | Data Modeling | Data Quality | Data Versioning | Data pipeline | Entity Resolution | Event Streaming | Identity resolution | Observability | Python | SQL | Schema Validation
Education
N/A
Related jobs
-
Senior Analytics Engineer – Product & Customer Analytics INR 2516K-3356KDBT | Data Documentation | Data Lineage | Data Modeling | Data QualityGlobal team collaboration | Remote-first work environmentSenior-level Full TimeIndia R10h ago
-
Staff Analytics Engineer – Customer Data Platform INR 2516K-3380KDBT | Data Architecture | Data Documentation | Data Modeling | Data TestingSenior-level Full TimeIndia R10h ago
-
Data Engineer (L3) INR 3200K-3880KAWS | Agile | Apache Hudi | Azure | DBTDonation support | Generous time off | Healthcare | Parental leave | Remote-first workSenior-level Full TimeRemote - India R20h ago
-
Staff Machine Learning Engineer INR 2520K-4500KAWS | Amazon Redshift | Big Data | Classification | ClusteringAdditional day off quarterly | Calm subscription | Employee assistance program | Extended parental leave | Flexible working hoursSenior-level Full TimeIndia R1d ago
-
Databricks/ Pyspark Developer INR 500K-1400KApache Kafka | Azure Data | Azure Data Factory | Data Factory | DatabricksFlexible work hours | Remote workSenior-level Full TimeChennai, IN - Remote R1d ago
-
Senior-level Full TimeJakarta, Indonesia; Chennai, India; Remote (India) R2d ago
-
Software Engineer, Backend - Remote (US or International) INR 1500K-2000KApache Cassandra | Apache Druid | Apache Hadoop | Apache Kafka | Apache PinotOpen Source Community Collaboration | Remote workEntry-level Full TimeRemote (India) R2d ago
-
Azure | Csharp | Data Ingestion | Data cleaning | Machine LearningCommunity forums | Laptop provided | Medical insurance | Mentorship | No weekend workSenior-level Full TimeIndia R3d ago
-
Senior Software Engineer INR 3200K-4225KAccess Control | Airflow | Artificial Intelligence | Astronomer | CI/CDSenior-level Full TimePune, India R4d ago
-
Data Engineer (Data Application Developer) INR 2516K-3356KAWS | Amazon Kinesis | Amazon SQS | DBT | Data ModelingCommunity involvement support | Employee support programs | Hybrid work options | In-person work options | Remote work optionsSenior-level Full TimeRemote (IND), India R4d ago
-
Senior-level Full TimeBangalore, INDIA R5d ago
-
AI Solutions Architect INR 2500K-4500KAWS | Convolutional Neural Networks | Data Governance | Distributed Systems | DockerSenior-level Full TimeIndia - Remote R5d ago
-
Senior Azure Fabric Data Engineer INR 1500K-2000KAirflow | Azure | Azure Data | Azure Data Factory | Azure Data LakeRemote work | Work from homeSenior-level Full TimeIndia - Remote R5d ago
-
Senior-level Full TimeRemote, India R5d ago
-
Senior Staff Engineer, Generative AI INR 2500K-4500KAPI Development | AWS | Airflow | Artificial Intelligence | Azure MLContinuous learning opportunities | Flexible work hours | Remote work optionsSenior-level Full TimeRemote, India R5d ago
-
Senior Staff Engineer, Generative AI INR 2500K-4500KAI Deployment | AI/ML | AI/ML cloud services | API Development | AirflowFlexible work hours | Health benefits | Learning and development opportunities | Remote workSenior-level Full TimeRemote, India R5d ago
-
Senior Staff Engineer, Generative Ai INR 2500K-4500KAPI Development | AWS SageMaker | Azure ML | BERT | BicepSenior-level Full TimeRemote, India R5d ago
-
Senior Staff Engineer, Generative Ai INR 2500K-4500KAI/ML | AI/ML libraries | AWS | Agentic AI | AirflowSenior-level Full TimeRemote, India R5d ago
-
Senior Software Engineer INR 3100K-3500KAPI Design | Audit Logging | Backward Compatibility | Compliance | Data ModelingSenior-level Full TimeIndia - Remote R6d ago
-
Senior Staff Engineer, ERP Oracle Technical (ODI & EBS) INR 300K-370KAnalytics Cloud | Automation | Autonomous Data Warehouse | Data Integrator | Data QualitySenior-level Full TimeRemote, India R6d ago
-
Sr. Data Engineer INR 2500K-3380KAWS | Azure | C# | Data Governance | Data ModelingHybrid work environmentSenior-level Full TimeIndia - Remote R6d ago
-
Senior Staff Engineer, ERP Oracle Technical (ODI & EBS) INR 300K-370KAnalytics Cloud | Autonomous Data Warehouse | Data Integrator | Data Modeling | Data WarehouseSenior-level Full TimeRemote, India R6d ago
-
Associate Principal Engineer, ERP Oracle Technical (ODI & EBS) INR 1500K-2000KAnalytics Cloud | Autonomous Data Warehouse | Data Integrator | Data Modeling | Data WarehouseMid-level Full TimeRemote, India R6d ago
-
Associate Principal Engineer, ERP Oracle Technical (ODI & EBS) INR 1500K-2000KAnalytics Cloud | Autonomous Data Warehouse | Data Integrator | Data Modeling | Data WarehouseMid-level Full TimeRemote, India R6d ago
-
Bash | C# | C++ | Docker | GoFlexible schedule | Global community | Project-based collaboration | Remote workSenior-level Full TimeIndia - Remote R6d ago