Sr. Data Engineer
Gurgaon, Haryana, India
INR 2800K-4000K (estimate) Senior-level Full Time
Tasks
- Build batch data processing workflows
- Build near real time data processing workflows
- Collaborate with data architects
- Collaborate with downstream consumers
- Collaborate with platform teams
- Design data pipelines
- Develop data pipelines
- Enrich data
- Ensure data governance compliance
- Ensure data security compliance
- Implement data transformations
- Implement error handling
- Implement logging
- Implement monitoring
- Integrate semi structured data sources
- Integrate structured data sources
- Maintain data pipelines
- Optimize Spark jobs
- Provide L2 support
- Provide L3 support
- Support production deployments
- Validate data
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Batch Processing | CI/CD | Cloud Composer | Compliance | Data Enrichment | Data Governance | Data Security | Data Transformation | Data Validation | Data Warehousing | Docker | Error Handling | GCP | Kafka | Kubernetes | Logging | Monitoring | Near-real-time | Near-real-time Processing | PySpark | Python | Real-time Processing | Time processing
Education
N/A
Roles
Related jobs
-
Software Engineer III - Python INR 3000K-5000KAlerting | CI/CD | Docker | Infrastructure as Code | KubernetesSenior-level Full TimeBengaluru, Karnataka, India3h ago
-
Senior-level Full TimeBANGALORE, Karnataka, Inde3h ago
-
Agentic AI | Agile | Artificial Intelligence | Docker | Generative AISenior-level ContractShaikpet, India3h ago
-
Senior AI Engineer / Data Scientist (Agentic AI) INR 2000K-5000KAI Search | AI Services | Agentic AI | Azure | Azure AISenior-level Full TimeChennai, India4h ago
-
Senior-level Full TimeIN-KA-Bangalore4h ago
-
GenAI Developer INR 1800K-4500KAPI Development | AWS | Azure | C++ | Cloud platformFull-time work | Work from OfficeSenior-level Full TimeIN-TN-Chennai4h ago
-
Expert Data Engineer INR 3500K-4200KAPI Integration | Access Control | Attribute-Based Access Control | Attribute-based access | Audit LoggingHybrid workSenior-level Full TimeBangalore, Karnataka, India4h ago
-
Mid-level Full TimeNoida, Uttar Pradesh, India5h ago
-
Amazon S3 | Apache Iceberg | Cloud Computing | Concurrency | Data ModelingCareer development | Health benefits | Paid time off | Remote work flexibilitySenior-level Full TimeHyderabad, India5h ago
-
Agile | Apache Airflow | Azure Data | Azure Data Factory | Azure DevOpsEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India5h ago
-
Application Engineer, Master Data Governance INR 1000K-1500KABAP | Data Governance | Data Management | Fiori | Master DataMid-level Full TimeHyderabad, Telangana, India5h ago
-
Staff Software Engineer, Machine Learning INR 3000K-4680KC++ | Cross-Functional Collaboration | Cross-functional | Data Pipelines | Fine TuningHealth and wellness programs | Time awaySenior-level Full TimeBengaluru, KA, India6h ago
-
Mid-level Full TimeBangalore, Karnataka, IN6h ago
-
Senior AI Engineer INR 2500K-5000KAPI Design | Agile | Azure | Azure AI | Azure ArcHybrid work | On-call supportSenior-level Full TimeBangalore, Karnataka, IN6h ago
-
Senior-level Full TimeHyderabad, Telangana, India7h ago
-
Principal Software Engineer INR 2000K-4800KAlgorithms | Auditability | Automation | Azure | Cause analysisSenior-level Full TimeIndia, Karnataka, Bangalore7h ago
-
Data Engineer - Associate Director INR 1500K-2600KAWS | AWS Glue | Agile | Apache Airflow | Apache HadoopMid-level Full TimeBangalore, Karnataka, India7h ago
-
Software Engineer III - ( Java , Python) INR 2000K-3500KAI Assisted Development | AWS | Agile | CI/CD | JavaSenior-level Full TimeHyderabad, Telangana, India8h ago
-
Entry-level Full TimeHyderabad9h ago
-
Software Engineer II - Java, UI, Spark, Kafka INR 1600K-2800KAngular | Apache Airflow | Apache Hive | Apache Spark | Automated testingMid-level Full TimeMumbai, Maharashtra, India9h ago
-
Mid-level Full TimeHyderabad, TS, India9h ago
-
Senior Data Scientist INR 2000K-5000KAWS | Artificial neural networks | Azure | Chunking | Data StructuresSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Senior Engineer, Agentic AI & MLOps Engineering (5-8 years) INR 3700K-5000KAPIs | Agent Orchestration | Agent systems | Deep learning | LLMOpsSenior-level Full TimeBengaluru, KA, India9h ago
-
Databricks Engineer INR 1500K-2700KAuto Loader | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsSenior-level Full TimeChennai, Tamil Nadu, India9h ago
-
AIOps and MLOps Engineer (Reliability, Monitoring & Governance) INR 3242K-4580KAIOps | Artificial Intelligence | Automation | Data Governance | Incident ManagementSenior-level Full TimeIndia11h ago