Manager
Tasks
- Build and optimize ETL/ELT workflows
- Collaborate with data scientists and analysts on data requirements
- Design and maintain data pipelines
- Develop automation scripts for pipeline execution and monitoring
- Develop data models and transformations
- Ensure data quality, documentation, and governance
- Monitor troubleshoot and optimize data pipelines
- Orchestrate data workflows with Apache Airflow
- Process large datasets with distributed computing
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Data Modeling | Data Pipeline Architecture | Data Processing | Data pipeline | Databricks | Distributed data | Distributed data processing | ELT | ETL | Pipeline architecture | PySpark | Python | SQL | Shell Scripting | Snowflake
Education
Related jobs
-
Software Developer / Software Engineer INR 960K-1935KAutomation | DBMS | Database | Excel | File handlingMid-level Full TimeTN, Chennai, India4h ago
-
Mid-level Full TimeHaryana, Gurugram, India1d ago
-
IT Automation and AI Engineer INR 1100K-1250KAsynchronous programming | CI/CD | Distributed Systems | Docker | Function CallingMid-level Full TimeBengaluru, India1d ago
-
GCP Cloud Database Specialist INR 1000K-1500KBackup and Restore | Cloud Databases | Cloud platform | Data Reconciliation | Data ValidationMid-level Full TimeIN-AP-Hyderabad1d ago
-
Senior Data Engineer INR 1500K-2040KAgile | Airflow | Amazon Web Services | Apache Kafka | Cause analysisSenior-level Full TimeBangalore, Karnataka, India1d ago
-
Data Engineer II INR 1500K-2000KApache Spark | Azure Data | Azure Data Explorer | Azure Data Factory | Azure SynapseMid-level Full TimeHyderabad, TS, IN1d ago
-
Databricks Engineer/ Architect INR 1500K-2000KAWS | Apache Spark | Azure | Cloud platform | Data GovernanceSenior-level Contract Full TimeIndia - Remote R1d ago
-
Software Engineer - DevOps INR 1400K-2000KAWS | Apache Kafka | Apache Tomcat | Auto Scaling | AzureMid-level Full TimeMumbai, Maharashtra, India1d ago
-
Principal Engineer - Analytics INR 2000K-3500KAgile | Cassandra | Continuous integration | Data Visualization | HadoopSenior-level Full TimeMumbai, Maharashtra, India1d ago
-
Entry-level Full TimeMumbai, Maharashtra, India1d ago
-
Associate Vice President - Product Engineering - DevOps/DBA INR 1000K-1500KAWS | Apache Kafka | Apache Tomcat | Auto Scaling | AzureMid-level Full TimeMumbai, Maharashtra, India1d ago
-
AWS | Agent systems | Agentic AI | Amazon SageMaker | Autonomous AgentsSenior-level Full TimeBengaluru Millenia, India1d ago
-
AWS SageMaker | Agent systems | Agentic AI | Autonomous Agents | Azure Machine LearningSenior-level Full TimeBengaluru Millenia, India1d ago
-
Agent systems | Amazon SageMaker | Autonomous Agents | Azure ML | Computer VisionFlexible work arrangements | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
Agent systems | Agentic AI | Amazon SageMaker | Autonomous Agents | Azure Machine LearningFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
Azure Databricks | CI/CD | Data Pipelines | Data integration | ETLSenior-level Full TimeGurugram 10 C, India1d ago
-
Azure Databricks | Data integration | ETL | Filtering | JoiningSenior-level Full TimeGurugram 10 C, India1d ago
-
IN_Associate_Python/Gen AI_AI in Delivery_Advisory_Bangalore INR 3000K-4000KAWS | Azure | CI/CD | Data Augmentation | Data PreprocessingFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
IN_Associate_Python/Gen AI_AI in Delivery_Advisory_Bangalore INR 3000K-4000KAWS | Adversarial Networks | Autoregressive models | CI/CD | Cloud platformFlexible work arrangements | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
IN_Associate_Python/Gen AI_AI in Delivery_Advisory_Bangalore INR 3000K-4000KAWS | Adversarial Networks | Autoregressive models | Azure | CI/CDFlexibility program | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
Module Lead - GCP Data Engineer Job INR 1650K-3200KBigQuery | Bigtable | CI/CD | Cloud Build | Cloud NetworkingContinuous learning | Flexible work arrangements | Stable employmentSenior-level Full TimeHyderabad, TG, IN1d ago
-
Senior Principal Engineer - Data Engineering INR 3000K-4000KAWS | Apache Spark | Automation | Azure Data | Azure Data FactoryHybrid work environment | Inclusive culture | Professional development opportunities | Supportive leadersSenior-level Full TimePune - Panchshil, India R1d ago
-
AI machine learning | Azure | Cloud platform | Compliance | Data ArchitectureExecutive-level Full TimeKolkata DN 57, India1d ago
-
Senior Data Engineer INR 3000K-4000KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Machine LearningSenior-level Full TimeIT Centre Bengaluru GDTC, India1d ago
-
API Development | Async Programming | Autogen | CI/CD | CrewAISenior-level Full TimeBengaluru Millenia, India1d ago