Lead Cloud Data Engineer (Databricks) - Assistant Vice President
Tasks
- Build ETL/ELT pipelines
- Collaborate with architects and stakeholders
- Design data pipelines
- Develop data workflows
- Ensure data consistency
- Ensure data governance
- Ensure data quality
- Ensure data system reliability
- Maintain data pipelines
- Optimize ETL/ELT pipelines
- Optimize data workload cost
- Support production deployments
- Translate requirements into technical implementation
- Troubleshoot data pipeline issues
- Tune performance of data workloads
Perks/Benefits
- Employee networks
- Flexible work/life support
- Inclusive development opportunities
- Paid volunteer days
Skills/Tech-stack
AWS | Apache Spark | Azure | CI/CD | Data Governance | Data Lakehouse | Data Modeling | Data pipeline | Data pipeline design | Databricks | Delta Lake | Docker | ELT | ETL | Kafka | Kubernetes | Performance Tuning | Pipeline Design | Python | SQL | Scala | Scalability
Education
N/A
Roles
Related jobs
-
Data Engineer I INR 1500K-2000KAWS | Apache Spark | Automated testing | Azure | Azure DataHybrid work | Inclusive work environment | Professional developmentMid-level Full TimeBangalore, Karnataka, IN4h ago
-
Data Engineer I INR 1500K-2000KApache Spark | Automated testing | Azure Data | Azure Data Factory | Azure DatabricksCareer development | Learning opportunitiesMid-level Full TimeBangalore, Karnataka, IN4h ago
-
Cloud Data Engineer - AI/ML INR 4000K-5000KAmazon Web Services | Apache Spark | Batch Processing | Cassandra | Cloud platformSenior-level Full TimeBengaluru; Hyderabad4h ago
-
Cloud Data Engineer - AI/ML INR 4000K-5000KAmazon Web Services | Azure | Cassandra | Cloud platform | Dimensional ModelingSenior-level Full TimeBengaluru; Hyderabad4h ago
-
Data Analyst/ETL Engineer INR 1500K-2500KAWS Glue | BI tools | Data Flows | Data Transformations | ETLMid-level Full TimeChennai, India5h ago
-
Senior AI Engineer INR 2500K-5000KAWS | Azure | BM25 | CI/CD | ContainerizationFlexible work environment | Remote work options | Self-care supportSenior-level Full TimeHyderabad, India6h ago
-
AI/ML Engineer INR 2500K-5000KFastAPI | Flask | Language Models | Language Processing | Large Language ModelsSenior-level Full TimeChennai, India6h ago
-
AI Agents | AWS Bedrock | Azure OpenAI | CI/CD | Claude CodeCareer growth | Continuous learning | Hybrid work | Remote work | Work-life balanceSenior-level Full TimeIndia6h ago
-
Senior Software Engineer II - Data Engineering and Platform INR 1500K-2000KAWS | Agile | Apache Airflow | Apache Kafka | Apache SparkMid-level Full TimeBangalore, India6h ago
-
AWS | Azure | Cloud platform | Data Analysis | Data ManagementAutonomy | Collaborative team environment | Continuous learning | Exposure to modern machine learning technologies | Flexible work scheduleMid-level Full TimeIndia R6h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Friendly work environmentMid-level Full TimeGurgaon, India7h ago
-
Bash | Data Pipelines | Data Processing | Docker | GCPAsynchronous culture | Competitive salary | Friendly work environment | Remote-friendlyMid-level Full TimeKolkata, India7h ago
-
Software Engineer II, Data Engineer INR 2520K-3880KAWS | Apache Airflow | Apache Spark | Azure | Cloud platformCareer growth | Learning & MentorshipSenior-level Full TimeBangalore, India7h ago
-
Senior Data Engineer - Supply Chain & Analytics Platforms INR 2800K-4200KAWS | Apache Airflow | Azure | CI/CD | Cloud StorageSenior-level Full TimeBengaluru, INDIA, India7h ago
-
AWS Glue | AWS IAM | AWS Lambda | Airflow | Amazon CloudWatchCollaborate with experienced engineering teams | Continuous learning and professional development | Remote work | Work on impactful business intelligence initiatives | Work on large scale cloud data transformation projectsMid-level Full TimeIndia8h ago
-
AI Foundry | AWS | Azure | Azure AI | Azure AI FoundryMid-level Full TimeBengaluru Millenia, India12h ago
-
ADLS Gen2 | AKS | Adaptive Query | Adaptive query execution | Apache SparkFlexibility programmes | Inclusive benefits | Mentorship | Mentorship programs | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India12h ago
-
IN_Sr Associate_ GenAI and Agentic AI_GCC_ Advisory_ Noida INR 2500K-5000KAPI Development | Agent safety | Async Programming | Autogen | CI/CDSenior-level Full TimeNoida, India12h ago
-
IN_Associate_GenAI and Agentic AI_GCC_Advisory_Bangalore INR 2000K-4500KAI/ML | AI/ML Pipelines | Agent systems | Apache Spark | AutogenEntry-level Full TimeBengaluru Millenia, India12h ago
-
Principal Engineer - Data USD 14KAzure Data | Azure Data Factory | Azure Databricks | Azure DevOps | CI/CDCertificate reimbursement | Health benefits | Medical insurance | Professional development | Shift allowanceSenior-level Full TimeGurugram, HR, IN12h ago
-
Azure Data | Azure Data Factory | Azure Databricks | Azure DevOps | CI/CDCertificate reimbursement | Health benefits | Medical insurance | Professional development | Shift allowanceSenior-level Full TimeGurugram, HR, IN12h ago
-
Subcontractor INR 1500K-2500KAWS Cloud | AWS Cloud Services | Access Controls | Cloud services | Data ModelingSenior-level ContractINDIA - PUNE - BIRLASOFT OFFICE …12h ago
-
AI and Machine Learning Engineer INR 2500K-4580KCalculus | Convolutional Neural Networks | Debugging | Decision Trees | Deep learningCareer Development Programs | Flexible work arrangement | Health & wellbeing benefitsSenior-level Full TimeBengaluru, Karnātaka, India12h ago
-
AI and Machine Learning Engineer INR 2500K-4200KAWS | Azure | CI/CD | Data Analysis | Deep learningHealth benefits | Hybrid work flexibility | Professional development programsMid-level Full TimeBengaluru, Karnātaka, India12h ago
-
AI and Machine Learning Engineer INR 2500K-4580KCalculus | Convolutional Neural Networks | Decision Trees | Deep learning | Generative ModelsHealth and wellbeing benefits | Inclusion and diversity | Professional development programsSenior-level Full TimeBengaluru, Karnātaka, India12h ago