Databricks Lead – Delivery & Engineering
Tasks
- Build and optimize Spark and Delta Lake pipelines
- Build scalable ETL/ELT pipelines
- Design and implement Databricks Lakehouse platforms
- Develop Delta Lake data modeling bronze silver gold
- Develop data ingestion batch and streaming
- Ensure reliability scalability and observability of data pipelines
- Implement CI/CD
- Implement Databricks Jobs and Workflows
- Lead Databricks delivery
- Lead data engineering teams
- Optimize Spark workloads for performance and cost
- Optimize performance and DBU usage
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | CI/CD | Cloud Native | Cloud-native data engineering | Data Engineering | Data Lineage | Data Modeling | Data Quality | Databricks | Delta Lake | ELT | ETL | Lakehouse | Performance Tuning | PySpark | Spark SQL | Troubleshooting
Education
N/A
Roles
Big Data Engineer | Data Engineer | Databricks Lead | Engineer | Lead
Related jobs
-
Senior Data Engineer (Databricks) (Remote) INR 1500K-2040KAI | API Keys | AWS Glue | AWS IAM | AWS LambdaCollaboration | Ownership culture | Remote workSenior-level Full TimeMaharashtra, Pune, India R2h ago
-
AI / ML Engineer (Remote) INR 2500K-4500KAWS | Amazon SageMaker | CI/CD | Computer Vision | Data PreprocessingRemote workSenior-level Full TimeMaharashtra, Pune, India R2h ago
-
Fullstack Data Engineer (India) INR 1500K-2300KAPI Keys | AWS Glue | AWS IAM | AWS Lambda | AWS S3Remote work | UK shift hoursSenior-level Full TimeMaharashtra, Pune, India2h ago
-
Principal Consultant, Cloud Data Engineer INR 2800K-4000KApache Spark | Azure | Big Data | CI/CD | Data ModelingAI certifications | Mentorship | World-class trainingSenior-level Full TimeIndia-Bangalore3h ago
-
Data Engineer INR 1500K-2000KAgentic AI | Cloud Computing | Cost Optimization | Data Architecture | Data PipelinesMid-level Full TimeIN-MH-Pune3h ago
-
Big Data -Snowflake INR 1200K-2000KBig Data | Data Engineering | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeIN-TN-Chennai3h ago
-
Senior-level Full TimePune, Maharashtra, India4h ago
-
Senior-level Full TimePune, India5h ago
-
Analytics Solutions Manager INR 1234K-1748KAccess Control | Anomaly Detection | Artificial Intelligence | CDC | CI/CDMid-level Full TimeBengaluru, Karnataka, India6h ago
-
Software Engineer II INR 1400K-1700KAPI Development | Apache Spark | Azure | Azure Cosmos | Azure Cosmos DBMid-level Full TimeBengaluru, KA, IN6h ago
-
Associate - Data Scientist INR 1800K-2500KAWS | Agent systems | Apache Spark | Azure | Cloud platformMid-level Full TimeNoida, Uttar Pradesh, India6h ago
-
Senior Data Engineer INR 2800K-3500KAPI Development | AWS | Algorithms | Amazon Kinesis | Amazon RedshiftCharity programs | Discounted gym memberships | Environmental initiatives | Flexible working | Life assuranceSenior-level Full TimeHyderabad, India7h ago
-
Lead Assistant Manager INR 1500K-2000KAWS | Apache Airflow | Azure | Data Governance | Data LakehouseSenior-level Full TimeIndia7h ago
-
Lead Data Visulaization INR 2000K-3900KAirflow | Amazon Redshift | Azure Data | Azure Data Factory | Azure SynapseSenior-level Full TimeTamil Nadu, Chennai, India7h ago
-
Data Engineer INR 1500K-2345KAWS | Amazon Kinesis | Amazon Redshift | Azure | BashDiscounted gym memberships | Flexible working | Life assurance | Medical insurance | Pension schemeMid-level Full TimeHyderabad, India7h ago
-
Applied AI/ML Associate Senior - Causal ML INR 1400K-2000KA/B | A/B Testing | Apache Hive | Apache Spark | B testingMid-level Full TimeBengaluru, Karnataka, India8h ago
-
Mid-level Full TimeMumbai, Maharashtra, India9h ago
-
Senior-level Full TimeBangalore, KA, IN9h ago
-
Senior_AI_developer INR 3125K-4763KAPI Orchestration | CI/CD | CNN | Computer Vision | Data PreprocessingSenior-level Full Timebangalore, India9h ago
-
Senior Data Engineer INR 2520K-3880KAWS CodePipeline | AWS DMS | AWS Glue | AWS Lambda | Amazon KinesisReward and recognition | Volunteering opportunities | Wellness program | Work-life balanceSenior-level Full TimeHyderabad, India9h ago
-
Lead Software Engineer – Python, Terraform, Snowflake, AWS INR 2000K-4000KAWS | Agile | Amazon Web Services | Application Resiliency | By DesignSenior-level Full TimeHyderabad, Telangana, India10h ago
-
Data Engineer INR 3000K-4000KAPI Design | Apache Iceberg | Apache Spark | Cloud Data | Cloud data warehousingSenior-level Full TimeIndia10h ago
-
Software Engineer II - Java Full Stack Developer + AI/ML INR 1400K-2200KAWS | AppDynamics | CI/CD | Curl | DynatraceMid-level Full TimeBengaluru, Karnataka, India11h ago
-
Senior Lead Software Engineer – Agentic SDLC & AI Engineer INR 2500K-4000KAgentic development | Anthropic Claude | Anthropic Claude Code | Artificial Intelligence | Automated testingSenior-level Full TimeMumbai, Maharashtra, India11h ago
-
Data Engineer for AZURE cloud Platform INR 1500K-2000KAPIs | Airflow | Azure | Azure Data | Azure Data FactoryMid-level Full TimeBengaluru, KA, India11h ago