Senior Databricks Data Engineer
Tasks
- Define data modeling standards and ingestion frameworks
- Design reusable Spark pipeline frameworks
- Implement Medallion architecture standards Bronze Silver Gold
- Implement governance lineage security and monitoring with Unity Catalog
- Lead Databricks data pipeline architecture and implementation
- Mentor engineers and conduct architectural reviews
- Modernize SSIS workflows to Spark pipelines
- Optimize Spark workloads for scalability and cost
- Partner with BI analytics and AI ML teams
- Rebuild dimensional models in Delta Lake
- Redesign SSAS cube logic into semantic models
- Replace SSRS reporting with governed analytics layers
- Serve as technical lead for SQL Server warehouse migration to Databricks Lakehouse
Perks/Benefits
Skills/Tech-stack
AWS | CI/CD | Databricks | Delta Lake | Lakehouse | MLflow | Medallion Architecture | Microsoft SQL | Microsoft SQL Server | Power BI | PySpark | SQL | SQL Server | SSAS | SSIS | SSRS | Spark | Unity Catalog
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Databricks Data Engineer - Contract-to-Hire INR 1500K-3000KAWS | Apache Spark | Azure | Data Reconciliation | Data ValidationMedical insurance | Paid time off | Professional development trainingSenior-level Full TimePune, MH, India7h ago
-
Lead AI Specialist (Finance Operations) INR 3264K-5449KAccess Control | Agent Orchestration | Audit Logging | BigQuery | Change ManagementCollaborative culture | Growth opportunities | Work-life balance programsSenior-level Full TimeMumbai, MH, India8h ago
-
Senior-level Full TimeBengaluru, Karnataka, India R8h ago
-
Lead Assistant Manager INR 1500K-2500KApache Airflow | BigQuery | Cloud Composer | Cloud Storage | Cloud platformSenior-level Full TimePune, Maharashtra, India8h ago
-
Senior ML Operations Engineer INR 2500K-5000KAI Governance | AWS | Automated retraining | Azure | Azure MLSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
Software Development Specialist INR 940K-940KAmazon CloudWatch | Amazon DynamoDB | Amazon S3 | Amazon Web Services | Azure DevOpsMid-level Full TimeBangalore, KA, IN R11h ago
-
VP - Sr Lead Data Engineer (Snowflake) INR 4000K-6000KAgile | Data Governance | Data Modeling | Data Pipelines | Data QualityExecutive-level Full TimeMumbai, Maharashtra, India11h ago
-
Senior Manager INR 1500K-2800KAWS Glue | AWS Secrets | AWS Secrets Manager | Adaptive Query | Adaptive query executionHybrid work modeSenior-level Full TimePune, Maharashtra, India R12h ago
-
Senior Consultant - Data Engineering- INR 2000K-3500KApache Kafka | Apache Spark | Cloud Computing | Data Modeling | Data PipelinesSenior-level Full TimeMumbai, Maharashtra, India12h ago
-
Mid-level Full TimeBengaluru, Pune12h ago
-
Mid-level Full TimeIndia, Telangana, Hyderabad13h ago
-
Senior Data Engineer INR 3000K-4200KAPI Management | AWS | Application Architecture | Azure API | Azure API ManagementSenior-level Full TimeIndia13h ago
-
Senior-level Full TimeGurugram, Haryana, India13h ago
-
Senior Workflow Engineer INR 1500K-3000KAzure DevOps | BPMN 2.0 | Collibra Data Intelligence | Collibra Data Intelligence Cloud | Collibra Data QualitySenior-level Full TimeChennai, TN, India15h ago
-
Data Science Chief Expert - Finance INR 3000K-4000KAWS | Azure | Cloud platform | Cypher | Data PreprocessingExecutive-level Full TimeBangalore, IN, 56006619h ago
-
IN_Manager_Agentic + Gen AI__ AI 50_ Advisory_Hyderabad INR 1000K-2500KAgentic AI | C plus plus | C# | CI/CD | DspyFlexibility programmes | MentorshipMid-level Full TimeHyderabad - Salarpuria, India19h ago
-
Sr Technical Lead-BI INR 1500K-3000KAgile | BI Data modeling | DAX | Data Modeling | Data WarehousingSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …19h ago
-
Senior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …19h ago
-
Technical Lead-BI INR 1500K-2700KDAX | Data Modeling | Data Warehousing | Dimensional Modeling | ETLSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …19h ago
-
Senior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …19h ago
-
IN_Sr Associate_Java Full Stack Engineer_D&A_Advisory_Bangalore INR 1200K-2500KAngular | Apache Kafka | App Service | Azure | Azure AppFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India19h ago
-
EY - GDS Consulting - AIA - AI Engineer- Senior INR 2500K-5000KAI Foundry | AI Observability | AWS Bedrock | Access Controls | Asynchronous programmingCoaching and confidence | Continuous learning | Diverse and inclusive culture | Flexible tools and supportSenior-level Full TimeChennai, TN, IN, 60003219h ago
-
API Integration | Access Control | Agent Orchestration | CI/CD | Change ManagementContinuous professional development | Flexible working | Inclusive workplaceMid-level Full TimeHyderabad, TG, IN, 50003219h ago
-
ESP Security Data Engineer INR 2000K-2700KAccess Controls | Automation | Cloud Platforms | Continuous Improvement | Data ArchitectureMid-level Full TimeBengaluru, KA, IN, 56004819h ago
-
APM | AWS | Ansible | Asynchronous programming | AzureSenior-level Full TimeIND - Chennai, Neville Towers, India19h ago