Lead Data Engineer
Tasks
- Build monitoring and self healing automation
- Coordinate Databricks vendor case management and escalations
- Design and develop scalable data pipelines
- Design and operationalize Unity Catalog for governance
- Drive CI CD workflows for Databricks assets
- Enforce best practices for partitioning clustering and workload isolation
- Lead design reviews and mentor data engineers
- Track performance trends and resolve high cost queries
- Tune Databricks workloads for performance and cost
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | CI/CD | DBSQL | Databricks | Databricks SQL | Delta Lake | DevOps | Infrastructure as Code | Observability | PySpark | SQL | Serverless compute | Unity Catalog | Workflow Orchestration | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior-level Full TimeBengaluru, Karnataka, India R5h ago
-
Senior ML Operations Engineer INR 2500K-5000KAI Governance | AWS | Automated retraining | Azure | Azure MLSenior-level Full TimeBengaluru, Karnataka, India8h ago
-
Software Development Specialist INR 940K-940KAmazon CloudWatch | Amazon DynamoDB | Amazon S3 | Amazon Web Services | Azure DevOpsMid-level Full TimeBangalore, KA, IN R8h ago
-
Senior Manager INR 1500K-2800KAWS Glue | AWS Secrets | AWS Secrets Manager | Adaptive Query | Adaptive query executionHybrid work modeSenior-level Full TimePune, Maharashtra, India R9h ago
-
Mid-level Full TimeBengaluru, Pune9h ago
-
Mid-level Full TimeIndia, Telangana, Hyderabad10h ago
-
Senior Data Engineer INR 3000K-4200KAPI Management | AWS | Application Architecture | Azure API | Azure API ManagementSenior-level Full TimeIndia10h ago
-
Data Science Chief Expert - Finance INR 3000K-4000KAWS | Azure | Cloud platform | Cypher | Data PreprocessingExecutive-level Full TimeBangalore, IN, 56006616h ago
-
IN_Manager_Agentic + Gen AI__ AI 50_ Advisory_Hyderabad INR 1000K-2500KAgentic AI | C plus plus | C# | CI/CD | DspyFlexibility programmes | MentorshipMid-level Full TimeHyderabad - Salarpuria, India16h ago
-
Sr Technical Lead-BI INR 1500K-3000KAgile | BI Data modeling | DAX | Data Modeling | Data WarehousingSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …16h ago
-
Senior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …16h ago
-
Technical Lead-BI INR 1500K-2700KDAX | Data Modeling | Data Warehousing | Dimensional Modeling | ETLSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …16h ago
-
Senior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …16h ago
-
IN_Sr Associate_Java Full Stack Engineer_D&A_Advisory_Bangalore INR 1200K-2500KAngular | Apache Kafka | App Service | Azure | Azure AppFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India16h ago
-
EY - GDS Consulting - AIA - AI Engineer- Senior INR 2500K-5000KAI Foundry | AI Observability | AWS Bedrock | Access Controls | Asynchronous programmingCoaching and confidence | Continuous learning | Diverse and inclusive culture | Flexible tools and supportSenior-level Full TimeChennai, TN, IN, 60003216h ago
-
API Integration | Access Control | Agent Orchestration | CI/CD | Change ManagementContinuous professional development | Flexible working | Inclusive workplaceMid-level Full TimeHyderabad, TG, IN, 50003216h ago
-
ESP Security Data Engineer INR 2000K-2700KAccess Controls | Automation | Cloud Platforms | Continuous Improvement | Data ArchitectureMid-level Full TimeBengaluru, KA, IN, 56004816h ago
-
APM | AWS | Ansible | Asynchronous programming | AzureSenior-level Full TimeIND - Chennai, Neville Towers, India16h ago
-
Java API & Microservices Engineer / Architect INR 2500K-3500KAPI Testing | Azure | CI/CD | Cassandra | DatadogSenior-level Full TimeBangalore, KA, IN R16h ago
-
EY - GDS Consulting - AIA - AI Platform Engineer- Senior INR 3000K-5449KAI Search | AI Services | Agent SDK | Autogen | Azure AISenior-level Full TimeBengaluru, KA, IN, 56004816h ago
-
Senior-level Full TimeBangalore, KA, IN R16h ago
-
Cloud AI / Agentic AI Developer - Manager INR 1860K-2500KAPI Gateway | AWS Lambda | Amazon API | Amazon API Gateway | Amazon BedrockEmployee networks | Flexible work/life support | Paid volunteer daysMid-level Full TimeHyderabad, India16h ago
-
AWS | Artifact versioning | Automation | CI/CD | DevOpsMid-level Full TimeBangalore (SDC) - Bagmane Tech Park, …16h ago
-
Data Engineer- Azure Databricks- Senior Associate INR 1500K-2000KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageMid-level Full TimeBangalore (SDC) - Bagmane Tech Park, …16h ago
-
Data Engineer- Azure Databricks- Senior Associate INR 1500K-2000KAnalysis Services | Apache Mahout | Apache Spark | Azure Analysis | Azure Analysis ServicesMid-level Full TimeBangalore (SDC) - Bagmane Tech Park, …16h ago