Databricks Platform Expert
Tasks
- Build and maintain production pipelines with Delta Lake and Databricks Jobs
- Collaborate on Azure networking and private link connectivity
- Create automated alerts and observability dashboards
- Debug and performance tune Spark jobs and pipelines
- Develop cost governance policies for clusters and SQL Warehouses
- Ensure enterprise data governance audit and logging compliance
- Implement CI CD pipelines for notebooks jobs and Delta Live Tables
- Implement manage Unity Catalog
- Manage RBAC SCIM provisioning and service principals
- Manage configure and administer Databricks workspaces
- Manage secrets and enforce secure credential handling
- Monitor platform performance cluster stability logs and job reliability
- Optimize cluster policies autoscaling and cost management
- Troubleshoot Databricks runtime issues and library conflicts
- Write efficient Spark code for ETL and data transformations
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | Apache Spark | Azure | Azure Monitor | CI/CD | Databricks | Databricks Jobs | Delta Lake | Delta Live Tables | Delta Live) | Git | IAM | Private Link | PySpark | RBAC | SCIM | SQL | SQL Warehouses | Scala | Secrets management | Service principals | Unity Catalog | VNet
Education
N/A
Related jobs
-
Sr Software Engineer INR 1500K-2700KAgile | Cloud services | Code review | Data Modeling | Design PatternsSenior-level Full TimePune, India12h ago
-
Software Engineer INR 1500K-2000KAgile | Cloud Computing | Code review | Data Modeling | Design PatternsMid-level Full TimePune, India12h ago
-
Sr Software Engineer INR 1500K-2700KAgile | Cloud services | Code review | Data Model | Data ModelingSenior-level Full TimePune, India12h ago
-
Associate Engineer - Embedded Software INR 1200K-3000KAgile | Assembly | Bare Metal | Black-Duck | C plus plusMid-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101314h ago
-
Senior-level Full TimeKochi, India16h ago
-
ML Developer (with Computer Vision Expertise)- Eurasia INR 2500K-5000KAWS | Adversarial Networks | Algorithms | Azure | Convolutional Neural NetworksSenior-level Full TimePune City, India16h ago
-
Software Developer (AIML) INR 1800K-2700KAWS | Azure | CI/CD | Computer Vision | Convolutional Neural NetworksFlexible working hours | Growth and learning | Referral bonus | Team building events | Training and booksMid-level Full TimePune, India16h ago
-
Data Science - AI ML INR 2500K-5000KAWS | Azure | Bayesian Models | Campaign Optimization | ChatbotsSenior-level Full TimeGurgaon, HR, India17h ago
-
Robotic Process Automation (RPA) Professional INR 2500K-4000K.NET | AI Center | API Integration | Automation Cloud | Azure CopilotSenior-level Full TimePune - Maharashtra - India17h ago
-
Senior-level Full TimePune, Maharashtra, India19h ago
-
AI and Automation Engineer INR 2000K-4800KAI Security | BigQuery | Chain-of-Thought | Claude API | Cloud FunctionsAnnual leave | Casual leave | Public holidays | Sick leaveSenior-level Full TimeGurugram, India19h ago
-
Lead - Data Scientist INR 2500K-4720KAnomaly Detection | Cloud Computing | Data Governance | Deep learning | DevSecOpsCareer development opportunities | Inclusive workplace | Mentoring and team developmentSenior-level Full TimeNoida,Uttar Pradesh,India R19h ago
-
Sr.Data Engineering INR 1500K-2500KAzure | Azure Data | Azure Data Factory | Azure Databricks | Azure Key VaultSenior-level Full Timebengaluru, India19h ago
-
Associate Consultant - Data Engineer(Python/Pyspark) INR 1500K-2200KAgile | Azure | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeBangalore, Karnataka, India20h ago
-
Engineering Lead - Data & Analytics INR 1600K-3000KAWS | Agentic Analytics | Agile | Artificial Intelligence | Automated DeploymentSenior-level Full TimePune, MAHĀRĀSHTRA, India21h ago
-
Java Fullstack Engineer with GenAI & React INR 1500K-2500KAgile Development | Azure | Design Patterns | Docker | Extreme programmingEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India22h ago
-
Data Scientist INR 1800K-3000KApache Spark | Artificial neural networks | Chunking | Data Structures | Deep learningMid-level Full TimeKarnataka, India22h ago
-
Senior-level Full TimeMumbai, Maharashtra, India22h ago
-
Data Engineer III INR 2000K-3000KAI tools | Data Modeling | Data Pipelines | Data analytics | NoSQLSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India23h ago
-
Software Engineer INR 1500K-2100KAWS | Agile | Bash | CI/CD | CloudWatchFamily wellbeing benefits | Health and wellbeing benefits | Hybrid work | Paid time off | Work-life balance benefitsMid-level Full TimeHyderabad, India R1d ago
-
CDP,Snowflake,SQL,Java, API, AWS - (6-9 Yrs) - CHN INR 2000K-3000KAPI Integration | AWS | Audience Management | Customer Data | Customer Data PlatformSenior-level Full TimeIndia1d ago
-
Lead Assistant Manager INR 1500K-3000KAWS | Agile | Angular | Application Monitoring | Artificial IntelligenceMentoring | Team collaboration | Technical leadershipSenior-level Full TimePune, Maharashtra, India1d ago
-
Digital Data Engineer / Plant Specialist Level 2 INR 600K-600KAWS | Data Pipelines | Data Quality | ETL | LinuxInclusion and diversity initiatives | Learning and development | Multicultural environment | Training hoursEntry-level Full TimePUNE R&D,IN-MH,India1d ago
-
Control Path Engineer INR 2000K-3000KAPI Design | Automation | C++ | CI/CD | Configuration ManagementSenior-level Full TimePune Office1d ago