Databricks Platform Expert
Tasks
- Build and maintain production pipelines with Delta Lake and Databricks Jobs
- Collaborate on Azure networking and private link connectivity
- Create automated alerts and observability dashboards
- Debug and performance tune Spark jobs and pipelines
- Develop cost governance policies for clusters and SQL Warehouses
- Ensure enterprise data governance audit and logging compliance
- Implement CI CD pipelines for notebooks jobs and Delta Live Tables
- Implement manage Unity Catalog
- Manage RBAC SCIM provisioning and service principals
- Manage configure and administer Databricks workspaces
- Manage secrets and enforce secure credential handling
- Monitor platform performance cluster stability logs and job reliability
- Optimize cluster policies autoscaling and cost management
- Troubleshoot Databricks runtime issues and library conflicts
- Write efficient Spark code for ETL and data transformations
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | Apache Spark | Azure | Azure Monitor | CI/CD | Databricks | Databricks Jobs | Delta Lake | Delta Live Tables | Delta Live) | Git | IAM | Private Link | PySpark | RBAC | SCIM | SQL | SQL Warehouses | Scala | Secrets management | Service principals | Unity Catalog | VNet
Education
N/A
Related jobs
-
Senior-level Full TimePune City, India9h ago
-
Entry-level Full TimePune, India10h ago
-
Senior AI Engineer/ Senior Associate Director, Tech SME INR 1860K-2800KAPIs | AWS | Agent Orchestration | Agent systems | AzureMid-level Full TimePune, Maharashtra, India R10h ago
-
Data Engineer INR 720K-720KAWS | Apache Airflow | Apache NiFi | Apache Superset | AzureChildcare support | Counseling sessions | Health insurance | Internet allowance | Learning SupportSenior-level Full TimeBangalore, India R11h ago
-
Junior Data Engineer INR 400K-700KAWS Glue | Amazon Redshift | Apache Airflow | Apache Spark | Azure DataEntry-level Full TimePune City, India11h ago
-
Senior AI Engineer INR 2500K-4500KAPI Development | Asynchronous programming | Azure | CI/CD | COMTRADESenior-level Full TimeBangalore - Karnataka - India11h ago
-
Senior-level Full TimeBangalore - Karnataka - India11h ago
-
Senior AI Engineer INR 2500K-4500KAsync Programming | Azure | CI/CD | Data Ingestion | Data Ingestion PipelineDiversity-focused workplace | Equity and inclusion | Learning opportunitiesSenior-level Full TimeBangalore - Karnataka - India11h ago
-
Senior AI Engineer INR 2500K-4500KAPI Development | Asynchronous programming | Azure | CI/CD | Dependency managementSenior-level Full TimeBangalore - Karnataka - India11h ago
-
Senior-level Full TimeBangalore - Karnataka - India11h ago
-
Senior-level Full TimeBangalore - Karnataka - India11h ago
-
Data Engineer ( 2 to 5 Years) INR 1500K-2100KAWS CDK | AWS CloudFormation | AWS Glue | AWS Lambda | Amazon AthenaContinuous learning | Flexible working | Hybrid working | Inclusive cultureMid-level Full TimeBangalore - Karnataka - India11h ago
-
AWS Lambda | Amazon DynamoDB | Amazon RDS | Amazon S3 | CI/CDHybrid workSenior-level ContractHyderabad, India R11h ago
-
Data Engineer INR 1500K-2700KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeHyderabad, Noida/ Chennai, India12h ago
-
Backend Engineer (Cloud & AI Integration) INR 1500K-3000KAPI Integration | AWS | Agile | Asynchronous processing | DocumentationMid-level Contract Full TimeBengaluru, KA, India12h ago
-
Senior-level Full TimePune, Maharashtra, India13h ago
-
Mid-level Full TimePune, Maharashtra, India13h ago
-
Senior-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101313h ago
-
Backend Engineer (Cloud & AI Integration) INR 1500K-3000KAPI Integration | AWS | Agile | Asynchronous processing | DocumentationMid-level Contract Full TimeBengaluru, KA, India14h ago
-
Senior Data Engineer (Python, SQL, Snowflake, Devops, Azure/ AWS, Co-pilot, Kubernetes, Docker) INR 3200K-4131KAWS | Airflow | Automated Lineage | Azure | Azure BicepEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Mentorship | Volunteer time offSenior-level Full TimeChennai, TN, India14h ago
-
Apache Airflow | Apache Beam | Apache Flink | Asynchronous Messaging | BigQueryFlexible working | Opportunities for growth | Professional developmentMid-level Full TimePune, Maharashtra, India R14h ago
-
Senior Data Engineer – Snowflake Developer INR 2800K-4000KAWS | Access Control | Airflow | Azure | Azure DataSenior-level Full TimePune, Maharashtra, India14h ago
-
Senior II Data Engineer INR 3715K-4995KBI Dashboards | BICC | CI/CD | Data Augmentation | Data CatalogFlexible working options | Health benefitsSenior-level Full TimeIndia14h ago
-
Capital Market Workforce Team Member-BPS INR 1000K-2500KApache Airflow | Data Mapping | Data Modeling | Data Quality | Data pipelineEntry-level Full TimeIndia15h ago
-
Senior-level Full Timetelengana, India15h ago