Databricks Platform Engineer
Tasks
- Administer Databricks environments
- Build Spark Structured Streaming pipelines
- Enforce cost controls using cluster policies
- Enforce security controls using cluster policies
- Implement CI/CD pipelines
- Ingest data from Kafka
- Manage Databricks Jobs and Workflows
- Manage Unity Catalog data governance
- Monitor platform health with Azure Monitor
- Monitor platform health with Dynatrace
- Operate Databricks clusters
- Provision infrastructure with Terraform
- Troubleshoot performance issues
- Troubleshoot scaling issues
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Azure Monitor | CI/CD | Cluster policies | Databricks | Dynatrace | Git | Kafka | Python | SQL | Spark Structured Streaming | Structured Streaming | Terraform | Unity Catalog
Education
Related jobs
-
Mid-level Full TimeKarnataka, Bengaluru, India2h ago
-
Junior Software Engineer INR 480K-800KAzure Data | Azure Data Factory | Azure Data Lake | Azure Synapse | Azure Synapse AnalyticsClient facing communication experience | Night shift rotation | Remote workEntry-level Full TimeHyderabad/Remote, India R2h ago
-
Software Engineer - Data Engineer INR 2475K-3465KAWS | Azure | CI/CD | Computer Vision | Data GovernanceCollaborative work environment | Professional development | Training opportunitiesSenior-level Full TimeBangalore - Karnataka, India - Kalyani …2h ago
-
Associate Engineer - AI/ML with Python INR 2500K-4000KAWS | Celery | Django | Docker | ElasticsearchCollaborative work environment | In-office 5 days a week | Learning and development opportunities | Professional growth and developmentMid-level Full TimeBangalore - Karnataka, India - Kalyani …2h ago
-
Senior-level Full TimeIN-TN-Chennai3h ago
-
Databricks Data Engineer INR 1500K-2200KApache Kafka | Apache Spark | Azure | Azure Data | Azure Data LakeMid-level Full TimeIN-KA-Bangalore3h ago
-
Agentic AI Engineer INR 2000K-5000KAI LLM Gateway | AI/LLM | Azure | Azure Cosmos | Azure Cosmos DBSenior-level Full TimeIN-KA-Bangalore3h ago
-
Mid-level Full TimeIN-KA-Bangalore3h ago
-
Data Engineer INR 1500K-2500KApache Airflow | BigQuery | Cloud Composer | Cloud Dataflow | Cloud FunctionsMid-level Full TimeIndia-Telangana-Hyderabad3h ago
-
Quality Data Analytics Specialist INR 1500K-2700KAlation | Data Management | Data Quality | Data Reconciliation | Data ValidationSenior-level Full TimePune, India5h ago
-
Assistant Manager - Analytics\..RBG - Analytics INR 1200K-2500KAdobe Analytics | Churn Prediction | Classification | Clickstream | ClusteringMid-level Full TimeBengaluru, Karnataka, India6h ago
-
Assistant Manager - Analytics\..RBG - Analytics INR 1200K-2500KAdobe Analytics | Churn Prediction | Classification | Clickstream analytics | ClusteringMid-level Full TimeBengaluru, Karnataka, India6h ago
-
Mid-level Full TimeBangalore, Karnataka, India6h ago
-
Software Engineer II - Big Data (ETL) / DevOps Engineer + AWS INR 1500K-2500KAWS | Agent Frameworks | Apache Spark | Athena | AuroraMid-level Full TimeBengaluru, Karnataka, India6h ago
-
Senior Software Engineer-Data INR 2000K-4580KAI Foundry | Agent Orchestration | Apache Spark | Azure AI | Azure AI FoundrySenior-level Full TimeIndia, Telangana, Hyderabad7h ago
-
Analyst-Data Science INR 2771K-4064KActive Learning | Algorithms | Apache Hive | Apache Spark | Bayesian MethodsEntry-level Full TimeGurugram, HR, India8h ago
-
Data Modeller INR 1500K-2500KData Governance | Data Modeling | Data Warehousing | Database Design | ETLSenior-level Full TimePune, Maharashtra, India9h ago
-
Applied AI ML Lead-AI Engineer INR 3500K-5000KAgentic AI | CI/CD | Cloud Computing | Code review | ContainerizationSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
GCP Data Engineer INR 1500K-2000KAnsible | Cloud Bigtable | Cloud SQL | Cloud Spanner | Cloud platformProfessional growthMid-level Full TimeBengaluru, Pune10h ago
-
Applied AI ML Lead-AI Engineer INR 3500K-5000KAgentic AI | CI/CD | Cloud Native | Cloud-native development | Code reviewSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Software Engineer III - Java & Databricks INR 2200K-3500KAgile | Amazon Web Services | Angular | Application Resiliency | C#Senior-level Full TimeHyderabad, Telangana, India11h ago
-
AI Specialist (Technical) INR 1860K-3465KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksMid-level Full TimeGurugram, HR, IN11h ago
-
Senior Robotics Engineer INR 1753K-2372KC++ | Java | Localization and mapping | MATLAB | Machine LearningFull Time Office Project Work | Relocation not provided | Travel for deployment and commissioningSenior-level Full TimeChennai, TN, IN11h ago
-
AI Specialist (ML & LLM) INR 2400K-4000KAI Search | Azure AI | Azure AI Search | Azure Functions | Azure Machine LearningMid-level Full TimeGurugram, HR, IN11h ago
-
AI Deployment Engineer INR 2000K-4800KCloud Architecture | Enterprise Architecture | Generative AI | JavaScript | Machine LearningHybrid work model | Relocation assistanceSenior-level Full TimeDelhi, India12h ago