Databricks SME (Subject Matter Expert)
Tasks
- Build ETL/ELT workflows
- Design Databricks data platforms
- Ensure security compliance coding standards
- Implement Delta Lake
- Implement data governance
- Implement data quality frameworks
- Integrate Apache Spark with Databricks
- Mentor Databricks Spark teams
- Migrate legacy data platforms to Databricks
- Optimize Databricks pipelines
- Optimize data processing cost efficiency
- Optimize data processing performance
- Provide Databricks architecture guidance
- Troubleshoot production issues
- Tune Databricks performance
Perks/Benefits
Skills/Tech-stack
Amazon Web Services | Apache Spark | Azure | CI/CD | Cloud platform | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | DevOps | Google Cloud | Google Cloud Platform | Kafka | Lakehouse Architecture | PySpark | Python | SQL | Scala | Structured Streaming | Unity Catalog | Web Services
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R13d ago
-
Contratação de Pessoa Jurídica para Desenvolvimento de Sistema de Coleta de Dados - Vaga Nº 107/2026 BRL 20K-31KAccess Control | Android | Audio capture | Audit Logging | AuthenticationSenior-level ContractHíbrido (Manaus, AM, Brasil) R5h ago
-
Azure | Azure Cloud | Azure Data | Azure Data Factory | Azure DatabricksRemote workMid-level ContractRemote R6h ago
-
Senior Data Engineer – Data Quality & Observability |1789 MXN 911K-1300KAlerting | Amazon Aurora | Amazon Aurora PostgreSQL | Amazon Redshift | Aurora PostgreSQLCross-functional collaboration | Fully remote | Opportunity for ownershipSenior-level Full TimeChihuahua, Mexico R7h ago
-
Data Engineer (Snowflake & DBT) INR 1000K-2000KAgile | Airbyte | CI/CD | Change Data Capture | DBTRemote workMid-level Full TimeRemote R7h ago
-
Senior AI Engineer (m/f/d) EUR 36K-45K.NET | AI Search | AIOps | API Integration | AngularChildcare support | Coursera access | Experience days | Flexible hours | Free specialist access for health wellbeingSenior-level Full TimeGranada, AN, Spain R9h ago
-
GCP, Airflow/CONSULTANT SPECIALIST INR 1500K-2500KAccess Control | Apache Airflow | Apache NiFi | Audit Logging | BigQueryFlexible working | Opportunities for professional developmentSenior-level Full TimePune, Maharashtra, India R10h ago
-
AWS SageMaker | Airflow | Amazon EC2 | Amazon EKS | Computer VisionContinuous learning and development | Flexible schedule | Fully remote | Mentorship opportunitiesMid-level Full TimeIndia R15h ago
-
Anthropic | Benchmarking | CI/CD | Docker | FastAPIAutonomy | Fully remote | PJ contract structureSenior-level Full TimeBrazil R16h ago
-
API documentation | Airflow | Amazon Web Services | Apache Beam | Apache SparkBirthday day off | Dental insurance | Educational support programs | Fitness and wellness partnerships | Fully remote workMid-level Full TimeBrazil R16h ago
-
Senior Machine Learning Engineer AUD 148K-210KAccess Control | Agent systems | CI/CD | Capabilities Based Access Control | Error RecoveryHealth and wellness benefits | Hybrid work schedule | Remote work flexibilitySenior-level Full TimeAustralia - Melbourne - Hybrid R21h ago
-
Staff Machine Learning Engineer AUD 140K-185KAWS | Amazon ECS | Amazon ElastiCache | Amazon S3 | Audit LoggingSenior-level Full TimeAustralia - Melbourne - Hybrid R21h ago
-
Senior Machine Learning Engineer AUD 148K-210KACID | Distributed Systems | Durable Execution | Event Sourcing | Fault ToleranceHybrid work | Remote work flexibilitySenior-level Full TimeAustralia - Melbourne - Hybrid R21h ago
-
API Development | Autogen | C# | C++ | Cloud PlatformsLong-term engagement | Paid Holidays | Remote work | Training and professional growthMid-level Full TimeIndia Gate, New Delhi, India - … R21h ago
-
Software Engineer AI/ML USD 112K-150KA/B | A/B Testing | AWS | Anomaly Detection | Automated testingDental insurance | Employee assistance program | Health and wellness programs | Health insurance | Retirement benefitsMid-level Full TimeEvendale, United States R21h ago
-
Azure MLOps | Azure Machine Learning | Azure Synapse | Data Deduplication | Data GovernanceSenior-level ContractRemote R21h ago
-
Principal Data Engineer INR 3740K-5876KAirflow | Amazon Athena | Amazon EMR | Amazon Redshift | Apache FlinkHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R21h ago
-
Senior Data Engineer INR 3715K-5449KAirflow | Batch Processing | Bitbucket | CICD | Data LakesHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R21h ago
-
Senior Data Engineer INR 3715K-5449KAirflow | Batch Processing | Bitbucket | CICD | Data LakeHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R21h ago
-
Senior Data Engineering Manager INR 3600K-5400KAgile | Amazon Web Services | Apache Spark | Data Architecture | Data ModelingHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R21h ago
-
Director, Data Architecture/Engineering INR 3584K-5000KAutomated testing | CI/CD | Data Preprocessing | Docker | EmbeddingsFlexible time off | Flexible-hybrid work | Healthcare | Retirement plan | Tuition reimbursementSenior-level Full TimeM11 - Commers III, 43, 44 … R21h ago
-
Databricks Data Engineer - Bengaluru INR 1500K-2500KAWS Glue | Alation | Amazon Athena | Amazon EMR | Amazon S3Career growth opportunities | Flexible schedule | Work from anywhereMid-level Full TimeIndia-Bangalore-Office-Kalyani Vista R21h ago
-
Azure Bicep | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactoryCareer development | Inclusive culture | Technical trainingSenior-level Full TimeIND-Hyderabad-Hybrid, India R21h ago
-
Senior Backend Engineer, AI Engineering: Chat GBP 91K-115KGraphQL | Langchain | Langgraph | Performance Tuning | PostgreSQLEquity compensation | Flexible paid time off | Growth and development fund | Parental leave | Remote workSenior-level Full TimeRemote, United Kingdom R21h ago
-
AI workflows | API Integration | Big Data | Big Data Technologies | Cloud ComputingFlexible work environment | Hybrid work | Professional development opportunities | Supportive leadershipSenior-level Full TimeMumbai - Hiranandani, India R21h ago