Senior Databricks Platform Engineer
Tasks
- Apply data governance and metadata management
- Build and configure workspaces clusters notebooks jobs
- Build platform observability
- Collaborate with cloud infrastructure teams
- Configure metastore and catalog hierarchies
- Configure storage integrations
- Connect source systems to Databricks
- Develop user provisioning processes
- Enforce compute policies and cluster configurations
- Evaluate new AWS and Databricks features
- Implement RBAC and security groups
- Implement data lineage and auditing
- Implement encryption and data masking
- Implement secrets management
- Integrate AWS services with Databricks
- Integrate identity management with Databricks
- Maintain Databricks enterprise platform
- Maintain technical documentation
- Manage Unity Catalog governance
- Manage serverless SQL warehouses
- Monitor and manage workload reliability
- Monitor and tune cluster utilization
- Optimize costs and resource allocation
- Troubleshoot platform authentication and connectivity issues
- Validate security controls across data lifecycle
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Active Directory | Apache Hive | Apache Spark | Audit Logging | Autoscaling | Azure | Bash | Cluster configurations | Compute policies | Data Factory | Data Lineage | Data masking | Databricks | Delta Lake | EMR | Glue | Hadoop | IAM | KMS | KPI Dashboards | Monitoring | Powershell | Python | RBAC | S3 | SAML | SCIM | SQL | Scala | Secrets management | Serverless SQL | Unity Catalog | Workload management
Education
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R25d ago
-
Senior Data Engineer-JT0224 USD 115K-175K.Net Core | .Net Framework | AWS CloudFormation | Apache Airflow | Azure401k match | Career growth opportunities | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R4h ago
-
Senior Machine Learning Engineer USD 140K-224KAIOps | AWS | Apache Spark | Cloud platform | Data Analysis401k match | Dental insurance | Flexible paid time off | Life insurance | Medical insuranceSenior-level Full TimeRemote, United States R4h ago
-
Lead Business Intelligence Engineer USD 100K-150KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeWestchester, IL, United States4h ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL5h ago
-
Senior-level ContractRaleigh, United States6h ago
-
AI Agents | AI Governance | Artificial Intelligence | Cybersecurity compliance | Enterprise SearchHybrid work | Local residency requirementEntry-level ContractAustin, United States6h ago
-
SYSTEM ENGINEER - Computer Network Support - AI/ML - 6+ yrs of Experience - TS/SCI w/Poly clearance is required - ES - 032726-2 A USD 136K-140KAI/ML | Agile | Confluence | Jira | LLM401k retirement plan | Dental insurance | Disability insurance | Federal Holidays | Floating holidaysMid-level Full TimeFort George G Meade, United States6h ago
-
AI Platform Engineer USD 108K-215KAutogen | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Key VaultMid-level Full TimeBelmont, NC, United States7h ago
-
AI Platform Engineer USD 108K-215KAI Agents | Agent Frameworks | Autogen | Azure Data | Azure Data LakeMid-level Full TimeBelmont, NC, United States7h ago
-
AI experimentation | Alerting | Artificial Intelligence | Data Annotation | Data ArchitectureSenior-level Full TimeNew York7h ago
-
Project GenAI Engineer III USD 113K-188KAWS | Bedrock | Generative AI | Machine LearningMentorship | Onsite work | Professional development | Travel up to 25%Senior-level Full TimeArlington/Rosslyn, Virginia, United States7h ago
-
AWS | Artificial Intelligence | Cloud platform | Data Architecture | Data GovernanceTravel 10 to 15 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Washington, District …7h ago
-
Project Lead Data Engineer III USD 119K-198KBTEQ | Database Administration | Patch Management | Performance Tuning | Query OptimizationSenior-level Full TimeSacramento, California, United States7h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R9h ago
-
Software Engineer, Systems ML Engineering USD 170K-251KAlerting | Benchmarking | C++ | CUDA | DashboardingSenior-level Full TimeSunnyvale, CA | Bellevue, WA | …9h ago
-
AI/HPC Network Performance Engineer USD 147K-226KAI Training | Alerting | Auto-remediation | C++ | Configuration ManagementOncall rotationSenior-level Full TimeMenlo Park, CA9h ago
-
Forward Deployed Engineer III, Google Cloud, Applied AI USD 174K-253KAPI Integration | Agent systems | Agentic Workflows | Chatbots | Cloud platformBenefits | Bonus | Equity | Travel up to 50 percentSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …9h ago
-
Research Software Engineer, Multimodal AI USD 174K-253KAgent Orchestration | Audio | C++ | Few-Shot Learning | Few-shotMid-level Full TimeSan Jose, CA, USA9h ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA9h ago
-
Customer Engineer, Data/Analytics, Google Public Sector USD 152K-222KApache Spark | Batch Processing | Big Data | Cloud Architecture | Data GovernanceBonus target | Equity | Health benefits | Paid time off | Retirement benefitsSenior-level Full TimeReston, VA, USA; Atlanta, GA, USA9h ago
-
Senior Software Engineer, AI/ML, Ads Training USD 174K-253KC++ | Distributed Systems | GPU | JAX | Machine LearningSenior-level Full TimeMountain View, CA, USA9h ago
-
Software Engineer, Product USD 208K-269KAgentic Frameworks | Anthropic | Claude Code | Cursor | DocumentationDental insurance | Health insurance | Paid time off | Retirement plan | Vision insuranceMid-level Full TimeSan Francisco, CA14h ago
-
C++ | Containers | Distributed Systems | Distributed Training | Distributed inferenceSenior-level Full TimeCupertino15h ago