Senior Databricks Platform Engineer
Tasks
- Apply IAM policies and data masking
- Build Databricks workspaces clusters notebooks jobs
- Build platform observability
- Collaborate on data models and reusable transformations
- Configure RBAC and security groups
- Configure metastore and catalog hierarchies
- Configure storage integrations for data access
- Enforce compute policies and cluster configurations
- Implement Unity Catalog governance
- Implement cost optimization frameworks
- Implement data lineage and auditing
- Implement user provisioning and deprovisioning
- Integrate SAML SCIM and Active Directory
- Integrate with AWS services S3 IAM and KMS
- Maintain Databricks platform
- Maintain technical documentation and operational procedures
- Manage cluster pools and serverless SQL warehouses
- Manage multiple workspaces
- Manage secrets and credential handling
- Optimize cluster utilization and performance
- Set up data ingestion and transformation patterns
- Support evaluation of new Databricks features
- Troubleshoot workspace connectivity and authentication issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS IAM | AWS KMS | Active Directory | Amazon S3 | Apache Hive | Apache Spark | Autoscaling | Azure? N/A | Bash | Bitbucket | CI/CD | Catalog Governance | Data Cataloging | Data Governance | Data Lineage | Data masking | Databricks | Delta Lake | Encryption | Git | GitHub | GitLab | Hadoop | Metadata Management | N A | Python | RBAC | SAML | SCIM | SQL | Scala | Secrets management | Unity Catalog | Unity Catalog Governance | Workload management
Education
Related jobs
-
AWS | Automated testing | Big Data | Cloud Native | Code reviewSenior-level Full TimePlano, TX, United States1h ago
-
LLM & Agentic AI Machine Learning Software Engineer, TikTok - Content Ecology Local Services USD 145K-355KAI Agents | Causal Inference | Computer Vision | Conversion Rate | Conversion rate optimizationSenior-level Full TimeSan Jose, California, United States2h ago
-
Data Engineer Intern (E-commerce) - 2027 Summer USD 80K-90KApache Spark | ClickHouse | Data Modeling | ETL | FlinkDevelopment workshops | Industry exposure | Learning programs | Social eventsEntry-level InternshipSan Jose, California, United States2h ago
-
Senior Research Scientist/Software Engineer, LLM/Agent Platform (TikTok-Content Ecology AI Innovation & Platform) USD 160K-230KCUDA | Deep learning | DeepSpeed ZeRO | Distributed Systems | Evaluation FrameworksSenior-level Full TimeSan Jose, California, United States2h ago
-
Senior data engineer (Python) con inglés USD 57K-73KAgile | Alerting | Azure | CI/CD | Clean CodeCertifications | Online training platformSenior-level Full TimeMadrid, MD, Spain2h ago
-
Software Engineer III, AI/ML Engineer USD 147K-210KAgentic Workflows | Alerting | Artificial Intelligence | Backend Development | Backend ServicesSenior-level Full TimeAtlanta, GA, USA; San Jose, CA, …3h ago
-
Data Engineer, Google Maps USD 106K-150KApache Spark | ArcGIS | Business Intelligence | Cloud platform | Data AnalysisMid-level Full TimeMountain View, CA, USA3h ago
-
Senior Software Engineer, AI/ML GenAI, YouTube USD 174K-252KC++ | Computer Vision | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA3h ago
-
Tax Data Systems and Process Analyst USD 116K-166KAPI Integration | Data Warehouse | GRPC | Java | JavaScriptMid-level Full TimeChicago, IL, USA3h ago
-
Machine Learning Co-Op (Fall 2026) USD 87K-124KDAX | Data Quality | Data Visualization | Deep learning | ForecastingNone Full TimeCanton, OH, United States3h ago
-
Sr. Systems Engineer - AI USD 98K-139KAI/ML | Alerting | Auto Scaling | Automation | Backup TechnologiesSenior-level Full TimeKansas City, KS, United States3h ago
-
Sr. Data Engineer - AI USD 99K-139KApache Airflow | Azure Data | Azure Data Factory | CI/CD | Data FactorySenior-level Full TimeKansas City, KS, United States3h ago
-
Bare Metal | Battery Management | Battery Management System | C# | C++401k match | Employee assistance program | Life insurance | Long-term disability | Paid time offSenior-level Full TimeRochester, NY, United States4h ago
-
Embedded Software Engineer - Energy Systems USD 73K-94KARM | Bare Metal | C plus plus | C# | CAN401k match | Dental insurance | Employee assistance program | Life insurance | Long-term disabilityEntry-level Full TimeRochester, NY, United States4h ago
-
Data Platform Engineer-Solutions Delivery USD 119K-160KAPI Development | API Management | Azure API | Azure API Management | Azure Functions401k match | Company holidays | Dental insurance | Employee discount | Medical insuranceSenior-level Full TimeMerriam, KS, United States4h ago
-
Senior Machine Learning Engineer USD 149K-224KAIOps | AWS | Apache Spark | Cloud platform | Data Analysis401k match | Career growth opportunities | Dental insurance | Employee resource groups | Flexible paid time offSenior-level Full TimeRemote, United States4h ago
-
Senior Data Engineer USD 120K-175K.Net Core | .Net Framework | AWS CloudFormation | Apache Airflow | Azure401k match | Career growth opportunities | Dental insurance | Employee resource groups | Life insuranceSenior-level Full TimeRemote, United States4h ago
-
Sr. Data and AI Engineer USD 180K-200KAPI Development | AWS | Agentic AI | Agile | Apache NiFiHybrid work | Public trust clearance supportSenior-level Full TimeARLINGTON, VA, United States4h ago
-
Mid-Level Data Engineer USD 90K-115KAPI Development | Azure | Azure Data | Azure Data Factory | Azure Data LakePublic trust clearance support | Remote workMid-level Full TimeWork from home, VA, United States4h ago
-
Data Engineer USD 145K-160KAWS Glue | AWS IAM | Apache Airflow | Apache Kafka | Apache SparkHybrid remote work | Public Trust eligibility requiredMid-level Full TimeFAIRFAX, VA, United States4h ago
-
AI Model Developer USD 125K-150KAWS | Angular | Azure | CI/CD | D3.jsHybrid remote onsite work | Public trust clearance support | Secret clearance supportMid-level Full TimeFAIRFAX, VA, United States4h ago
-
Senior Databricks Migration Engineer USD 130K-158KActive Directory | Azure Active Directory | Column-Level Security | Data Quality | Data SkippingSenior-level Full TimeArlington, VA, United States4h ago
-
Data Engineer USD 110K-165KAmazon Web Services | Artificial Intelligence | Computer Vision | Data Pipelines | Data ProcessingHealth benefits | Hybrid work | Onsite at customer location | Paid time off | Retirement benefitsSenior-level Full TimeFAIRFAX, VA, United States4h ago
-
Jr Data Engineer USD 104K-125KAmazon Web Services | Artificial Intelligence | CSS | Data Processing | Geospatial DataHybrid remote work | Top-Secret Clearance support | US citizenship requirementEntry-level Full TimeFAIRFAX, VA, United States4h ago
-
AI Requirements Engineer USD 120K-145KAPI Integration | AWS | Agile | Artificial Intelligence | AutomationHybrid workMid-level Full TimeArlington, VA, United States4h ago