Senior Databricks Platform Engineer
Arlington, VA, United States
USD 180K-247K (estimate) Senior-level Full Time
Tasks
- Apply data masking and security controls
- Build Databricks workspaces clusters notebooks jobs
- Build platform observability dashboards
- Configure metastore and catalog hierarchies
- Configure storage integrations S3 ADLS GCS
- Create technical documentation and operational procedures
- Design RBAC security frameworks
- Enable Delta Lake data storage
- Enforce compute policies and cluster configurations
- Ensure data governance, privacy, and security compliance
- Evaluate new AWS Databricks features
- Implement IAM policies and KMS encryption
- Implement cost optimization frameworks
- Implement data lineage and auditing
- Implement secrets management
- Implement user provisioning and deprovisioning
- Integrate identity management SAML SCIM Active Directory
- Maintain enterprise Databricks platform
- Manage Unity Catalog governance
- Manage connections to source systems databases APIs file systems
- Manage serverless SQL warehouses
- Monitor and optimize cluster utilization
- Troubleshoot platform connectivity authentication and performance
- Tune cluster workloads with autoscaling
Perks/Benefits
Skills/Tech-stack
Active Directory | Apache Hive | Apache Spark | Auditing | Autoscaling | Azure Data | Azure Data Factory | Bash | Cost Optimization | Data Cataloging | Data Factory | Data Lineage | Data masking | Databricks | Delta Lake | EMR | Encryption | Glue | Hadoop | IAM | KMS | Metadata Management | Monitoring | Python | RBAC | S3 | SAML | SCIM | SQL | SQL Warehouses | Scala | Secrets management | Serverless SQL | Serverless SQL Warehouses | Unity Catalog | Unity Catalog Governance | Workload management
Related jobs
-
Featured Feat. Applied AI Engineer - Bay Area USD 211K-263KArtificial Intelligence | C plus plus | C# | Embeddings | Feature Engineering401k | Comprehensive health and wellness benefits | Learning and development opportunities | Unlimited time offMid-level Full TimeHQ (San Francisco)26d ago
-
Access Control | Change Management | Cybersecurity | Data Governance | DevicewiseMid-level Full TimeHeath, OH, United States7h ago
-
Apache Spark | Data Lineage | Data Modeling | Data Quality | Data Quality ManagementRemote work | W2 employmentMid-level Full TimeChicago, IL. R8h ago
-
Software Engineer, Applied AI USD 100K-140KAWS | CUDA | Cloud platform | Computer Vision | Deep learningAnnual learning and development budget | Company offsites | Dental insurance | Flexible PTO | Health insuranceEntry-level Full TimeSan Francisco8h ago
-
AI Engineer USD 141K-236KAzure OpenAI | Azure Security | Azure Security Center | Azure Synapse | DatabricksDisability insurance | Health insurance | Holiday pay | Learning and development | Life insuranceSenior-level Full TimeUSA-DC-Washington10h ago
-
Senior Software Engineer – Data Strategy USD 160K-240KAPIs | Algorithms | Apache Flink | Apache Iceberg | Automation401k match | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimeNew York10h ago
-
Distributed Systems | Machine Learning | Model Serving | Monitoring | Online InferenceSenior-level Full TimeSan Jose, California, United States10h ago
-
Lead Machine Learning Engineer (Multiple Positions) USD 194K-355KAlgorithm Optimization | Code review | Data Pipelines | Infrastructure Optimization | MLOpsSenior-level Full TimeSan Jose, California, United States11h ago
-
Senior Machine Learning Engineer (Multiple Positions) USD 194K-355KA/B | A/B Testing | B testing | Data Analysis | Data PipelinesSenior-level Full TimeSan Jose, California, United States11h ago
-
Senior Machine Learning Engineer (Multiple Positions) USD 194K-355KApache Spark | Big Data | Data Preparation | Deep learning | Experiment designSenior-level Full TimeSeattle, Washington, United States11h ago
-
AI Risk | AI Risk Assessment | Bias Mitigation | C# | C++Senior-level Full TimeBellevue, WA | Menlo Park, CA …11h ago
-
AI workflows | Bias Mitigation | C++ | Capacity Planning | Data ModelingSenior-level Full TimeMenlo Park, CA | Seattle, WA …11h ago
-
Production Engineer (University Grad) USD 177K-200KAI tool integration | APIs | Agent Orchestration | C plus plus | CDNSenior-level Full TimeMenlo Park, CA | Burlingame, CA11h ago
-
C++ | Data Processing | Debugging | Fine Tuning | JAXSenior-level Full TimeMountain View, CA, USA11h ago
-
Software Engineer III, AI/ML, Display Ads USD 147K-211KAlgorithms | C++ | Data Analysis | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA11h ago
-
Staff Software Engineer, ML Fleet Systems USD 207K-301KC++ | Cluster management | Data Structures | Data Structures and Algorithms | DebuggingBonus | Equity | Health benefits | Paid time off | Professional developmentSenior-level Full TimeSunnyvale, CA, USA12h ago
-
ACE | APB | ARM | AXI | Constrained randomSenior-level Full TimeMountain View, CA, USA12h ago
-
Software Engineer, AI/ML, Google Workspace USD 147K-211KData Processing | Debugging | Distributed Computing | Fine Tuning | Generative AIEmployee assistance | Health insurance | Paid time off | Retirement planMid-level Full TimeSunnyvale, CA, USA12h ago
-
Software Engineer III, Speech Production, Infrastructure USD 147K-211KAutomatic Speech Recognition | C++ | Data Structures | Data Structures and Algorithms | Distributed SystemsSenior-level Full TimeMountain View, CA, USA; New York, …12h ago
-
Machine Learning Engineer USD 142K-215KAgentic AI | Bayesian Inference | Code review | Deep learning | Error correctionMid-level Full TimeChicago, IL, US17h ago
-
Customer Success Physicist- Chicago USD 130K-184KInformation science | Python | Quantum Computing | Quantum Control | Quantum InformationTravel for customer engagementMid-level Full TimeChicago, IL, US17h ago
-
Senior Machine Learning Engineer (Computer Vision & AI) USD 149K-198KAgile Development | Anomaly Detection | Azure | CUDA | Cloud ComputingTraining opportunitiesSenior-level Full TimeAUBURN HILLS HQ R&D, MI, US17h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Company impact on learning accessibility | Flexible management approach | Remote-friendlyMid-level Full TimeBellevue, WA, USA17h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Bonus | Equity | Friendly atmosphere | Remote-friendlyMid-level Full TimeDenver, CO, USA17h ago
-
Bash | Cloud platform | Data Pipelines | Data Processing | DockerAsynchronous culture | Career growth | Competitive compensation | Friendly work environment | Remote workMid-level Full TimePalo Alto, CA, USA17h ago