Senior Databricks Platform Engineer
Arlington, VA, United States
USD 180K-247K (estimate) Senior-level Full Time
Tasks
- Apply data governance metadata management and cataloging
- Build platform observability
- Collaborate with AWS infrastructure teams
- Configure compute autoscaling and workload management
- Configure metastore and catalog hierarchies
- Configure storage integrations and external locations
- Create Databricks workspaces clusters notebooks jobs and Delta Lake
- Enforce compute policies and cluster configurations
- Establish connections to source systems
- Evaluate and test new Databricks and AWS features
- Implement RBAC security frameworks
- Implement data lineage and auditing
- Implement encryption and data masking
- Implement user provisioning processes
- Integrate identity management using SAML SCIM and Active Directory
- Maintain enterprise Databricks platform
- Maintain technical documentation
- Manage Unity Catalog governance
- Manage secrets and credential handling
- Manage serverless SQL warehouses
- Monitor and tune cluster utilization and performance
- Optimize cost using cost optimization frameworks
- Troubleshoot platform-level issues
- Validate security and compliance controls
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Data | AWS Data Factory | AWS Glue | AWS IAM | AWS KMS | Active Directory | Amazon EMR | Amazon S3 | Apache Airflow | Apache Hadoop | Apache Hive | Apache Spark | Auditing | Autoscaling | CI/CD | Data Cataloging | Data Factory | Data Governance | Data Lineage | Data masking | Databricks | Delta Lake | Encryption | Git | Metadata Management | Python | RBAC | SAML | SCIM | SQL | Scala | Serverless SQL | Unity Catalog | Workload management
Education
Related jobs
-
AI Engineer USD 114K-190KAI orchestration | GPU Inference | GPU inference optimization | Inference Optimization | KubernetesDisability insurance | Health insurance | Holiday pay | Learning and development | Life insuranceSenior-level Full TimeUSA-DC-Washington2h ago
-
API Design | C++ | CI/CD | CMake | Docker401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeNew York2h ago
-
Benchmarking | Code review | Data Pipelines | Distributed Systems | EvaluationSenior-level Full TimeMenlo Park, CA3h ago
-
APIs | Agent systems | Cloud platform | CrewAI | Data PipelinesSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …3h ago
-
C++ | Code review | Compute Technologies | Data Analysis | Data StructuresSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Software Engineer, AI/ML, AI and Infrastructure USD 174K-252KC++ | Data Processing | Data Storage | Data Structures | Data structures algorithmsSenior-level Full TimeMountain View, CA, USA; Kirkland, WA, …3h ago
-
Software Engineer III, AI/ML, Google Workspace USD 147K-211KC++ | Data Processing | Debugging | Distributed Computing | Information RetrievalSenior-level Full TimeKirkland, WA, USA3h ago
-
AI Solution Engineer USD 109K-155KAPIs | AWS | Azure | Embeddings | GCP401k match | Basic life insurance | Dental insurance | Disability coverage | Medical insuranceMid-level Full TimePiscataway, NJ, US7h ago
-
Entry-level Full TimePalo Alto, CA, US, 9430412h ago
-
AWS | Agile | CI/CD | Code review | Distributed Systems401k match | Commuter benefits | Disability insurance | Electric Car Charging Station | Employee assistance programSenior-level Full TimeSeattle, USA14h ago
-
AWS | Agile | CI/CD | Code review | Data Processing401k match | Commuter benefits | Electric Car Charging Station | Employee assistance program | Flexible spending accountsSenior-level Full TimeSeattle, USA14h ago
-
Junior Software Engineer USD 72K-110KDebugging | Problem Solving | Production Code | Python | Software ArchitectureEntry-level Full TimeUnited States or Canada14h ago
-
Robotics Software Engineer, Verification & Validation USD 191K-253KBare Metal | C++ | CANbus | CI/CD | Embedded LinuxEquity grants | Health benefits | Travel opportunityMid-level Full TimeCosta Mesa, California, United States14h ago
-
API Design | Bare Metal | C++ | Embedded Linux | GRPCHealth benefits | Recovery Benefits | Security clearance sponsorship | Travel opportunitiesSenior-level Full TimeCosta Mesa, California, United States14h ago
-
Staff Advanced Concepts Optimization Engineer USD 130K-240KAdjoint methods | Automatic Differentiation | CasADi | Cloud Computing | Computational Fluid DynamicsSenior-level Full TimeSan Jose, California, United States15h ago
-
Principal Software Engineer USD 142K-304K.NET | Active Directory | Amazon S3 | Authentication and Authorization | AzureSenior-level Full TimeRedmond, WA, US15h ago
-
Senior Machine Learning Engineer USD 198K-287KData Engineering | Fine Tuning | Foundation Models | GenAI | Incident ResponseOn-call rotationSenior-level Full TimeRemote - US R15h ago
-
Agentic AI Engineer USD 176K-265KAccess Control | Anthropic | Arize | Audit Logging | CI/CDHybrid work arrangement | On-site collaborationSenior-level Full TimeSan Francisco, CA16h ago
-
Forward-Deployed Engineer, Data Centers USD 140K-165KAPI Integration | Agent workflows | Azure | CI/CD | Deployment401k match | Dental insurance | Device purchase support | Disability insurance | Flexible spending accountsMid-level Full TimeCalifornia16h ago
-
Robotics Software Engineer, Behaviors USD 146K-194KArduPilot | Autonomy | Behavior Trees | C++ | Computer VisionMid-level Full TimeCosta Mesa, California, United States16h ago
-
Data Analytics Lead USD 159K-190KDBT | Dashboards | Data Modeling | Data Pipelines | ETLHybrid work | In-office cultureSenior-level Full TimeNew York, NY16h ago
-
Sr. Solutions Engineer - Oil, Gas, Energy USD 152K-209KAWS | Account Management | Artificial Intelligence | Azure | Big DataSenior-level Full TimeHouston, Texas16h ago
-
Senior-level Full TimeRemote, US R17h ago
-
Senior Backend Engineer, ML Inference Systems USD 135K-237KCI/CD | Distributed Systems | Docker | GCP | GolangCommute subsidy | Employee resource groups | Employee stock ownership | Generous vacation and personal days | Global employee assistance programSenior-level Full TimeMountain View, CA, USA17h ago
-
Data Analyst with AI USD 100K-158KArtificial Intelligence | Business Intelligence | Cloud Platforms | Dashboarding | Data GovernanceContract role | Hybrid work scheduleMid-level Contract Full TimeHouston, TX, United States17h ago