Lead Databricks Platform Engineer / Solutions Architect
Tasks
- Advise clients on data architecture and delivery roadmaps
- Build and contribute accelerators frameworks and reusable assets
- Build and optimize batch and streaming data pipelines using Spark and Delta Lake
- Create reference architectures playbooks and best practices
- Define platform standards reusable patterns and guardrails
- Deliver end to end data use cases from requirements to production
- Design and deploy Databricks Lakehouse platforms
- Develop data products and domain oriented pipelines
- Establish data governance and security in Unity Catalog
- Implement CI CD and DevOps automation for Databricks
- Implement role based access control and data lineage
- Improve data quality observability and pipeline reliability
- Lead adoption of Databricks governance pipelines and AI assisted development
- Lead workshops and solution design sessions
- Mentor junior engineers
- Optimize platform performance cost and reliability
Perks/Benefits
- 401k match
- Dental insurance
- Employee assistance program
- Health savings account
- Life insurance
- Long-term disability
- Medical insurance
- Paid time off
- Short-term disability
- Vision insurance
Skills/Tech-stack
AWS | Access Control | Apache Spark | Azure | CI/CD | Compliance | Data Governance | Data Lineage | Databricks | Dataflow | Delta Lake | DevOps | GCP | Kafka | Lakehouse | MLOps | Model Serving | Python | Role-Based Access | Role-Based Access Control | Scala | Security | Spark Structured Streaming | Structured Streaming | Unity Catalog
Education
N/A
Related jobs
-
Agent systems | Agentic AI | Autonomous workflows | CI/CD | Data LakesRemote workSenior-level Full TimeRemote, USA; Information Technology R5h ago
-
Senior Computer Vision Engineer ID72408 USD 170K-240KAnomaly Detection | Clustering | Computer Vision | Edge Computing | Image SegmentationFlexible schedule | Mentorship | Professional growth | TechtalksSenior-level Full TimeMiami, United States R8h ago
-
Senior Computer Vision Engineer ID72408 USD 170K-240KAnomaly Detection | Clustering | Computer Vision | Edge Computing | Image SegmentationFlexible schedule | Mentorship | Professional growth | TechtalksSenior-level Full TimeJacksonville, United States R8h ago
-
Senior Computer Vision Engineer ID72408 USD 170K-240KAnomaly Detection | Clustering | Computer Vision | Computer vision hardware | Data AnalysisFlexible schedule | Mentorship | Personalized growth roadmaps | Tech talksSenior-level Full TimeBaltimore, United States R8h ago
-
Senior Computer Vision Engineer ID72408 USD 170K-240KAnomaly Detection | Clustering | Computer Vision | Data Science | Edge ComputingFlextime | Mentorship | Personalized growth roadmaps | Remote and office options | TechtalksSenior-level Full TimeOrlando, United States R8h ago
-
Senior Computer Vision Engineer ID72408 USD 170K-240KAnomaly Detection | Clustering | Computer Vision | Edge Computing | Fine TuningFlextime | Mentorship | Personalized growth roadmaps | Remote and office options | TechtalksSenior-level Full TimeTallahassee, United States R8h ago
-
Partner Engineer, Generative AI USD 159K-223KAWS | Agent Orchestration | Bias Mitigation | C# | C++Entry-level Full TimeMenlo Park, CA10h ago
-
Enterprise Systems Engineer USD 165K-200KAnsible | As-a-Service | Chef | Configuration Management | Container OrchestrationEntry-level Full TimeMenlo Park, CA10h ago
-
Software Engineer, SystemML - AI Networking USD 124K-220KC# | C++ | CUDA | CUDA programming | Data-parallelEntry-level Full TimeMenlo Park, CA10h ago
-
Customer and Product Solutions Engineer, GenAI USD 127K-182KC++ | CSS | Cloud Platforms | Data Analysis | Gemini APIEquity compensation | Health insurance | Paid time off | Retirement benefitsMid-level Full TimeSunnyvale, CA, USA10h ago
-
AI Algorithms | APIs | Backend Services | C++ | Cloud infrastructureMid-level Full TimeSunnyvale, CA, USA10h ago
-
Senior Software Engineer, Generative AI, Search Health USD 174K-252KA/B | A/B Testing | B testing | Computer Vision | Data AnalysisSenior-level Full TimeMountain View, CA, USA10h ago
-
RTL Design Engineer, Machine Learning Accelerators USD 138K-197KASIC design | Code review | High Performance | High-Performance Design | Learning algorithmsMid-level Full TimeSunnyvale, CA, USA10h ago
-
C++ | Co-design | Hardware acceleration | Hardware co-design | Kernel optimizationMid-level Full TimeMountain View, CA, USA; San Francisco, …10h ago
-
Research Engineer, Winslow, DeepMind USD 174K-252KAudio Synthesis | Autoregressive models | Deep learning | Dialog Systems | Diffusion ModelsMid-level Full TimeMountain View, CA, USA10h ago
-
API Development | Azure | Azure Data | Azure Data Factory | Azure Data LakeRemote workSenior-level Full TimeWork from home, VA, United States15h ago
-
ClaimsXM Data Engineer III USD 125K-140KAnalysis Services | Azure | Azure Data | Azure Data Factory | Data Cleansing401k employer match | Health insurance | Paid Holidays | Paid Long Term Disability | Paid Short Term DisabilitySenior-level Full TimeUnited States15h ago
-
A Star | AutoWare | Behavior planning | C Sharp | C#Senior-level Full TimeUnited States, Washington, Redmond20h ago
-
Acceptance Testing | Cause analysis | Data Analysis | Electrical troubleshooting | IMUSecurity screening required | Travel opportunitiesSenior-level Full TimeUnited States, Washington, Redmond20h ago
-
Apache Kafka | Apache Spark | Azure Monitor | Azure Networking | Azure ResourceRemote work | Travel up to 30 percentSenior-level Full TimeUnited States R20h ago
-
AWS CDK | AWS CloudFormation | AWS CloudTrail | AWS Direct Connect | AWS IAMContinuous learning | Internal training | Mentorship | Remote work | Travel up to 30 percentSenior-level Full TimeUnited States R20h ago
-
Senior-level Full TimeUnited States21h ago
-
Sr. Forward Deployed Engineer USD 182K-250KAI APIs | AWS | Apache Spark | Application development | Artificial IntelligenceCustomer travel 20 percent timeSenior-level Full TimeSan Francisco, California21h ago
-
Sr. Forward Deployed Engineer USD 182K-250KAWS | Apache Spark | Application development | Artificial Intelligence | AzureCustomer-facing role | Travel 20 percent timeSenior-level Full TimeRemote - Washington D.C. R21h ago
-
Sr. Forward Deployed Engineer USD 182K-250KAWS | Apache Spark | Azure | CI/CD | DatabricksTravel 20 percentSenior-level Full TimeNew York City, New York21h ago