Lead Databricks Platform Engineer / Solutions Architect
Tasks
- Advise clients on data architecture and delivery roadmaps
- Build and contribute accelerators frameworks and reusable assets
- Build and optimize batch and streaming data pipelines using Spark and Delta Lake
- Create reference architectures playbooks and best practices
- Define platform standards reusable patterns and guardrails
- Deliver end to end data use cases from requirements to production
- Design and deploy Databricks Lakehouse platforms
- Develop data products and domain oriented pipelines
- Establish data governance and security in Unity Catalog
- Implement CI CD and DevOps automation for Databricks
- Implement role based access control and data lineage
- Improve data quality observability and pipeline reliability
- Lead adoption of Databricks governance pipelines and AI assisted development
- Lead workshops and solution design sessions
- Mentor junior engineers
- Optimize platform performance cost and reliability
Perks/Benefits
- 401k match
- Dental insurance
- Employee assistance program
- Health savings account
- Life insurance
- Long-term disability
- Medical insurance
- Paid time off
- Short-term disability
- Vision insurance
Skills/Tech-stack
AWS | Access Control | Apache Spark | Azure | CI/CD | Compliance | Data Governance | Data Lineage | Databricks | Dataflow | Delta Lake | DevOps | GCP | Kafka | Lakehouse | MLOps | Model Serving | Python | Role-Based Access | Role-Based Access Control | Scala | Security | Spark Structured Streaming | Structured Streaming | Unity Catalog
Education
N/A
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R11d ago
-
Access Control | Audio Visual | Audio/Visual Technologies | Cause analysis | Display systemsRetirement offeringsMid-level Full TimeLos Angeles, CA, United States3h ago
-
Machine Learning Engineer, Agentic Systems - Moveworks USD 116K-192KAsyncio | Backpressure | Distributed locking | Distributed tracing | DynamoDBFlexible work location | Remote flexibilitySenior-level Full TimeMountain View, CALIFORNIA, United States4h ago
-
Senior Data Engineer USD 105K-120KAPIs | Automated testing | CI/CD | DBT | Data Quality401k | Insurance & Health | Insurance dental | Insurance vision | Life insuranceSenior-level Full TimeRemot, USA, United States R8h ago
-
Senior Algorithms Engineer USD 144K-200KAWS | Bayesian Inference | CI/CD | Code review | Django3-day weekend | 4-day work week | Fully remote within the United States | Mentoring and training | Travel 1 to 3 times per yearSenior-level Full TimeCary, NC, United States R8h ago
-
Machine Learning Co-Op (Summer 2026) USD 115K-172KCloud Data | Cloud Data Tools | DAX | Data Tools | Deep learningMid-level Full TimeCanton, OH, United States11h ago
-
Machine Learning Co-Op (Fall 2026) USD 104K-172KCloud Data | Cloud Data Tools | DAX | Data Tools | Deep learningMid-level Full TimeCanton, OH, United States11h ago
-
Lead Data Engineer USD 130K-174KAlerting | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage ADLSSenior-level Full TimeColumbus, GA, United States12h ago
-
AI Solution Architect | Onsite | Local to Florida | Full Time | USC/GC Preferred | 12+ years exp. USD 150K-195KAI Governance | AI Solution | AI Solution Architecture | AI/ML | APIsSenior-level Contract Full TimeTampa, FL, United States12h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI ingestion | AWS | Airflow | Apache Spark | AvroFlexible schedule | Mentorship | Office options | Professional growth | Remote work optionsSenior-level Full TimeBlacksburg, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroEducation budget | Exciting projects | Fitness budget | Flexible schedule | MentorshipSenior-level Full TimeMiami, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Apache Spark | Avro | Data LakeFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Professional growthSenior-level Full TimeBoca Raton, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Avro | DuckDB | ELTFlexible schedule | Mentorship | Office work options | Professional growth | Remote work optionsSenior-level Full TimeTexas City, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Airflow | Avro | Big DataFlexible schedule | Mentorship | Office work options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeFort Lauderdale, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeBoston, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Avro | Big Data | Columnar StorageFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeJacksonville, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimePort Charlotte, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Avro | DuckDB | ELTEducation budget | Fitness budget | Flexible schedule | Mentorship | Remote work optionsSenior-level Full TimeBaltimore, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeRichmond, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Apache Spark | Avro | Data LakeEducation budget | Fitness budget | Flexible schedule | Mentorship | Office workSenior-level Full TimeAtlanta, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Avro | Big Data | Columnar StorageEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeJersey City, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Airflow | Apache Spark | AvroEducation budget | Exciting projects | Fitness budget | Flexible schedule | MentorshipSenior-level Full TimeIrving, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Avro | Columnar Storage | Data LakeFlexible schedule | Mentorship | Office work option | Personalized growth roadmaps | Professional growthSenior-level Full TimeLos Angeles, United States13h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Airflow | Apache Spark | AvroEducation budget | Exciting projects | Fitness budget | Flexible schedule | MentorshipSenior-level Full TimeNew York, United States14h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | AWS SageMaker | Airflow | Apache SparkEducation budget | Fitness budget | Flextime | Mentorship | Office optionsSenior-level Full TimeSan Francisco, United States14h ago