Principal Data Solution Architect (Databricks)
Tasks
- Architect Databricks lakehouse environments
- Architect metadata systems and data lineage
- Build ingestion pipelines for batch and streaming
- Collaborate with data engineering and data science teams
- Create end to end data architecture roadmaps
- Define canonical data models
- Design Delta Lake data storage
- Design data platform modernization
- Design data transformation layers
- Design semantic layers
- Develop data serving layers for analytics and AI
- Develop reusable architecture patterns and playbooks
- Enforce schema evolution and data contracts
- Ensure Zero Trust access controls
- Implement CDC ingestion
- Implement Unity Catalog governance
- Implement data quality frameworks
- Lead enterprise data modeling strategy
- Mentor data engineers and architects
- Optimize compute orchestration and cost performance
- Orchestrate cluster and job scheduling
- Oversee enterprise platform governance and reviews
- Support cloud readiness assessments
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Spark | Azure | Containers | Data Contracts | Data Governance | Data Lineage | Data Processing | Data Quality | Data fabric | Data mesh | Databricks | Delta Lake | Distributed data | Distributed data processing | Flink | Google Cloud | Kafka | Lakehouse | MDM | Metadata Management | Python | SQL | Schema evolution | Serverless | Structured Streaming | Unity Catalog | Zero Trust
Education
Related jobs
-
Senior Data Platform Engineer, Remote USD 142K-180KAWS | AWS Lambda | Access Control | Amazon Aurora | Amazon CloudWatch401k matching | Dental insurance | Flexible time off | Flexible work environment | Medical insuranceSenior-level Full TimeUnited States, UNITED STATES, United States R13h ago
-
Principal Data Scientist, Pricing Strategy USD 191K-295KCausal Inference | Experiment design | Forecasting | Machine Learning | Optimization401k | Dental insurance | Flexible spending account | Health insurance | Paid time offSenior-level Full TimeBellevue, Washington; San Francisco, California16h ago
-
Principal Research Scientist – Scaling USD 280K-350KBenchmarking | Compute efficiency | Distributed Training | Energy Efficiency | Evaluation metricsSenior-level Full TimeSan Francisco, California18h ago
-
Sr. Principal Software Engineer, AI Solutions USD 125K-180KAPI Gateway | API first | API-First Development | AWS | AWS CDKSenior-level Full TimeRemote, United States R19h ago
-
Data Architecture | Data Governance | Data Modeling | Data Systems | Data VisualizationSenior-level ContractBaton Rouge, United States21h ago
-
Data Architect USD 110K-174KAWS Redshift | Amazon Web Services | Azure Synapse | Azure Synapse Analytics | Cloud ComputingTravel opportunitiesSenior-level Full TimeTallahassee, United States1d ago
-
Principal Data Scientist - Search USD 147K-240KA/B | A/B Testing | APIs | AWS | Agile401k savings plan | Adoption benefits | Career development | Employee assistance program | Employee discountsSenior-level Full TimeChicago, Illinois, United States1d ago
-
Principal, Data Engineer USD 107K-216KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | CI/CDSenior-level Full Time11 Keewaydin Dr, Salem NH, United …1d ago
-
Analytics | Analytics Cloud | Business Data Cloud | Business Intelligence | CDS Views401k plan | Dental insurance | Employee assistance program | Health insurance | Health savings accountSenior-level Full TimeUSA-MN Mounds View South, United States1d ago
-
Principal Data Scientist USD 180K-205KAnomaly Detection | Artificial Intelligence | Cloud Platforms | Data extraction | Deep learning401k matching | Dental insurance | Disability insurance | Flexible time off | Life insuranceSenior-level Full TimeRemote USA, United States R1d ago
-
Principal AI/ML Engineer USD 128K-188KAutomation | Data Governance | Data Pipelines | Data Quality | Language ProcessingHybrid work modelSenior-level Full TimeMalvern, PA, United States1d ago
-
AI Solutions Architect USD 100K-110KAI Governance | AIOps | AWS | Agentic AI | AgileDiscount programs | Fitness classes and recreation center access | Flexible health and dental options | Free RTD EcoPass | Generous paid time offSenior-level Full TimeColorado, United States1d ago
-
Principal Data Scientist USD 180K-205KAnomaly Detection | Artificial Intelligence | Cloud Computing | Deep learning | Distributed Computing401k plan | Dental insurance | Flexible time off | Medical insurance | Paid parental leaveSenior-level Full TimeRemote USA, United States R1d ago
-
APIs | Agent systems | Agile | Angular | Apache KafkaMedical, dental & vision coverage | Paid time off | Retirement savings options | Wellness programsSenior-level Full TimeBuffalo Grove-2100 E Lake Cook, United …1d ago
-
Principal Analytics Architect (Decision Science) USD 144K-288KAnalytics engineering | Data Architecture | Data Modeling | Excel Power Pivot | ForecastingMedical, dental & vision coverage | Paid time off | Retirement savings options | Wellness programsSenior-level Full TimeChicago-525 West Monroe, United States1d ago
-
Sr. Data Architect USD 151K-226KAI-Ready Data | Amazon Kinesis | Avro | Data Architecture | Data Catalog401k plan | Education assistance | Flexible spending accounts | Paid Holidays | Professional growth and development programsSenior-level Full TimeEl Segundo, United States1d ago
-
Senior Applied AI Solutions Architect, Federal Financial USD 153K-207KAgentic Systems | Anomaly Detection | Artificial Intelligence | Classification | Cloud ArchitectureCareer growth | Flexible work arrangements | Mentorship | Work-life balanceSenior-level Full TimeArlington, Virginia, USA1d ago
-
Sr. Advanced Compute Specialist SA - National Security, National Security Specialists Solutions Architecture Team USD 153K-207KAWS | AWS Graviton | AWS Inferentia | AWS Trainium | Amazon BraketSenior-level Full TimeArlington, Virginia, USA1d ago
-
Principal AI Lead – Surgical AI USD 157K-271KArtificial Intelligence | Auditability | Cloud Computing | Computer Vision | Convolutional Neural NetworksSenior-level Full TimeUS328 CA Santa Clara - 5490 … R1d ago
-
Access Control | Collibra | Data Architecture | Data Contracts | Data GovernanceHybrid work | Reasonable accommodations for disabilities | Work-from-home opportunitiesSenior-level Full TimeNew York, NY, US, 101721d ago
-
AI/ML | AI/ML Pipelines | AWS | Apache Airflow | Apache SparkSenior-level Full TimeTexas, Texas, United States1d ago
-
Solutions Architect, Agentic AI USD 152K-241KAPI Integration | Agent systems | Agents SDK | Benchmarking | C++Benefits | EquitySenior-level Full TimeUS, CA, Santa Clara, United States1d ago
-
Senior Associate, AI Solution Architect USD 110K-140KAI Governance | Architecture Documentation | Artificial Intelligence | Cloud Platforms | Data GovernanceAdoption Assistance | Discretionary bonus | Hybrid work schedule | Leave programs | Overtime eligibleSenior-level Full TimeTampa, FL, US1d ago
-
Principal Engineer I - Senior Data Engineer USD 140K-180KAgile | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake401k matching | Dental insurance | Employee volunteer program | Medical insurance | Time offSenior-level Full TimeBlock 23, United States1d ago
-
Principal Engineer I - Lead Data Engineer USD 140K-156KAgile | Azure Data | Azure Data Factory | Azure DevOps | Azure Functions401k matching | Dental insurance | Employee volunteer program | Medical insurance | Ownership stakeSenior-level Full TimeBlock 23, United States1d ago