Databricks Solution Architect
Tasks
- Architect and lead Databricks lakehouse implementation
- Build batch and streaming data pipelines
- Conduct security and data privacy compliance reviews
- Define data modeling standards
- Establish data governance with Unity Catalog
- Implement CI CD and code quality standards
- Manage cloud infrastructure using Terraform
- Mentor data engineering teams
- Operationalize machine learning workloads on Databricks
- Optimize Spark workloads for performance and cost
Perks/Benefits
- N/A
Skills/Tech-stack
AWS S3 | Apache Spark | Autoscaling | Azure Data | Azure Data Lake | Azure Data Lake Storage | CI/CD | Cloud Storage | Data Governance | Data Lake Storage | Data Lineage | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | Encryption | Git | Google Cloud | Google Cloud Storage | IAM | MLflow | Medallion Architecture | Networking | Partitioning | Photon | PySpark | Python | Query Tuning | SQL | Secrets management | Spark SQL | Structured Query | Structured Query Tuning | Structured Streaming | Terraform | Unity Catalog | Z-Order
Education
N/A
Related jobs
-
Lead Analytics Manager USD 120K-150KAffiliate Marketing | Business Intelligence | Dashboard Design | Data Ingestion | Data LiteracySenior-level Full TimeAustin, TX (remote); Dallas, TX (remote); … R11h ago
-
C++ | Cloud Computing | Code Reviews | Deployment Automation | Distributed Systems401k match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R13h ago
-
ARM | C Programming | Device Drivers | Encryption algorithms | EthernetBackground check clearance | Drug Test Clearance | Hybrid schedule | Onsite Days Per WeekMid-level Full TimeLos Angeles R14h ago
-
Manager, AI Engineering - Analytics USD 197K-267KAgent systems | Artificial Intelligence | Data Modeling | Data Warehousing | EvalsHybrid work flexibility | Professional growth opportunities | Stock equity | Work-life balanceMid-level Full TimeHybrid - San Francisco R14h ago
-
Machine Learning Operations Engineer USD 133K-167KAWS SageMaker | Docker | GitHub Actions | Machine Learning | NumPyCareer development | Communities | Commuting cost coverage | Corporate giving programs | Daily free lunchMid-level Full TimeBoston, Massachusetts, United States R14h ago
-
Embedded Software Engineer USD 96K-134KARM | Board Support | Board Support Package | C# | Common CriteriaHybrid work schedule | Onsite Days Per WeekMid-level Full TimeLos Angeles R15h ago
-
Applied AI Engineer, Investments USD 134K-183KAPIs | Artificial Intelligence | Cloud technologies | Data Pipelines | Data Processing401k match | Family-forming benefits | Paid time off | Relocation support | Volunteer time offEntry-level Full TimeRedwood City, CA (Hybrid) R15h ago
-
Senior-level Full TimeRemote - United States R15h ago
-
Solutions Architect - Sports USD 180K-247KAnalytics architecture | Apache Spark | Customer Engagement | Data Analytics Architecture | Data analyticsTravel up to 30%Senior-level Full TimeCentral - United States; Minnesota; Northeast … R16h ago
-
Forward Deployed AI Engineer, West USD 125K-175KAWS | Azure | Docker | GCP | Generative AI401k plan | Dental insurance | Medical insurance | Parental leave | Unlimited paid time offMid-level Full TimeRemote (San Francisco) R18h ago
-
Senior Machine Learning Engineer, Reinforcement Learning USD 150K-250KDomain Randomization | Embedded Systems | Gazebo | Isaac-Gym | Mujoco401k retirement plan | Dental insurance | Employee referral bonus | Flexible PTO | Free lunchSenior-level Full TimeColumbus, Ohio or Remote R18h ago
-
Senior Data Platform Engineer USD 140K-220KApache Hudi | Apache Spark | CI/CD | Delta Lake | Distributed StorageSenior-level Full TimePittsburgh, PA or Remote R19h ago
-
Principal Architect – Data & AI Solutions USD 168K-500KArchitecture Reviews | Artificial Intelligence | Azure | Azure AI | Azure Synapse401k match | Dental insurance | Disability insurance | Employee assistance program | Flexible work locationSenior-level Full TimeRemote - US R19h ago
-
Staff Software Engineer, Data Platform USD 170K-240KAPI Design | Backend Services | Frontend Development | JavaScript | Node.js401k match | Dental insurance | Equity stock options | Health insurance | Learning GrantSenior-level Full TimeRemote - USA R19h ago
-
Senior Engineer - Data Platform USD 148K-201KAirgapped environments | CI/CD | CRD | ConnectRPC | Consistency models401k retirement plan | Conference support | Dental insurance | Disability insurance | Flexible time offSenior-level Full TimeRemote, United States R19h ago
-
Associate Engineer - Data Platform USD 102K-138KAI Assisted Development | Command Line | Command-line Interface | Containerization | Docker401k retirement plan | Conferences travel lodging fees | Dental & vision insurance | Disability insurance | Flexible time offMid-level Full TimeRemote, United States R20h ago
-
Data Science Engineer (Shreveport, LA) USD 37K-40KData Historian | Data Visualization | Data analytics | Excel | Machine Learning401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeAtlanta, GA, United States R22h ago
-
C++ | CUDA | CUDA kernels | Concurrency | Distributed SystemsSenior-level Full TimePittsburgh, PA or Remote R22h ago
-
C++ | CUDA | Data parallelism | GRPC | GoSenior-level Full TimePittsburgh, PA or Remote R22h ago
-
Data Scientist 1 USD 95K-115KAWS | Azure | BigQuery | Cloud platform | Data Engineering401k match | Continuing education reimbursement | Employer Paid Parental Leave | Flexible working hours | Paid time offMid-level Full TimeUnited States - Remote R23h ago
-
Staff Machine Learning Engineer, Underwriting and Credit USD 276K-415KA/B | A/B Testing | AWS | Airflow | B testingFlexible time off | Medical insurance | Modern family planning | Remote work | Retirement savings plansSenior-level Full TimeBay Area, CA, United States of … R23h ago
-
Director, Data Science: Data Science Tools USD 123K-201KAI Agents | Airflow | Bash | Celery | DashHybrid work schedule | Occasional travel | Professional development | Remote workExecutive-level Full TimeRemote, Remote, United States R1d ago
-
BI Analytics Engineer USD 113K-165KAgile | Azure | CI/CD | Data Modeling | Data PipelinesHealth and welfare benefits | Hybrid work | Paid time off | Retirement benefits | Unpaid time offSenior-level Full TimeMoon Township, Pennsylvania, USA, 15108 R1d ago
-
Assistant Director, Data Engineering USD 148K-148KCognos | Collibra | Data Governance | Data Hubs | Data Lakes401a Plan | 401k or 403b plan | 457 plan | EcoPass | Free parkingExecutive-level Full TimeDenver R1d ago
-
Assistant Director, Data Engineering USD 148K-148KContinuous Improvement | Customer Relationship Management | Customer relationship | Data Governance | Data HubsFree onsite parking | Free transit pass | Healthcare benefits | Paid Holidays | Paid Vacation LeaveExecutive-level Full TimeDenver R1d ago