Databricks Platform Architect
Chandler, United States
USD 180K-247K (estimate) Senior-level Contract
Tasks
- Build distributed processing frameworks
- Convert stored procedures to pyspark and structured pipelines
- Define databricks target architecture
- Design migration strategies
- Design pipeline orchestration and workflows
- Establish ai assisted code quality practices
- Implement caching and partition pruning
- Implement data governance and lineage
- Implement full and incremental recalculation flows
- Integrate logging and observability
- Lead code reviews and design walkthroughs
- Manage schema evolution with unity catalog
- Optimize spark performance
- Tune cluster sizing and autoscaling
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Autoscaling | CI/CD | Caching | Cluster Sizing | DBT | Data Governance | Data Lineage | Databricks | Databricks Workflows | Delta Lake | FinOps | Git | Infrastructure as Code | Partition Pruning | Pipelines | PySpark | Pytest | SQL Server | Schema evolution | Stored Procedures | Structured Streaming | T-SQL | Terraform | Unity Catalog | “as-code”
Related jobs
-
People Data Analytics Lead USD 180K-270KATS | Access Control | Ashby | Attrition modeling | Culture AmpSenior-level Full TimeDallas13h ago
-
People Data Analytics Lead USD 180K-270KATS | Access Control | Ashby | Attrition Forecasting | Culture AmpSenior-level Full TimeChicago13h ago
-
Staff Data Platform Engineer USD 210K-240KAuditing | Azure Event | Azure Event Hubs | Batch Processing | CI/CDHealth plan subsidies | Paid global offsites | Remote-first work culture | WFH office reimbursementSenior-level Full TimeRemote - US R13h ago
-
Data Architect - Fleet Aviation Team USD 138K-233KAWS | Amazon Redshift | Apache Airflow | Apache Hive | Apache Kafka401k matching | Dental insurance | Health insurance | Life insurance | Paid federal holidaysSenior-level Full TimeVienna, VA15h ago
-
Principal AI/ML Architect, Applied Field Engineering USD 189K-248KAWS | Apache Airflow | Apache Spark | As-a-Service | Cloud platformSenior-level Full TimeUS-NY-New York15h ago
-
Senior AI/ML Architect, Applied Field Engineering USD 165K-216KAWS | Apache Airflow | Apache Spark | As-a-Service | Cloud platformSenior-level Full TimeUS-NY-New York15h ago
-
Analytics Engineering Manager USD 250KAccess Control | Airflow | BigQuery | CI/CD | DBT401k matching | Catered lunches | Flexible schedule | Happy hours | Life insuranceMid-level Full TimeNew York, NY15h ago
-
AI Solutions Architect- Federal USD 170K-240KAWS | Adversarial Machine Learning | Agentic AI | Air-gapped | Air-gapped environmentsAnnual workspace upgrades | Flexible time off | Fully remote | Home office stipend | Internet and phone stipendSenior-level Full TimeRemote- US R16h ago
-
Staff AI/ML Architect, Embodied AI USD 153K-220KAction models | C++ | CUDA | Critical Systems | DaggerSenior-level Full TimeSunnyvale, CA, United States17h ago
-
Analytics Lead, Japan Partnerships USD 160K-200KAutomation | Dashboard Development | Data Storytelling | Data Visualization | DatabricksAI tools access | Course access | Learning budget | Monthly UDaySenior-level Full TimeSan Francisco, CA17h ago
-
Analytics Lead, Japan Partnerships USD 160K-200KAutomation | Dashboarding | Data Visualization | Data Warehousing | DatabricksAI experimentation time | Hybrid work schedule | Learning access | Monthly UDay | Professional development budgetSenior-level Full TimeSan Francisco, CA17h ago
-
Director, AI Solutions Architect (Remote) USD 152K-223KAPI Gateway | Amazon Bedrock | Azure OpenAI | BigQuery | CI/CDFlexible after hours support for releases | Occasional travel | Overtime available | Remote workSenior-level Full TimeOak Brook, IL, United States R19h ago
-
AI Architect Lead (Hybrid-within BankUnited's footprint) USD 142K-200KAI Foundry | AWS Bedrock | Artificial Intelligence | Autogen | Azure AIHybrid workSenior-level Full TimeMiami Lakes, FL, United States R19h ago
-
Data Integration & Pipeline Lead USD 106K-154KAWS | Agile | Cloud Migration | DBT | Data GovernanceSenior-level Full TimeMiami Lakes, FL, United States19h ago
-
AI prompt engineering | Cloud Data | Cloud Data Warehouse | Data Architecture | Data GovernanceSenior-level ContractAustin, United States21h ago
-
Technical Manager – AI & Data Risk Management USD 134K-265KAlation | Alteryx | Amazon Redshift | Analytics | Apache SparkMid-level Full TimeJersey City, New Jersey, United States; …22h ago
-
340B | Billing Compliance | Business Intelligence | Business Objects | CPTSenior-level Full TimeMaryland-Silver Spring22h ago
-
AWS | Azure | Cloud Native | Data Governance | Data PipelinesCompany-sponsored team events | Flexible time off | Wellness resourcesSenior-level Full TimeLondon, United Kingdom; New York, New …23h ago
-
Manager, Data Engineering USD 90K-105KAgile | Azure Data | Azure Data Services | Azure DevOps | CI/CDDental benefits | Disability benefits | Health and wellness programs | Inclusive culture | Medical benefitsMid-level Full TimeAtlanta, GA, US, 303381d ago
-
Data Engineer IV USD 133K-210KAWS | Airflow | CI/CD | DBT | DagsterPaid Holidays | Paid parental leave | Paid sick leave | Paid vacation | Relocation assistanceSenior-level Full TimeSeattle, WA, United States1d ago
-
Data Platform Administrator USD 112K-179KAWS | AWS IAM | AWS Lambda | AWS S3 | Access ControlPublic Trust ClearanceSenior-level Full TimeUnited States1d ago
-
Sr. Solutions Architect - Agencies USD 219K-301KApache Spark | Cloud Computing | Data Engineering | Data analytics | JavaSenior-level Full TimeNew Jersey; New York City, New … R1d ago
-
Solutions Architect - Sports USD 180K-247KAnalytics architecture | Apache Spark | Cloud Proof of Concept | Data Analytics Architecture | Data analyticsTravel up to 30 percentSenior-level Full TimeCentral - United States; Minnesota; Northeast … R1d ago
-
Data Architect | Onsite - Delaware USD 72K-252KAPI Integration | Batch Processing | Data Architecture | Data Dictionary | Data Governance401k retirement plan | Dental benefits | Independent contractor | Medical benefits | Paid HolidaysSenior-level Full TimeUnited States1d ago
-
Prin AI & Analytics Data Architect USD 160K-235KAPIs | AWS | BI | Cloud Computing | DSPM401k matching | Education reimbursement | Flexible work options | Health and well-being programs | Life and disability insuranceSenior-level Full TimeUnited States - Shakopee, MN (GHQ), …1d ago