Databricks Platform Architect
Chandler, United States
USD 180K-247K (estimate) Senior-level Contract
Tasks
- Build distributed processing frameworks
- Convert stored procedures to pyspark and structured pipelines
- Define databricks target architecture
- Design migration strategies
- Design pipeline orchestration and workflows
- Establish ai assisted code quality practices
- Implement caching and partition pruning
- Implement data governance and lineage
- Implement full and incremental recalculation flows
- Integrate logging and observability
- Lead code reviews and design walkthroughs
- Manage schema evolution with unity catalog
- Optimize spark performance
- Tune cluster sizing and autoscaling
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Autoscaling | CI/CD | Caching | Cluster Sizing | DBT | Data Governance | Data Lineage | Databricks | Databricks Workflows | Delta Lake | FinOps | Git | Infrastructure as Code | Partition Pruning | Pipelines | PySpark | Pytest | SQL Server | Schema evolution | Stored Procedures | Structured Streaming | T-SQL | Terraform | Unity Catalog | “as-code”
Related jobs
-
People Data Analytics Lead USD 180K-270KATS | Access Control | Ashby | Attrition modeling | Culture AmpSenior-level Full TimeDallas15h ago
-
People Data Analytics Lead USD 180K-270KATS | Access Control | Ashby | Attrition Forecasting | Culture AmpSenior-level Full TimeChicago15h ago
-
Staff Data Platform Engineer USD 210K-240KAuditing | Azure Event | Azure Event Hubs | Batch Processing | CI/CDHealth plan subsidies | Paid global offsites | Remote-first work culture | WFH office reimbursementSenior-level Full TimeRemote - US R15h ago
-
Data Architect - Fleet Aviation Team USD 138K-233KAWS | Amazon Redshift | Apache Airflow | Apache Hive | Apache Kafka401k matching | Dental insurance | Health insurance | Life insurance | Paid federal holidaysSenior-level Full TimeVienna, VA17h ago
-
Principal AI/ML Architect, Applied Field Engineering USD 189K-248KAWS | Apache Airflow | Apache Spark | As-a-Service | Cloud platformSenior-level Full TimeUS-NY-New York17h ago
-
Senior AI/ML Architect, Applied Field Engineering USD 165K-216KAWS | Apache Airflow | Apache Spark | As-a-Service | Cloud platformSenior-level Full TimeUS-NY-New York17h ago
-
Analytics Engineering Manager USD 250KAccess Control | Airflow | BigQuery | CI/CD | DBT401k matching | Catered lunches | Flexible schedule | Happy hours | Life insuranceMid-level Full TimeNew York, NY17h ago
-
AI Solutions Architect- Federal USD 170K-240KAWS | Adversarial Machine Learning | Agentic AI | Air-gapped | Air-gapped environmentsAnnual workspace upgrades | Flexible time off | Fully remote | Home office stipend | Internet and phone stipendSenior-level Full TimeRemote- US R18h ago
-
Staff AI/ML Architect, Embodied AI USD 153K-220KAction models | C++ | CUDA | Critical Systems | DaggerSenior-level Full TimeSunnyvale, CA, United States18h ago
-
Analytics Lead, Japan Partnerships USD 160K-200KAutomation | Dashboard Development | Data Storytelling | Data Visualization | DatabricksAI tools access | Course access | Learning budget | Monthly UDaySenior-level Full TimeSan Francisco, CA19h ago
-
Analytics Lead, Japan Partnerships USD 160K-200KAutomation | Dashboarding | Data Visualization | Data Warehousing | DatabricksAI experimentation time | Hybrid work schedule | Learning access | Monthly UDay | Professional development budgetSenior-level Full TimeSan Francisco, CA19h ago
-
Director, AI Solutions Architect (Remote) USD 152K-223KAPI Gateway | Amazon Bedrock | Azure OpenAI | BigQuery | CI/CDFlexible after hours support for releases | Occasional travel | Overtime available | Remote workSenior-level Full TimeOak Brook, IL, United States R21h ago
-
AI Architect Lead (Hybrid-within BankUnited's footprint) USD 142K-200KAI Foundry | AWS Bedrock | Artificial Intelligence | Autogen | Azure AIHybrid workSenior-level Full TimeMiami Lakes, FL, United States R21h ago
-
Data Integration & Pipeline Lead USD 106K-154KAWS | Agile | Cloud Migration | DBT | Data GovernanceSenior-level Full TimeMiami Lakes, FL, United States21h ago
-
AWS | Agile | Anomaly Detection | Apache Airflow | Apache IcebergBackground check required | Hybrid work schedule | Monday to Friday scheduleExecutive-level Full TimeUnited States22h ago
-
AI prompt engineering | Cloud Data | Cloud Data Warehouse | Data Architecture | Data GovernanceSenior-level ContractAustin, United States23h ago
-
Technical Manager – AI & Data Risk Management USD 134K-265KAlation | Alteryx | Amazon Redshift | Analytics | Apache SparkMid-level Full TimeJersey City, New Jersey, United States; …1d ago
-
340B | Billing Compliance | Business Intelligence | Business Objects | CPTSenior-level Full TimeMaryland-Silver Spring1d ago
-
AWS | Azure | Cloud Native | Data Governance | Data PipelinesCompany-sponsored team events | Flexible time off | Wellness resourcesSenior-level Full TimeLondon, United Kingdom; New York, New …1d ago
-
Manager, Data Engineering USD 90K-105KAgile | Azure Data | Azure Data Services | Azure DevOps | CI/CDDental benefits | Disability benefits | Health and wellness programs | Inclusive culture | Medical benefitsMid-level Full TimeAtlanta, GA, US, 303381d ago
-
Data Engineer IV USD 133K-210KAWS | Airflow | CI/CD | DBT | DagsterPaid Holidays | Paid parental leave | Paid sick leave | Paid vacation | Relocation assistanceSenior-level Full TimeSeattle, WA, United States1d ago
-
Data Platform Administrator USD 112K-179KAWS | AWS IAM | AWS Lambda | AWS S3 | Access ControlPublic Trust ClearanceSenior-level Full TimeUnited States1d ago
-
Sr Specialist - Data Engineering USD 120K-160KAttunity | BigQuery | Change Data Capture | Cloud platform | Control-M401k match | Dental insurance | Employee stock purchase plan | Health insurance | Hybrid remote work scheduleSenior-level Full TimeWestlake, TX, United States1d ago
-
Sr. Solutions Architect - Agencies USD 219K-301KApache Spark | Cloud Computing | Data Engineering | Data analytics | JavaSenior-level Full TimeNew Jersey; New York City, New … R1d ago
-
Solutions Architect - Sports USD 180K-247KAnalytics architecture | Apache Spark | Cloud Proof of Concept | Data Analytics Architecture | Data analyticsTravel up to 30 percentSenior-level Full TimeCentral - United States; Minnesota; Northeast … R1d ago