Databricks Platform Architect
Chandler, United States
USD 180K-247K (estimate) Senior-level Contract
Tasks
- Build distributed processing frameworks
- Convert stored procedures to pyspark and structured pipelines
- Define databricks target architecture
- Design migration strategies
- Design pipeline orchestration and workflows
- Establish ai assisted code quality practices
- Implement caching and partition pruning
- Implement data governance and lineage
- Implement full and incremental recalculation flows
- Integrate logging and observability
- Lead code reviews and design walkthroughs
- Manage schema evolution with unity catalog
- Optimize spark performance
- Tune cluster sizing and autoscaling
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Autoscaling | CI/CD | Caching | Cluster Sizing | DBT | Data Governance | Data Lineage | Databricks | Databricks Workflows | Delta Lake | FinOps | Git | Infrastructure as Code | Partition Pruning | Pipelines | PySpark | Pytest | SQL Server | Schema evolution | Stored Procedures | Structured Streaming | T-SQL | Terraform | Unity Catalog | “as-code”
Related jobs
-
Sr Developer Snowflake Data Platform (MDM) USD 166K-218KAccess Control | Data Governance | Data Modeling | Data Warehousing | Data maskingSenior-level Full TimeCollierville, Tennessee, United States14h ago
-
Architect, Data & Analytics USD 119K-231KAWS | Azure | Business Intelligence | Data Governance | Data ModelingSenior-level Full TimeRemote, United States R14h ago
-
Tech Lead Manager, Data Engineer USD 251K-310KBigQuery | C++ | Data Governance | Data Lake | Data ModelingBonus program | Company benefits program | Equity incentive plan | Hybrid work scheduleSenior-level Full TimeMountain View, CA15h ago
-
Apache Hive | Apache Iceberg | Apache Kafka | Authentication | AuthorizationSenior-level Contract Full TimeCincinnati, Ohio, United States17h ago
-
Revenue Analytics Lead USD 182K-243KAutomation | DBT | Data Governance | Data Modeling | LLMEquity | Health insurance | Paid time offSenior-level Full TimeRemote, USA; San Mateo, CA R17h ago
-
Data Architect (Remote) USD 156KDDL | DML | Data Architecture | Data Documentation | Data Governance401k matching | Disability coverage | Healthcare | Life insurance | Paid parental leaveSenior-level Full TimeBoston, Massachusetts, United States; New York, … R18h ago
-
Power BI Architect - SME USD 120K-175KCybersecurity compliance | Dashboard Development | Data Governance | Data Modeling | Data integrationSenior-level Full TimeFAIRFAX, VA, United States18h ago
-
Manager, Data Engineering USD 170K-205KAWS | Agile | Airflow | DBT | FivetranDental insurance | Disability insurance | Employee assistance program | Health insurance | Life insuranceSenior-level Full TimeWashington DC18h ago
-
Principal Data AI Architect USD 197K-337KAnalytics | Artificial Intelligence | Big Data | Cloud | Data ArchitectureCareer development support | Coaching and mentoring | Learning and development programsSenior-level Full TimeChicago, Illinois, USA R20h ago
-
Tech Lead, GTM Applied AI and Analytics USD 138K-225KAirflow | Amazon SageMaker | DBT | Databricks | KubeflowSenior-level Full TimeSan Francisco, CA, United States20h ago
-
AI Solution Architect USD 90K-148KAPI Design | Agent Orchestration | Data Governance | Distributed Systems | Event DrivenHybrid work schedule | Relocation not availableSenior-level Full TimeDallas, TX, United States21h ago
-
Solution Architect (AI & Data Applications) USD 145K-260KAngular | Autogen | CI/CD | Databricks | Distributed SystemsMentoring systemSenior-level Full TimeJersey City, NJ, United States1d ago
-
Manager of Data Governance USD 136K-177KAccess Control | Confluence | Data Cataloging | Data Governance | Data LineageSubject matter expertise guidance | Team collaboration | Training sessionsMid-level Full TimeFairfield, CA, United States1d ago
-
Delivery Solutions Architect - Public Sector USD 180K-247KData Engineering | Data Governance | Data analytics | Distributed Systems | Program ManagementHybrid work | Travel up to 30%Senior-level Full TimeCharlotte, North Carolina1d ago
-
Manager, Data Engineering USD 127K-183KAutosys | BigQuery | Change Management | Cloud Build | Cloud Deployment Manager401k match | Employee stock purchase plan | Health, dental, vision insurance | Hybrid remote work schedule | Paid parental leaveMid-level Full TimeAustin, TX, United States1d ago
-
Sr Manager, Data Engineering Senior USD 107K-157KCloud Data | Cloud Data Proc | Cloud Service | Cloud Storage | Control-M401k company match | Employee stock purchase plan | Health, dental, vision insurance | Hybrid remote work schedule | Paid parental leaveSenior-level Full TimeAustin, TX, United States1d ago
-
Principal Data Solution Architect (Databricks) USD 165K-210KAWS | Amazon Redshift | Apache Spark | Azure | Azure SynapseSenior-level Full TimeMcLean, VA, United States1d ago
-
Lead AI Engineer (ML Ops) USD 116K-170KAPIs | AWS | Agile Scrum | Azure | CI/CD401k match | Employee assistance program | Employee charity match | Employee stock purchase plan | Health and wellness allowanceSenior-level Full TimeIrving - 6011 Connection, United States1d ago
-
Adobe Experience | Adobe Experience Manager | Adobe Firefly | Agile | CMS401k plan | Bonus opportunities | Dental insurance | Life insurance | Long-term disabilitySenior-level Full TimeSan Francisco, 415 Mission Street, Corp, …1d ago
-
AI Architect USD 149K-248KAWS | Agentic AI | Azure | CI/CD | Cloud platform401k retirement plan | Adoption Assistance | Employee referral program | Health savings account | Parental Leave and Adoption AssistanceSenior-level Full TimeHome Office: Tysons, VA, United States1d ago
-
Sr. Data Engineering Lead, Onsite, VP USD 120K-202KAgile | Azure | Azure Data | Azure Data Factory | Azure DatabricksDental insurance | Employee assistance program | Flexible work schedule | Health insurance | Life insuranceSenior-level Full TimeQuincy, Massachusetts, United States1d ago
-
API | Agile | Azure | Azure Data | Azure Data FactoryContract-to-hire | Hybrid workMid-level Full TimeBoston, MA R1d ago
-
Lead Data Engineer USD 90K-160KAI BI | Airflow | Amazon Web Services | Azure | CI/CDCompany-Paid Holidays | Disability insurance | Employee assistance program | Employee assistance programs | Life insuranceSenior-level Full TimeIrving, TX, US1d ago
-
Director - Data Analytics & Engineering USD 180K-210KAnalytics engineering | Automated testing | Azure Data | Azure Data Factory | Azure Data LakeExecutive-level Full TimeNJ-Warren, United States1d ago
-
Lead Data Engineer (multiple positions available) USD 151K-184KAuditing | Automation | Customer 360 | Data Modeling | Data Services401k retirement plan | Accident insurance | Basic life insurance | Critical illness insurance | Dental insuranceSenior-level Full TimeIselin, NJ, US R1d ago