Databricks Platform Architect
Chandler, United States
USD 180K-247K (estimate) Senior-level Contract
Tasks
- Build distributed processing frameworks
- Convert stored procedures to pyspark and structured pipelines
- Define databricks target architecture
- Design migration strategies
- Design pipeline orchestration and workflows
- Establish ai assisted code quality practices
- Implement caching and partition pruning
- Implement data governance and lineage
- Implement full and incremental recalculation flows
- Integrate logging and observability
- Lead code reviews and design walkthroughs
- Manage schema evolution with unity catalog
- Optimize spark performance
- Tune cluster sizing and autoscaling
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Autoscaling | CI/CD | Caching | Cluster Sizing | DBT | Data Governance | Data Lineage | Databricks | Databricks Workflows | Delta Lake | FinOps | Git | Infrastructure as Code | Partition Pruning | Pipelines | PySpark | Pytest | SQL Server | Schema evolution | Stored Procedures | Structured Streaming | T-SQL | Terraform | Unity Catalog | “as-code”
Related jobs
-
Power BI Technical Lead (New Jersey / US) USD 113K-132KAgile | Azure | Azure DevOps | BI Gateway | DAXSenior-level Full TimeNew Jersey, United States14h ago
-
A/B | A/B Testing | Apache Airflow | Apache Kafka | Apache SparkSenior-level ContractSanta Monica, United States14h ago
-
AWS EMR | AWS Glue | AWS Lambda | AWS S3 | Amazon RedshiftSenior-level Full TimeUnited States1d ago
-
Client Director, Frontier Data - US USD 255K-325KAPIs | Automation | Data Annotation | Data Validation | Databricks5 days per week | Flexible working hours | Hybrid work | Supportive work cultureExecutive-level Full TimePalo Alto, California, United States; San …1d ago
-
Data Intelligence Lead USD 114K-140KAdvanced Analytics | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeRemote, United States R1d ago
-
AI/ML Specialist Solutions Architect USD 220K-375KAWS | Ansible | Azure | CUDA | Cloud platform401k match | Career growth | Collaborative culture | Disability insurance | Equity participationSenior-level Full TimeCanada; Remote - United States R1d ago
-
Analytics Engineer Manager, CNBC & MS NOW USD 110K-130KData Documentation | Data Modeling | Data Pipelines | Data Quality | Data Testing401k | Dental insurance | Discounts and perks | Hybrid work | Medical insuranceMid-level Full TimeNew York, NY, United States1d ago
-
Analytics Engineer Manager, CNBC & MS NOW USD 110K-130KBronze Silver Gold | Data Documentation | Data Modeling | Data Pipelines | Data Quality401k | Dental insurance | Discounts | Dry Cleaning Service | Gourmet cafeteriaMid-level Full TimeEnglewood Cliffs, NEW JERSEY, United States1d ago
-
Director, Data Governance & Observability USD 170K-200KAPI Design | Alerting | Automated testing | CI/CD | Data ContractsHealth insurance | Paid time off | Retirement plansExecutive-level Full TimeNew York, NEW YORK, United States1d ago
-
Director, Data Engineering USD 165K-210KAWS | Agile | Amazon Redshift | Azure | BigQueryHealth insurance | Paid time off | Reasonable accommodation during hiring process | Retirement plansExecutive-level Full TimeNew York, NEW YORK, United States1d ago
-
Senior-level Full TimeNew Jersey, United States1d ago
-
AI Architect Lead (Hybrid-within BankUnited's footprint) USD 140K-205KAI Foundry | AWS | AWS Bedrock | Agentic AI | Architecture ReviewHybrid work modelSenior-level Full TimeMiami Lakes, FL, United States R1d ago
-
Data Integration & Pipeline Lead USD 106K-160KAWS | Agile | DBT | Data Governance | Data LineageHybrid work environmentSenior-level Full TimeMiami Lakes, FL, United States1d ago
-
Lead Software Engineer - Data Engineer USD 170K-185KAWS | AWS Glue | AWS Glue Data Catalog | AWS S3 | Apache IcebergBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeJersey City, NJ, United States1d ago
-
Sr. Solutions Architect - Strat Media USD 219K-301KApache Spark | Cloud | Data analytics | Java | Pre-salesSenior-level Full TimeNortheast - United States; Remote - … R1d ago
-
Data Architect USD 112K-174KAcceptance Testing | Agile | Application Lifecycle Management | Application lifecycle | Business IntelligenceSenior-level Full TimeNYC, NY1d ago
-
Head of Enterprise Applications & Data Engineering USD 160K-190KChange Management | Cybersecurity | Data Architecture | Data Governance | Data Lake401k | Career advancement | Community involvement | Disability insurance | Legal insuranceExecutive-level Full TimeColumbia, Maryland, United States1d ago
-
Senior Solutions Architect - AI/ML - Services Delivery USD 160K-210KAI Operations | AWS | AWS SageMaker | Apache Kafka | Apache SparkSenior-level Full TimeUS-NC-Remote R1d ago
-
AI Program Lead USD 159K-265KAI Governance | Agent-Based Artificial Intelligence | Agent-based | Artificial Intelligence | Business IntelligenceMentorship | Professional development | Travel up to 25 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States1d ago
-
AI orchestration | Alerting | Caching | Distributed Systems | DockerDental insurance | Medical insurance | Paid time off | Savings plan options | Vision insuranceSenior-level Full TimeSan Francisco, CA, United States R1d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksCollaborative cross-functional team | Exposure to large scale insurance analytics | Fully remote work | High-impact leadership role | Long term contract with extension potentialSenior-level Contract Full TimeCanada R1d ago
-
AI Search | Agent systems | Agentic Systems | Azure AI | Azure AI SearchCertification support | Dental insurance | Disability coverage | Discount perks | Employee assistance programSenior-level Full TimeCanada1d ago
-
Big Data | Continuous Delivery | Data Engineering | Data Governance | Data LakesFlexible working arrangements | Health, dental, and vision insurance | Mentorship and career growth | Professional development | Remote-friendlySenior-level Full TimeCanada1d ago
-
Data Engineer Lead USD 98K-199KAWS | AWS Glue | AWS Lambda | Agile | Amazon S3401k | Charitable giving | Continuing education opportunities | Dental insurance | Employee assistance programSenior-level Full TimeLake Elmo, MN, United States1d ago
-
Lead, Data Governance USD 200K-220KAPI Integration | Agentforce | Customer360 | Data Governance | Data LineageSenior-level Full TimeHoboken, NJ, United States2d ago