Senior Databricks Engineer
Tasks
- Architect ETL/ELT pipelines
- Build and maintain production data pipelines
- Conduct code reviews and mentor data engineers
- Design medallion data architecture
- Drive root cause analysis and continuous improvement
- Enforce data quality governance and lineage
- Implement Delta Live Tables
- Implement Unity Catalog access control
- Monitor pipelines and troubleshoot failures
- Optimize Spark jobs and cluster configurations
- Orchestrate Databricks workflows
- Set up CI/CD pipelines
Perks/Benefits
Skills/Tech-stack
AWS | Autoscaling | Azure | CI/CD | Caching | Cluster configuration | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | ELT | ETL | GCP | Git | Infrastructure as Code | Medallion Architecture | Partitioning | PySpark | Python | SQL | Spark SQL | Unity Catalog | “as-code”
Education
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R13d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R26d ago
-
Lead Machine Learning Engineer CAD 156K-251KAWS | Azure | CI/CD | Cloud Computing | Cloud platformAutonomy | Career development support | Learning and development programs | Mentorship and coachingSenior-level Full TimeToronto, Canada R18h ago
-
Lead Machine Learning Engineer USD 172K-276KAmazon Web Services | Azure | CI/CD | Cloud platform | DatabricksLearning and development programs | Mentorship and coachingSenior-level Full TimeChicago, Illinois, USA R18h ago
-
Solution Engineer USD 165K-216KAnalytics | Cloud Computing | Data Lake | Data Warehousing | Data fabricSenior-level Full TimeUS-FL-Remote R19h ago
-
Senior Data Engineer USD 126K-180KAmazon Kinesis | Apache Airflow | Apache Beam | Apache Flink | Apache Spark401k match | Bonus | Equity grant | Flexible time off | Health plansSenior-level Full TimeNew York, New York; Miami, Florida; … R20h ago
-
Senior Analytics Engineer USD 126K-180KAutomation | BI reporting | DBT | Data Modeling | Data Quality401k with company matching | Bonus | Equity grant | Flexible time off | Health plansSenior-level Full TimeNew York, New York; Miami, Florida; … R20h ago
-
Senior Applied Scientist USD 170K-286KApache Spark | C++ | Computer Vision | Data Processing | Edge inferenceComprehensive health insurance | Parental leave | Professional development stipend | Remote work | Travel up to 5 percentSenior-level Full TimeRemote - US R20h ago
-
AI Integrations Staff Engineer USD 150K-230KAI Models | Agent Harness | Agent SDK | Back-end | Back-end development401k contribution | Company retreats | Dental insurance | Employee referral program | EquitySenior-level Full TimeRemote R22h ago
-
Senior Data Engineer-JT0224 USD 115K-175K.Net Core | .Net Framework | AWS CloudFormation | Apache Airflow | Azure401k match | Career growth opportunities | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R1d ago
-
Senior Machine Learning Engineer USD 140K-224KAIOps | AWS | Apache Spark | Cloud platform | Data Analysis401k match | Dental insurance | Flexible paid time off | Life insurance | Medical insuranceSenior-level Full TimeRemote, United States R1d ago
-
Staff Analytics Engineer USD 145K-190KCI/CD | Cloud Data | Cloud Data Warehouse | DBT | Data Contracts401k | Dental insurance | Equity | Fertility treatment benefit | Health insuranceSenior-level Full TimeRemote - United States R1d ago
-
Senior WEM AQM Prompt Engineer USD 88K-247KA/B | A/B Testing | API Integration | B testing | CalibrationInclusive environment | Remote work | Team-first cultureSenior-level Full TimeUnited States (Remote) R1d ago
-
Analytics Engineer USD 195K-264KAirflow | Cloud Data | Cloud Data Warehouse | DBT | Data ModelingCommuter stipend | Dental coverage | Flexible PTO | Lunch and dinner stipends | Medical coverageSenior-level Full TimeSan Francisco, CA & New York, … R1d ago
-
Cloud DevOps Engineer, Data & Analytics USD 195K-250KAWS | Azure | Bash | CI/CD | Disaster RecoveryHybrid work | Remote 2 days per week | Supportive collaborative environmentMid-level Full TimeAtlanta; Boston; Charlotte; Chicago; Dallas; Los … R1d ago
-
Staff Designated Support Engineer USD 122K-160KAI workflows | AWS | As-a-Service | Azure | Cloud DataSenior-level Full TimeUS-TX-Remote R1d ago
-
Senior Machine Learning Engineer (Fraud) CAD 153K-213KAirflow | Apache Spark | Batch Processing | CatBoost | DaskDental coverage | Employee Stock Purchase Plan (ESPP) | Flexible spending wallets | Health coverage | Remote work optionSenior-level Full TimeRemote Canada R1d ago
-
AI Foundry | Agent systems | Anthropic Claude | Azure AI | Azure AI FoundryCaregiver leave | Continuous development support program | Employee assistance program | Flexible paid time off | Parental leaveMid-level Full TimeMississauga R1d ago
-
Senior, Machine Learning Engineer - 3D Perception USD 177K-212K3D Perception | Camera-based perception | Cloud processing | Computer Vision | Data Preparation401k employer match | Accidental death and dismemberment insurance | Flexible schedule | Holiday office closures | Life insuranceSenior-level Full TimeRemote, U.S, Ann Arbor, MI, Fort … R1d ago
-
Principal Data Engineer USD 135K-180KAWS | Airflow | Auditability | CI/CD | DBTCell phone allowance | Comprehensive health coverage | Equity grants | Home office setup allowance | Performance bonusesSenior-level Full TimeRemote US R1d ago
-
Azure ML | Batch Processing | Cloud Machine Learning | Data Pipelines | Data Versioning401k employer match | Discretionary paid time off | Emotional & mental wellness support | Employee resource groups | Fitness programsMid-level Full TimeScottsdale, Arizona, United States R1d ago
-
Senior Software Engineer II (TASER Data Science) USD 148K-237KAzure ML | Batch Processing | Data Engineering | Data Pipelines | Databricks401k employer match | Emotional & mental wellness support | Employee resource groups | Fitness programs | Learning and development programsMid-level Full TimeWashington, United States R1d ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R1d ago
-
Storage Operations Engineer - Multiple Openings USD 80K-110KAnsible | Bash | Ceph | Chef | Collectd401k match | Dental insurance | Gym reimbursement | Health insurance | Holiday payMid-level Full TimeRemote - United States R1d ago
-
Machine Learning Engineer (Platform) USD 156K-211KAWS | CPU Optimization | Data Processing | Data throughput | Distributed TrainingRemote workSenior-level Full TimeRemote-US R1d ago