Senior Data Engineer - Databricks
Tasks
- Apply CI CD best practices to pipeline delivery
- Build automation for data delivery
- Design Delta Lake architecture
- Enforce data security and access control
- Implement data quality monitoring and observability
- Implement multi tenant data isolation
- Ingest and transform ERP and CRM data at scale
- Integrate with product ecosystem pipelines
- Maintain SLA performance metrics
- Maintain documentation runbooks and ADRs
- Migrate legacy ETL ELT to Databricks
- Monitor pipelines and respond to incidents
- Optimize Databricks compute costs
- Own Databricks production support
- Participate in on-call rotation
- Provision and validate multi tenant tenant pipelines
Perks/Benefits
- 401k match
- Educational resources
- Health insurance
- Legal services
- Paid Wellness Reimbursement
- Paid parental leave
- Referral bonus
- Travel discounts
- Unlimited paid time off
Skills/Tech-stack
ACID transactions | AWS | Access Control | Apache Spark | Azure | CI/CD | Data Governance | Data Observability | Data Security | Databricks | Delta Lake | Delta Lake Schema | Delta Lake schema evolution | PostgreSQL | PostgreSQL Query Optimization | PySpark | Python | Query Optimization | Role-Based Access | Role-Based Access Control | SQL | Schema evolution | Secrets management | Unity Catalog
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R28d ago
-
Senior, Data Analyst/BI Engineer USD 108K-145KAWS | Airflow | Amazon Redshift | BigQuery | DataStage401k matching | Healthcare dental vision coverage | PTO | Paid Company Holidays | Paid parental leaveSenior-level Full TimeUnited States R7h ago
-
AI Agents | Agent Orchestration | Code Execution | Enterprise systems | GolangFlexible work environment | Remote flexibilitySenior-level Full TimeMountain View, CALIFORNIA, United States R11h ago
-
Staff Machine Learning Engineer, Agentic App Platform USD 166K-291KAPI Design | Cloud Computing | Concurrency | Debugging | Distributed Systems401k match | ESPP | Family leave | Flexible spending accounts | Flexible time offSenior-level Full TimeMountain View, CALIFORNIA, United States R12h ago
-
Senior AI Engineer (NYC) USD 144K-236KA/B | A/B Testing | B testing | Deep learning | Fine TuningHealth and wellness programs | Hybrid work flexibility | Mentorship | Paid time offSenior-level Full TimeNew York, NY, United States R12h ago
-
Software Engineer, Data Platform Team USD 152K-179KAI | Apache Airflow | BI | Data Quality | DatabricksMid-level Full TimeRemote - USA R12h ago
-
Applied AI Engineer USD 122K-176KAI Agents | AWS | Automated testing | Azure | CI/CD401k match | Educational assistance program | Employee assistance program | Employee discounts | Employee referral programMid-level Full TimeRemote (United States) R13h ago
-
Staff Analytics Engineer, Compliance Data USD 207K-244KAirflow | DBT | Data Contracts | Data Modeling | Data QualityQuarterly in person surges | Remote-first work environmentSenior-level Full TimeRemote - USA R13h ago
-
Sr. Software Engineer, Customer Experience Engineering USD 200K-287KAPI Design | Cloud Computing | Code review | Container Services | DebuggingSenior-level Full TimeUS-CA-Menlo Park R14h ago
-
Data Engineer USD 100K-120KData Quality | Data Reconciliation | Data Validation | Data Warehousing | ELT401k employer match | Adoption support | Calm app access | Company holidays | Fertility supportMid-level Full TimeRemote - US R14h ago
-
Software Engineer, Cost Optimization (Infrastructure) USD 130K-215KAWS | Caching | ClickHouse | Cost Explorer | Cost and Usage Reports100 Percent Healthcare Coverage | 401k | Generous vacation policy | Mobile reimbursement | PensionMid-level Full TimeUS - New York, NY R14h ago
-
Data Scientist USD 84K-110KAnomaly Detection | CI/CD | Data Privacy | Databricks | Drift monitoringDiversity and inclusion working groups | Flexible working practices | Paid volunteer time off VTO | Parental leaveMid-level Full TimeAtlanta, GA - Remote R14h ago
-
AI Prompt Engineer / AI Application Engineer USD 84K-110KAgentic Workflows | Anthropic Claude | Authentication | Azure DevOps | CI/CDEntry-level Full TimeNew York, NY, United States R15h ago
-
Senior AI Engineer, VideoAI USD 144K-236KA/B | A/B Testing | Apache Spark | B testing | Data MiningHealth and wellness programs | Hybrid work schedule | Time away from workSenior-level Full TimeMountain View, CA, United States R16h ago
-
Applied AI Engineer, Enterprise Tech USD 200K-320KAI Product Development | Agent Development | Architecture Design | Deployment | LLM EvaluationFlexible working hours | Generous vacation | Optional equity donation matching | Parental leave | Visa sponsorshipMid-level Full TimeSan Francisco, CA | New York … R16h ago
-
Robotics Software Engineer USD 130K-200K3D sensors | C++ | Closed Loop | Closed loop control | Communication Protocols401k matching | Company equity | Dental insurance | Flexible paid time off | Medical insuranceMid-level Full TimePittsburgh R16h ago
-
Data & ML Engineer USD 150K-200KAWS Glue | Abstention Policy | Apache Airflow | Apache Kafka | Apache NiFiCareer growth opportunity | Employer-paid health insurance | Fully remote | Paid parental leave | Unlimited PTOMid-level Full TimeRemote, USA R16h ago
-
Data & ML Engineer USD 150K-200KAWS Glue | Airflow | Apache Kafka | Apache Spark | Audit LoggingEmployer-paid health insurance | Flexible work environment | Fully remote | Paid parental leave | Unlimited PTOMid-level Full TimeRemote, USA R16h ago
-
Staff Data Engineer USD 170K-220KAPIs | AWS | Airflow | Alerting | CI/CD401k | Dental insurance | Fertility treatment benefit | Health insurance | Life insuranceSenior-level Full TimeRemote - United States R17h ago
-
Senior-level Full TimeBellevue R18h ago
-
Senior Data Engineer USD 196K-230KAWS | Alerting | Apache Airflow | Apache Spark | CI/CD401k match | Disability insurance | Flexible time off | Health, dental, and vision insurance | Life insuranceSenior-level Full TimeUnited States R18h ago
-
AI Search | AI Security | AWS | AWS Bedrock | Agent memorySenior-level Contract Full TimeJersey City, NJ, United States R18h ago
-
Principal Engineer, Embedded Vehicle Software USD 347K-457KAUTOSAR | Abstraction layer | Automotive Software | C plus plus | C#Senior-level Full TimeSunnyvale R19h ago
-
Staff Software Engineer, Data Infrastructure USD 232K-386KAmazon Kinesis | Amazon S3 | Apache Airflow | Apache Druid | Apache Flink401k matching | Flexible time off | Learning and development stipends | Parental leaveSenior-level Full TimeSan Francisco R19h ago
-
Machine Learning Engineer II (Underwriting ML) USD 165K-225KAirflow | CatBoost | Dask | Kubeflow | LightGBMDental and vision coverage | ESPP | Flexible spending wallets | Health coverage | Remote-first flexibilityMid-level Full TimeRemote US R20h ago