Senior Data Engineer - Databricks
Tasks
- Apply CI CD best practices to pipeline delivery
- Apply multi tenant data isolation
- Automate ETL ELT pipeline transitions
- Design Delta Lake architecture
- Enforce data security and access control
- Implement data quality monitoring and observability
- Integrate pipelines with product ecosystem
- Maintain SLA performance metrics
- Maintain technical documentation and runbooks
- Migrate legacy ETL to Databricks
- Monitor pipelines and respond to incidents
- Onboard customers by provisioning tenant data pipelines
- Optimize Databricks compute costs and throughput
- Own Databricks production support
- Participate in on-call incident response
Perks/Benefits
- 401k match
- Career development program
- Corporate Benefit Program
- Educational resources
- Employee Referral Bonus Program
- Financial planning services
- Health and wellness reimbursement
- Healthcare package
- Legal services online
- Paid parental leave
- Pet insurance discount
- Travel discounts
- Unlimited paid time off
Skills/Tech-stack
AWS | Access Control | Apache Spark | Azure | CI/CD | Data Governance | Data Quality | Data Security | Data quality monitoring | Databricks | Delta Lake | Incremental processing | Informatica | Multi-tenant | Multi-tenant architecture | Observability | PostgreSQL | PySpark | Python | Quality monitoring | Role-Based Access | Role-Based Access Control | SQL | SSIS | Secrets management | Streaming processing | Tenant architecture
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R4d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R17d ago
-
Batch Processing | Bias Mitigation | C++ | Data Processing | Experimentation401k employer match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R8h ago
-
Principal Engineer - Data Platform USD 221K-387KAccess Control | Apache Spark | Audit Logging | CDC | CompactionSenior-level Full TimeSanta Clara, California, United States R8h ago
-
Senior-level Full TimeDeerfield, IL, United States R12h ago
-
AWS | AWS Config | Checkov | CloudTrail | CloudWatchDomestic travel up to 100 percent | On-call support | Remote workMid-level Full TimeRemote - USA R14h ago
-
Azure Data | Azure Data Factory | Azure DevOps | Azure Synapse | Azure Synapse Analytics401k plan | Company holidays | Floating holidays | Paid time off | Remote workSenior-level Full TimeUnited States R17h ago
-
Principal AI Platform Engineer (US) USD 125K-200KAuthentication | Authorization | CI/CD | Container Security | Cost OptimizationRemote workSenior-level Full TimeRemote, USA R1d ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R1d ago
-
Sr. Data Engineer USD 109K-145KAWS | Agile | CI/CD | Compliance | Data Governance401k matching | Employee assistance program | Flexible working hours | Learning platform access | Medical, dental, and vision insuranceSenior-level Full TimeRemote, TX, United States R1d ago
-
Edge AI Engineer USD 100K-150KBenchmarking | Bias Evaluation | C++ | Core ML | Edge inferenceCareer growth | Direct W2 employment | Remote work | Technical coding assessmentSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAccelerator hardware | Computer Vision | Data Quality | Data quality monitoring | Deep learningCareer growthMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAccelerator hardware | Computer Vision | Data Quality | Data labeling | Data quality monitoringCareer growth | Equal opportunity employment | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Computer Vision | Data Quality | Data quality monitoringKnowledge sharing | Long-term career growth | Mentorship | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAirflow | Apache Flink | Apache HBase | Apache Hive | Apache Hudi100 percent remote | Career growth | Full-time W2 employmentSenior-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAirflow | Apache Flink | Apache Hive | Apache Hudi | Apache IcebergSenior-level Full TimeUnited States - Remote R1d ago
-
AI Data Engineer USD 100K-150KActive Learning | Apache Beam | Apache Spark | CI/CD | Code reviewCareer growth | Health benefits | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
AI Data Engineer USD 100K-150KApache Beam | CI/CD | Data Ingestion | Data Lineage | Data ModelingRemote workMid-level Full TimeUnited States - Remote R1d ago
-
AI Data Engineer USD 100K-150KApache Beam | Apache Spark | CI/CD | Caching | Code review100 percent remote | Benefits | Career growth potentialMid-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Data Engineer USD 88K-102KAWS | Data Lake | Data Modeling | Data Transformation | Data WarehouseEmployee discounts | Health care options | Paid leave | Remote work option | Retirement contributionsMid-level Full TimeIthaca (Main Campus), United States R1d ago
-
LLM Engineer USD 100K-150KAdapter-Tuning | Benchmarking | DPO | Distributed Training | Efficient AttentionCareer growth | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
LLM Engineer USD 100K-150KAdapter-Tuning | Attention Optimization | Benchmarking | DPO | Distributed TrainingMid-level Full TimeUnited States - Remote R1d ago