Senior Data Engineer - Databricks
Tasks
- Apply CI CD best practices for pipeline delivery
- Automate data delivery and reduce manual intervention
- Build Databricks pipelines for ERP and CRM data
- Design Delta Lake schema and partitioning
- Enforce data security and access control
- Ensure multi-tenant data isolation
- Implement data quality monitoring and observability
- Integrate data pipelines with product ecosystem
- Maintain runbooks and technical documentation
- Migrate legacy ETL pipelines to Databricks
- Monitor and respond to data incidents
- Onboard and provision tenant data pipelines
- Optimize Databricks pipelines to reduce compute cost
- Own Databricks production support
- Provide on-call incident response
- Track data pipeline SLA performance
Perks/Benefits
- 401k match
- Corporate Benefit Program
- Discounted pet insurance
- Educational resources
- Employee Referral Bonus Program
- Financial planning services
- Health and wellness reimbursement
- Healthcare package
- Online Legal Services
- Paid parental leave
- Paid travel discounts
- Unlimited paid time off
Skills/Tech-stack
AWS | Access Control | Amazon Web Services | Apache Spark | Automated testing | Azure | Azure Databricks | CI/CD | Compliance | Data Governance | Data Quality | Data isolation | Databricks | Delta Lake | Delta Lake ACID | Deployment tooling | ELT | ETL | Incremental processing | Informatica | Multi-tenant | Multi-tenant architecture | Observability | PostgreSQL | PySpark | Python | Role-Based Access | Role-Based Access Control | SQL | SSIS | Schema evolution | Secrets management | Streaming processing | Tenant architecture | Version control | Web Services
Education
N/A
Roles
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R3d ago
-
AWS Bedrock | Agent systems | Anthropic API | Autogen | Azure401k matching program | Adoption Assistance | Development and career growth opportunities | Fertility treatments | Flexible work schedulesSenior-level Contract Full TimeRemote, OR, United States R14h ago
-
Software Engineer, Applied AI USD 190K-280KContext engineering | Data Processing | Data Storage | Debugging | Docker401k match | Dental insurance | Health insurance | Hybrid work model | Professional developmentSenior-level Full TimeNew York City R16h ago
-
Data Engineer USD 74K-133KAgile | Apache Airflow | BigQuery | Cloud Composer | Cloud Data401k retirement plan | Dental insurance | Disability insurance | Flexible time off | Health insuranceMid-level Full TimeLisle, IL, United States R16h ago
-
Staff Data Engineer USD 185K-220KAWS | Apache Airflow | Apache Kafka | Benthos | Big DataDental insurance | Disability insurance | Flexible work hours | Health insurance | Health savings accountSenior-level Full TimeRosslyn, VA or Remote R17h ago
-
API Testing | Cypher | Data Quality | DataOps | DevOpsBenefits | Competitive pay | Growth opportunity | Remote work | Travel requiredSenior-level Full TimeReston, VA, United States R18h ago
-
Principal Engineer - Data Platform USD 221K-387KAWS | Airflow | Apache Hive | Apache Iceberg | Apache ImpalaRemote workSenior-level Full TimeSanta Clara, California, United States R19h ago
-
IT Infrastructure Engineer II USD 139K-178KAWS Organizations | Access Management | Amazon Web Services | Ansible | Application FirewallHealth and wellness resources | Remote work | Wellness FridaysSenior-level Full TimeRemote - United States R19h ago
-
Data Manipulation | Distributed Systems | Embeddings | Java | KubernetesCollaborative flat culture | Direct access to technical leadership | Exposure to cutting edge generative AI | Flexible schedule | High autonomyEntry-level Full TimeCanada R1d ago
-
Cloud Data | Cloud Data Platforms | DBT | Data Architecture | Data ProcessingCoworking allowance | Flexible paid time off | Health, dental and vision coverage | Home office stipend | Mental health resourcesSenior-level Full TimeCanada R1d ago
-
Azure Data | Azure Data Lakehouse | Azure SQL | DBT | Data GovernanceCareer growth | Continuous learning | Dental insurance | Flexible work arrangements | Global collaborationSenior-level Full TimeCanada R1d ago
-
API Integration | CRM | Context Management | Distributed Systems | Document stores401k retirement plan | Company offsite and team events | Flexible PTO | Fully remote | Health, dental, and vision insuranceSenior-level Full TimeCanada R1d ago
-
AWS Glue | AWS Lambda | AWS S3 | Access Control | Data GovernanceCareer growth opportunities | Collaborative and inclusive work environment | Diverse and inclusive culture | Flexible work arrangements | Permanent remote working modelSenior-level Full TimeCanada R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Edge AI Engineer USD 100K-150KC++ | Core ML | Cross Platform Inference | Cross-platform | DSPCareer growth potential | Full-time remote work | H1B transfer supportSenior-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Data Quality | Data labeling | Data quality monitoring100 percent remote | Career growth | Full-time employment | W2 employmentMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Agentic Systems | Computer Vision | Data QualityMid-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache SparkRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HiveRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
AI Data Engineer USD 100K-150KActive Learning | Apache Beam | CI/CD | Caching | Code review100 percent remote | Career growth | Full-time employment | H1B transfer support | W2 employmentMid-level Full TimeUnited States - Remote R1d ago
-
Engineer – Data Engineer III USD 86K-123KDimensional Modeling | Informatica PowerCenter | Perl | Python | SQLMentorship programs | Paid caregiver leave | Paid parental leave | Training programs | Volunteer activitiesSenior-level Full TimeUSA - PA - Conshohocken - … R1d ago
-
AI Data Engineer USD 100K-150KApache Beam | CI/CD | Code review | Data Lineage | Data Modeling100 percent remote | Career growthMid-level Full TimeUnited States - Remote R1d ago
-
LLM Engineer USD 100K-150KAdapter methods | DPO | Deep reinforcement learning | Distributed Training | Efficient AttentionBenefits | Career growth | Mentorship | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
LLM Engineer USD 100K-150KDPO | Deep learning | Distributed Training | Efficient Attention | Efficient Fine TuningRemote workMid-level Full TimeUnited States - Remote R1d ago
-
Prompt Engineer USD 100K-150KAgent architecture | Agent architectures | Agentic Workflows | Chunking | Deterministic systemsLong-term engagement | Mentorship | Remote workMid-level Full TimeUnited States - Remote R1d ago