Senior Data Engineer - Databricks
Tasks
- Apply CI CD best practices to pipeline delivery
- Apply multi tenant data isolation
- Automate ETL ELT pipeline transitions
- Design Delta Lake architecture
- Enforce data security and access control
- Implement data quality monitoring and observability
- Integrate pipelines with product ecosystem
- Maintain SLA performance metrics
- Maintain technical documentation and runbooks
- Migrate legacy ETL to Databricks
- Monitor pipelines and respond to incidents
- Onboard customers by provisioning tenant data pipelines
- Optimize Databricks compute costs and throughput
- Own Databricks production support
- Participate in on-call incident response
Perks/Benefits
- 401k match
- Career development program
- Corporate Benefit Program
- Educational resources
- Employee Referral Bonus Program
- Financial planning services
- Health and wellness reimbursement
- Healthcare package
- Legal services online
- Paid parental leave
- Pet insurance discount
- Travel discounts
- Unlimited paid time off
Skills/Tech-stack
AWS | Access Control | Apache Spark | Azure | CI/CD | Data Governance | Data Quality | Data Security | Data quality monitoring | Databricks | Delta Lake | Incremental processing | Informatica | Multi-tenant | Multi-tenant architecture | Observability | PostgreSQL | PySpark | Python | Quality monitoring | Role-Based Access | Role-Based Access Control | SQL | SSIS | Secrets management | Streaming processing | Tenant architecture
Education
N/A
Roles
Related jobs
-
Senior-level Full TimeRemote, Canada; Computer/Software; Canada, Canada; Remote, … R3h ago
-
Cloud Software Engineer 3 - (Cloud, Hadoop, Java) USD 228K-278KAnsible | Apache Kafka | Apache Lucene | Apache Solr | Avro401k matching | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offSenior-level Full TimeFort Meade, Maryland R4h ago
-
Senior Data Engineer USD 122K-161KAPI Integration | AWS | Apache Beam | Apache Flink | Apache SparkRemote workSenior-level Full TimeUnited States R16h ago
-
AI Software Engineer USD 160K-250KAWS | Agent Orchestration | Async Python | CI/CD | Context ManagementEquity awards | Health insurance | Performance based cash bonus | Remote work opportunityMid-level Full TimeNew York, New York, United States R20h ago
-
Mid-level Full TimeUSA VA Home Office (VAHOME), United … R20h ago
-
MLOps Automation Engineer Lead USD 77K-154KAWS | AWS EKS | Ansible | Backup | Business ContinuityFlexible work arrangement | Work from homeSenior-level Full TimeEaston Ops Cols C Oh, United … R20h ago
-
Senior-level Full TimeUnited States - Remote R20h ago
-
Senior-level Full TimeUnited States - Remote R20h ago
-
Data Engineer-Nordic OMOP USD 93K-163KAWS | Azure | CI/CD | Data Marts | Data ModelingChildcare discounts | Comprehensive medical, dental and vision benefits | Continued education and training | Discounts to wellness centers | Employer-funded pension planSenior-level Full TimeRemote - VA, United States R20h ago
-
Data Engineer, Senior USD 77K-176KAgile | Azure Data | Azure Data Factory | Azure DevOps | Azure FunctionsDependent care | Paid leave | Professional development | Tuition assistance | Work-life programsSenior-level Full TimeUSA, GA, Atlanta (575 Morosgo Drive … R20h ago
-
Embedded Robotics Engineer USD 125K-145KAutonomous Vehicles | C++ | Concurrent Systems | Control | Critical softwareMentorship | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
Sr. Linux Systems Engineer - Object Storage (Remote) USD 140K-215KAWS S3 | Ansible | Chef | ELK Stack | GoEmployee networks | Paid adoption leave | Paid parental leave | Professional development opportunities | Remote workSenior-level Full TimeUSA CA Remote, United States R20h ago
-
Senior AI Engineer USD 90K-150KAPI Integration | AWS | AWS Bedrock | Agentic Workflows | Containers401k match | Health insurance | PTO | Stock options | Stock purchase planSenior-level Full TimeRemote - United States R1d ago
-
Artificial Intelligence Engineer USD 200K-250KAWS | CI/CD | Code Interpreter | Data parallelism | Deep learningMid-level Full TimeCalifornia, United States - Remote R1d ago
-
Data Engineer USD 190K-240KAWS | Airflow | CI/CD | Docker | DynamoDBCollaboration with detection engineers | Equity | Remote-first | Small high context teamSenior-level Full TimeCalifornia, United States - Remote R1d ago
-
Mid-level Full TimeCenter, Center District, IL R1d ago
-
Cloud Data Engineer USD 119K-140KAWS | Azure Data | Azure Data Factory | Cloud Architecture | Cloud platformGovernment security clearance | Remote workMid-level Full TimeAustin, TX, US R1d ago
-
Senior Computer Vision AI/ML Engineer (TS/SCI) {S} USD 130K-200KAWS EC2 | AWS Lambda | AWS S3 | Automatic Target Recognition | Computer Vision401k retirement plan | Bonus program | Dental insurance | Employee assistance program | Flexible schedulingSenior-level Full TimeAurora, Colorado, United States R1d ago
-
Senior Computer Vision AI/ML Engineer (TS/SCI) {S} USD 126K-200KAWS | Automatic Target Recognition | CI/CD | Computer Vision | Convolutional Neural Networks401k matching | Annual bonus | Employee assistance program | Flexible schedule | Life Insurance AD D Disability InsuranceSenior-level Full TimeKing Of Prussia, Pennsylvania, United States R1d ago
-
Senior DevOps Engineer- Posture USD 130K-220KAWS | ArgoCD | Bash | Blue-Green Deployment | Blue/greenSenior-level Full TimeTel Aviv-Jaffa, Tel Aviv District, IL R1d ago
-
Azure Data Engineer USD 114K-150KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Contract Full TimeUnited States - Remote R1d ago
-
Machine Learning Engineer, Voice USD 200K-300KASR | Data Pipelines | Data labeling | Deep learning | GPUsMid-level Full TimeSan Francisco R1d ago
-
Senior Robotics Research Engineer USD 160K-224KC++ | Cloud processing | Collision detection | Computer Vision | Contact Physics401k match | Dental insurance | Equity ownership | Free Lunches | Free parkingSenior-level Full TimeBerkeley, California R1d ago
-
APIs | CRM | Call Routing | Evaluation | GraphQL401k match | Childcare reimbursement | Commuter reimbursement | Dental insurance | Health insuranceEntry-level Full TimeNew York Office R1d ago
-
APIs | BigQuery | CRM automation | Data Structures | Evaluation401k matching | Childcare reimbursement | Generous parental leave policy | Internet reimbursement | Medical/Dental/Vision insuranceEntry-level Full TimeSeattle Office R1d ago