Data Engineering Manager
Tasks
- Collaborate with BI, Data Science, and Product teams
- Design tenant specific transformation logic
- Develop PySpark ETL frameworks
- Drive data observability monitoring and alerting
- Implement data validation frameworks
- Implement governance version control and rollback
- Improve cost transparency with DBU tracking
- Manage CI CD pipelines and Databricks Asset Bundles
- Mentor and lead distributed engineering team
- Own Databricks Lakehouse architecture
- Scale multi tenant ingestion pipelines
- Support customer onboarding with scalable transformations
- Tune performance with clustering and partitioning
Perks/Benefits
- N/A
Skills/Tech-stack
Alerting | Apache Spark | Azure Cosmos | Azure Cosmos DB | Azure Table | Azure Table Storage | CI/CD | Clustering | Cosmos DB | DBU Tracking | Data Modeling | Data Observability | Databricks | Delta Lake | Infrastructure as Code | Monitoring | Partitioning | PySpark | SQL | Serverless computing | Table Storage | Unity Catalog | “as-code”
Education
N/A
Related jobs
-
Apache Hudi | Apache Iceberg | Cloud Native | Containerization | Data Ingestion401k match | Learning stipend | Paid time off | Wellness stipendMid-level Full TimeSunnyvale, California, United States7h ago
-
Senior Manager, Data Engineering - Valorant USD 178K-249KAWS | Apache Airflow | Apache Spark | CCPA | DBT401k match | Dental insurance | Flexible work schedules | Life insurance | Medical insuranceSenior-level Full TimeLos Angeles, USA7h ago
-
Lead Data Architect USD 173K-276KAI metadata | Cloud Data | Cloud Data Platforms | Data Architecture | Data Governance401k | Dental insurance | FSA | HSA | Life insuranceExecutive-level Full TimeBellevue, WA, United States9h ago
-
Senior Manager, Data Engineering & Architecture USD 214K-285KAirflow | DBT | Data Governance | Data Lineage | Data ModelingBenefits package | Hybrid work environmentSenior-level Full TimeHQ - American Fork, Utah11h ago
-
Data Engineering Architect USD 120K-164KAPI Integration | AWS Glue | Amazon Kinesis | Apache Flink | Apache KafkaBonus program | Disability insurance | Employee assistance program | Health insurance | Life insuranceSenior-level Full TimeOak Brook, IL14h ago
-
Staff FE Engineer - Inference USD 175K-275KAPIs | Batch Processing | Data Modeling | Distributed Systems | GraphQLHigh-impact work | Job stability | Low bureaucracy | Mentorship opportunities | Small team environmentSenior-level Full TimeSunnyvale CA or Toronto Canada14h ago
-
Senior Lead Application Developers USD 116K-207KAWS Athena | AWS Cloud | AWS Cloud Development Kit | AWS EC2 | AWS GlueSenior-level Full TimeParsippany, NJ, United States14h ago
-
Associate, Commercial Analytics USD 113K-208KAI tools | Amazon Redshift | Asana | Automation | Business AnalyticsSenior-level Full TimeLos Angeles15h ago
-
Manager, Supply Chain Reporting & Analytics USD 86K-114KAdobe Analytics | Databricks | Excel | Google Analytics | Microstrategy401k plan with employer match | Adoption support | Basic life insurance | Dental insurance | Education assistanceMid-level Full TimePortland, OR, US1d ago
-
Principal Product Manager – Health Data Platform USD 120K-138KAI Machine Learning Pipelines | AI machine learning | API | Canonical Data Model | Cloud ArchitectureAllyship and inclusion communities | Caregiver leave | Continuous development support program | Employee assistance program | Employee recognitionSenior-level Full TimeRemote - US R1d ago
-
AI-enabled | AI-enabled analytics | AI/ML | AI/ML concepts | Cloud DataDisability coverage | Discounted tuition | Flexible paid time off | Flexible spending account | Health savings accountSenior-level Full TimeSalt Lake City Office, United States1d ago
-
Engineering Lead Analyst - VP USD 125K-188KAI Automation | AI coding | AI coding assistant | CI/CD | Cloud401k | Medical/Dental/Vision | Paid time offSenior-level Full Time6400 LAS COLINAS BLVD IRVING, United …1d ago
-
Sr. AI Engineering Manager USD 114K-194KAPI Development | Amazon Web Services | Auditability | Azure | CI/CD401k matching | Career Development Programs | Dental insurance | Employee assistance program | Employee stock purchase programSenior-level Full TimeUSA-MI-Ann Arbor-KLA, United States1d ago
-
Sr. AI Engineering Manager USD 114K-194KAPIs | AWS | Application development | Artificial Intelligence | Auditability401k matching | Dental | Employee Assistance Program (EAP) | Employee stock purchase program | Family care and bonding leaveSenior-level Full TimeUSA-MI-Ann Arbor-KLA, United States1d ago
-
Data Scientist Lead, Amazon Music - DISCO USD 159K-247KAnomaly Detection | Customer Segmentation | Dashboards | Data Pipelines | ExperimentationSenior-level Full TimeSunnyvale, California, USA1d ago
-
Associate Director, Commercial Analytics USD 189K-221KAnalytics | BI platforms | Claims data | Data Analysis | Data StorytellingMid-level Full TimeRadnor, United States1d ago
-
API Development | AWS | Azure | Cloud | ComplianceExecutive-level Full TimePhiladelphia, Pennsylvania, United States1d ago
-
Data Engineering | Data Modeling | Databricks | Machine Learning | Power BITravel to client locationsSenior-level Full TimeColumbus, Ohio, United States1d ago
-
Lead Workstation Agentic AI Engineer USD 120K-160KAgentic AI | Automated testing | Automation | Business Intelligence | CI/CDContinuing education program | Continuous learning resources | Family-friendly perks | Financial wellness programs | Flexible time offSenior-level Full TimeUS - NJ - PRINCETON ONE …2d ago
-
Chief of Reporting and Analytics USD 106K-118KBig Data | Data Visualization | Geospatial Analytics | Oracle | SQLDeferred compensation | Defined pension plan | Health insuranceExecutive-level Full TimeNew York City, NY, United States2d ago
-
Adobe Campaign | Automated testing | Azure | Azure Data | Azure Data FactorySenior-level ContractSan Francisco, United States2d ago
-
Data Analyst, Product (Mandarin Speaker) USD 167K-312KBusiness Intelligence | Data Analysis | Data Modeling | Data Product Management | Data VisualizationMid-level Full TimeSan Jose, California, United States2d ago
-
Data Science Manager USD 208K-235KDescriptive Statistics | Experimentation | Forecasting | Inferential Statistics | MATLABMid-level Full TimeMenlo Park, CA2d ago
-
Lead SAP Data Engineer - IT USD 105K-152KCDS Views | Data Governance | Data Lineage | Data Modeling | Data PipelinesAdoption Assistance | Education assistance | Health and life insurance | Paid Holidays | Paid time offSenior-level Full TimeLibertyville, IL, US, 60048-37813d ago
-
Associate Technical Architect - DE - Databricks USD 121K-202KAWS S3 | AWS Secrets | AWS Secrets Manager | AWS Step Functions | AirflowCareer development | Innovation-focused culture | Learning and growth | Work-life balanceMid-level Full TimeUSA - Remote, United States R3d ago