Data Engineering Manager
Tasks
- Collaborate with BI, Data Science, and Product teams
- Design tenant specific transformation logic
- Develop PySpark ETL frameworks
- Drive data observability monitoring and alerting
- Implement data validation frameworks
- Implement governance version control and rollback
- Improve cost transparency with DBU tracking
- Manage CI CD pipelines and Databricks Asset Bundles
- Mentor and lead distributed engineering team
- Own Databricks Lakehouse architecture
- Scale multi tenant ingestion pipelines
- Support customer onboarding with scalable transformations
- Tune performance with clustering and partitioning
Perks/Benefits
- N/A
Skills/Tech-stack
Alerting | Apache Spark | Azure Cosmos | Azure Cosmos DB | Azure Table | Azure Table Storage | CI/CD | Clustering | Cosmos DB | DBU Tracking | Data Modeling | Data Observability | Databricks | Delta Lake | Infrastructure as Code | Monitoring | Partitioning | PySpark | SQL | Serverless computing | Table Storage | Unity Catalog | “as-code”
Education
N/A
Related jobs
-
Apache Hudi | Apache Iceberg | Cloud Native | Containerization | Data Ingestion401k match | Learning stipend | Paid time off | Wellness stipendMid-level Full TimeSunnyvale, California, United States8h ago
-
Senior Manager, Data Engineering - Valorant USD 178K-249KAWS | Apache Airflow | Apache Spark | CCPA | DBT401k match | Dental insurance | Flexible work schedules | Life insurance | Medical insuranceSenior-level Full TimeLos Angeles, USA9h ago
-
APIs | Chart of Accounts | Chart of Accounts design | Data Governance | Data LineageSenior-level Full TimeCentral, HK, HK9h ago
-
Head of Marketing Operations and GTM Analytics USD 160K-240KAttribution Modeling | BI Dashboards | BigQuery | Campaign tracking | DBT401k plan | Dental insurance | Flexible vacation | Health insurance | Life insuranceExecutive-level Full TimeNew York, NY9h ago
-
Lead Data Architect USD 173K-276KAI metadata | Cloud Data | Cloud Data Platforms | Data Architecture | Data Governance401k | Dental insurance | FSA | HSA | Life insuranceExecutive-level Full TimeBellevue, WA, United States10h ago
-
Engineering Manager of Managers, Data Platform USD 280K-420KAmazon S3 | Apache Airflow | Apache Celeborn | Apache Iceberg | Apache Spark401k plan | Company bonus | Equity | Medical, dental, vision benefits | Wellness stipendsMid-level Full TimeSeattle, San Francisco12h ago
-
Head of Data & Analytics USD 190K-250KAmazon Redshift | BigQuery | Common table expressions | Cost monitoring | DBT401k plan | Clubs and team lunches | Dental insurance | Education stipend | Executive coaching and career development programsExecutive-level Full TimeNew York, New York, United States12h ago
-
Senior Manager, Data Engineering & Architecture USD 214K-285KAirflow | DBT | Data Governance | Data Lineage | Data ModelingBenefits package | Hybrid work environmentSenior-level Full TimeHQ - American Fork, Utah13h ago
-
Senior Product Manager, Data Platform USD 170K-190KAI | Access Control | Business Intelligence | Compute Cost Optimization | Cost OptimizationCollaborative team | Remote-first environmentSenior-level Full TimeRemote - United States R14h ago
-
Manager of Product Analytics USD 145K-220KA/B | A/B Testing | AWS | B testing | BI tools401k match | Accidental insurance | Dental insurance | Fitness reimbursement | Flexible PTOSenior-level Full TimePlymouth Meeting, PA14h ago
-
Data Engineering Architect USD 120K-164KAPI Integration | AWS Glue | Amazon Kinesis | Apache Flink | Apache KafkaBonus program | Disability insurance | Employee assistance program | Health insurance | Life insuranceSenior-level Full TimeOak Brook, IL15h ago
-
Staff FE Engineer - Inference USD 175K-275KAPIs | Batch Processing | Data Modeling | Distributed Systems | GraphQLHigh-impact work | Job stability | Low bureaucracy | Mentorship opportunities | Small team environmentSenior-level Full TimeSunnyvale CA or Toronto Canada16h ago
-
Senior Lead Application Developers USD 116K-207KAWS Athena | AWS Cloud | AWS Cloud Development Kit | AWS EC2 | AWS GlueSenior-level Full TimeParsippany, NJ, United States16h ago
-
Associate, Commercial Analytics USD 113K-208KAI tools | Amazon Redshift | Asana | Automation | Business AnalyticsSenior-level Full TimeLos Angeles16h ago
-
Product Owner - AI/ML Platform USD 90K-109KData Analysis | Data Validation | Data cleaning | Excel | LLM401k matching | Flex PTO | Hybrid work environment | Life insurance | Medical/Dental/Vision insuranceMid-level Full TimeAtlanta, GA; Charlotte, NC; Denver, CO17h ago
-
Manager, Marketing Analytics - Migraine USD 137K-223KData Analysis | Data Modeling | Forecasting | Machine Learning | Marketing Mix401k | Dental insurance | Medical insurance | Paid time off | Short Term Incentive ProgramMid-level Full TimeFlorham Park, NJ, United States19h ago
-
Lead Data Scientist USD 170K-215KA/B | A/B Testing | Airflow | B testing | BigQueryCo-working stipend | Health and dental benefits | Mental health budget | Open and inclusive culture | Parental leave top-upSenior-level Full TimeNorth America | Utah23h ago
-
Manager, Supply Chain Reporting & Analytics USD 86K-114KAdobe Analytics | Databricks | Excel | Google Analytics | Microstrategy401k plan with employer match | Adoption support | Basic life insurance | Dental insurance | Education assistanceMid-level Full TimePortland, OR, US1d ago
-
Senior Manager, Business Analytics USD 90K-130KAccounting | Data Analysis | Data Validation | Database Management | Financial Systems401k matching | Career growth opportunities | Company holidays | Dental insurance | Medical insuranceSenior-level Full TimeNORTH BETHESDA, MD, United States R1d ago
-
Principal Product Manager – Health Data Platform USD 120K-138KAI Machine Learning Pipelines | AI machine learning | API | Canonical Data Model | Cloud ArchitectureAllyship and inclusion communities | Caregiver leave | Continuous development support program | Employee assistance program | Employee recognitionSenior-level Full TimeRemote - US R1d ago
-
AI-enabled | AI-enabled analytics | AI/ML | AI/ML concepts | Cloud DataDisability coverage | Discounted tuition | Flexible paid time off | Flexible spending account | Health savings accountSenior-level Full TimeSalt Lake City Office, United States1d ago
-
Engineering Lead Analyst - VP USD 125K-188KAI Automation | AI coding | AI coding assistant | CI/CD | Cloud401k | Medical/Dental/Vision | Paid time offSenior-level Full Time6400 LAS COLINAS BLVD IRVING, United …1d ago
-
Sr. AI Engineering Manager USD 114K-194KAPI Development | Amazon Web Services | Auditability | Azure | CI/CD401k matching | Career Development Programs | Dental insurance | Employee assistance program | Employee stock purchase programSenior-level Full TimeUSA-MI-Ann Arbor-KLA, United States1d ago
-
Sr. AI Engineering Manager USD 114K-194KAPIs | AWS | Application development | Artificial Intelligence | Auditability401k matching | Dental | Employee Assistance Program (EAP) | Employee stock purchase program | Family care and bonding leaveSenior-level Full TimeUSA-MI-Ann Arbor-KLA, United States1d ago
-
Data Scientist Lead, Amazon Music - DISCO USD 159K-247KAnomaly Detection | Customer Segmentation | Dashboards | Data Pipelines | ExperimentationSenior-level Full TimeSunnyvale, California, USA1d ago