Sr Machine Learning Engineering Manager - AI Quality and Governance
Tasks
- Build AI governance traceability and approval workflows
- Define AI quality strategy
- Define operational readiness metrics
- Design scalable evaluation platform
- Detect regressions and model drift
- Drive incident response and root cause analysis
- Implement offline and online evaluation
- Lead multidisciplinary engineering team
- Manage evaluation datasets and benchmarks
- Set quality bars and automated quality gates
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | AWS | Adversarial Testing | Agents | Azure | B testing | CI/CD | Dataset Management | Deployment Monitoring | Drift Detection | Embeddings | Experiment tracking | Generative AI | Golden Test | Golden Test Sets | Golden datasets | Google Cloud | Human evaluation | Hybrid search | Infrastructure as Code | Kubernetes | LLM | Lineage | Machine Learning | Microservices | Model Drift | Model Drift Detection | Model Monitoring | Model versioning | Prompt engineering | Prompt orchestration) | Prompt versioning | Red Teaming | Regression testing | Retrieval-Augmented Generation | Risk classification | Statistical Metrics | Telemetry | Test Sets | Tool use | Traceability | Vector Search | “as-code”
Education
Related jobs
-
API Integration | Azure | Cloud Architecture | Data analytics | DatabricksConnectivity reimbursement | Equipment provided | Remote workMid-level Full TimeUSA - OR (Remote), United States R1d ago
-
Manager Content Data Engineering USD 136K-217KCanonical Data | Canonical Data Model | Data Model | Data Modeling | Data QualityFlexible hours | Sabbatical | Shared parental leave | Study assistance | Wellbeing initiativesMid-level Full TimeHome based-North Carolina, United States R1d ago
-
Machine Learning Manager, Feed Ecosystems USD 253K-354KAI measurement | Candidate Retrieval | Cold Start | Cold Start Modeling | Content Understanding401k employer match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careMid-level Full TimeRemote - United States R1d ago
-
Machine Learning Manager, Feed Relevance (Retrieval) USD 253K-354KA/B | A/B Testing | B testing | Debugging | Embeddings401k employer match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careMid-level Full TimeRemote - United States R1d ago
-
Automated Monitoring | Backtesting | Credit Risk | Credit risk modeling | Data IntegrityDental coverage | Employee stock purchase plan | Flexible spending accounts | Health coverage | Paid time offSenior-level Full TimeRemote US R1d ago
-
Engagement Manager USD 170K-175KData Visualization | Excel | Jira | Machine Learning | Power BITelecommuting | Travel to client sitesSenior-level Full TimeNew York, United States R1d ago
-
Project Manager USD 170K-175KAgile project management | Big Data | CI/CD | Data Modeling | Data QualityTelecommuting allowedMid-level Full TimeNew York, United States R1d ago
-
Cost Optimization | Data Pipelines | Feature Engineering | Machine Learning | Machine Learning InfrastructureEquity compensationMid-level Full TimeSan Mateo, CA, United States R1d ago
-
Manager, Quant Data Analytics and Insights USD 126K-141KAPI Integration | Autosys | Back Testing | CI/CD | Data VisualizationMid-level Full Time245 Summer St, Boston MA, United … R2d ago
-
AI integration | API Design | Canonical Data | Canonical Data Modeling | Cross-functional401k match | Disability insurance | Employee assistance program | Health insurance | Life insuranceSenior-level Full TimeUtah | Hybrid R2d ago
-
Sr. Manager, AI Software Engineering USD 200K-275KAWS | CI/CD | Cloud Native | Distributed Systems | Docker401k match | Dental insurance | Employee assistance program | Fertility coverage | Flexible time offSenior-level Full TimeUnited States - Remote R2d ago
-
Business & Marketing Data Science Manager USD 240K-334KArtificial Intelligence | Cloud Machine Learning | Dashboarding | Data Engineering | Data VisualizationBenefits determined by role level and location | Hybrid scheduleSenior-level Full TimeSeattle, WA, USA R2d ago
-
Analytics | Artificial Intelligence | BigQuery | Cloud Data | Cloud data platform401k | Disability insurance | Employee stock purchase plan | Health insurance | Life insuranceMid-level Full Time#Wayne, PA, US, 19087 R2d ago
-
Manager, Platform Analytics USD 122K-154KAnalytics | Artificial Intelligence | Availability | Business Intelligence | Capacity Planning401k match | Education assistance program | Employee assistance program | Flexible time off | Hybrid workMid-level Full TimeRemote, United States R2d ago
-
VP AI ML Engineering, Medicare & Retirement USD 200K-343KAI SDLC | APIs | AWS | Application Security | Artificial IntelligenceRemote work flexibility | Work from office minimum four days per weekExecutive-level Full TimeMinnetonka, Minnesota, UNITED STATES; REMOTE R3d ago
-
Product Manager - Business Intelligence & Analytics USD 120K-150KAPI Integration | Business Intelligence | Cloud Data | Dashboarding | Data Integrity401k company match | Employee stock purchase plan | Medical/Dental/Vision | Paid Holidays | Paid parental leaveMid-level Full TimeIllinois, US Offsite, United States R3d ago
-
Senior Vice President - Data Science and Artificial Intelligence Group Manager - Hybrid USD 192K-288KA/B | A/B Testing | Agentic AI | B testing | BitbucketSenior-level Full TimeNew York, UNITED STATES; Wilmington, Delaware, … R3d ago
-
Data/AI Scientist II USD 95K-127KAnalytics Dashboards | Artificial Intelligence | Care management | Claims data | Clinical pharmacyMid-level Full TimeUS - Remote R3d ago
-
Technical Delivery Manager USD 133K-173KAWS | Agile | Amazon Redshift | Apache Airflow | Apache HadoopFlexible time zones | Ownership | Remote workMid-level Full TimeUS - Remote R3d ago
-
Engineering Manager, Machine Learning Platform USD 230K-290KData Quality | Deep learning | Deployment Workflows | Distributed Systems | Evaluation DataEmployee stock purchase plan | Flexible spending accounts | Remote work | Subsidized medical coverage | Time offSenior-level Full TimeRemote US R3d ago
-
AVP of Data Platform Services – Engineering & DevOps USD 177K-266KAzure DevOps | CI/CD | Data Architecture | Data Engineering | Data Factory401k | Generous PTO | Hybrid schedule | Longevity bonuses | Medical/Dental/VisionExecutive-level Full TimeWilkes Barre, PA, United States R3d ago
-
AI Agents | AI/ML | AWS | Agentic AI | Agile401k contribution | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programs | Remote work flexibilityMid-level Full TimeEden Prairie, Minnesota, UNITED STATES; REMOTE R4d ago
-
Amazon DynamoDB | Ansible | Apache Druid | Apache Flink | Apache HadoopAdoption leave | Employee networks | Flexible remote work | Paid parental leave | Paid time offSenior-level Full TimeAustin, United States R4d ago
-
Manager, Data Science USD 128K-276KData Analysis | Data Products | Elasticsearch | Language Processing | Machine LearningCareer growth | Family support | Health benefits | Paid time off | Retirement planMid-level Full TimeRemote - Michigan, United States R4d ago
-
Manager, Applied AI Solutions & Enablement, Marketing USD 108K-135KAPI Integration | Analytics | Artificial Intelligence | Automation | Data integration401k matching | Child care benefits | Commuter benefits | Dental insurance | Health insuranceMid-level Full TimeNew York, NY R6d ago