Sr Machine Learning Engineering Manager - AI Quality and Governance
Tasks
- Build AI governance traceability and approval workflows
- Define AI quality strategy
- Define operational readiness metrics
- Design scalable evaluation platform
- Detect regressions and model drift
- Drive incident response and root cause analysis
- Implement offline and online evaluation
- Lead multidisciplinary engineering team
- Manage evaluation datasets and benchmarks
- Set quality bars and automated quality gates
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | AWS | Adversarial Testing | Agents | Azure | B testing | CI/CD | Dataset Management | Deployment Monitoring | Drift Detection | Embeddings | Experiment tracking | Generative AI | Golden Test | Golden Test Sets | Golden datasets | Google Cloud | Human evaluation | Hybrid search | Infrastructure as Code | Kubernetes | LLM | Lineage | Machine Learning | Microservices | Model Drift | Model Drift Detection | Model Monitoring | Model versioning | Prompt engineering | Prompt orchestration) | Prompt versioning | Red Teaming | Regression testing | Retrieval-Augmented Generation | Risk classification | Statistical Metrics | Telemetry | Test Sets | Tool use | Traceability | Vector Search | “as-code”
Education
Related jobs
-
Manager, Global Sourcing Analytics USD 75K-115KAI-enabled | AI-enabled tools | Amadeus | Analytics automation | Artificial Intelligence401k match | Employee Discounts on Cruises | Employee assistance program | Employee stock purchase plan | Health benefitsMid-level Full TimeMiami, FL, United States R22h ago
-
AWS | Ansible | Automation | BAM | Bioinformatics401k match | Commuter benefits | Employee referral program | Fertility care benefits | Free employee genetic testingSenior-level Full TimeUS Remote R1d ago
-
Technical Product Manager, AI Engineering & Systems USD 130K-170KAPI Design | Artificial Intelligence | Chatbot | Computer use | Data Analysis401k employer matching | Dental insurance | Health insurance | Paid parental leave | Paid time offMid-level Full TimeNew York, New York R1d ago
-
API Design | Agentic AI | Artificial Intelligence | Data Analysis | Experimentation401k matching | Dental insurance | Equity opportunities | Health insurance | Paid parental leaveSenior-level Full TimeNew York, New York R1d ago
-
Sr Technical Account Manager - West USD 150K-200KAWS | Active Directory | Apache Spark | Azure | Cloud platform401k match | Commuter benefits | Dental insurance | Employer paid disability coverage | Flexible spending accountsSenior-level Full TimeUnited States, Remote R1d ago
-
Data & Analytics Engineering Manager USD 185K-200KA/B | A/B Testing | AI tools | Agile | Alert triageSenior-level Full TimeNew York, NY R1d ago
-
Sr. Manager, Field Engineering USD 192K-264KBig Data | Cloud Data | Cloud data platform | Customer Success | Data EngineeringSenior-level Full TimeChicago, Illinois; Connecticut; New Jersey; Remote … R1d ago
-
Head of Applied Machine Learning - Application Fraud USD 210K-260KAmazon Web Services | Applied statistics | Data Governance | Elasticsearch | Experimentation401k match | Company in person events | Employer-paid health insurance | Flexible paid time off | Home office stipendExecutive-level Full TimeUnited States R1d ago
-
Applied Machine Learning Manager - Application Fraud USD 200K-250KAWS | Applied statistics | Code review | Data labeling | Elasticsearch401k match | Company in person events | Employer-paid health insurance | Flexible paid time off | Home office stipendSenior-level Full TimeUnited States R1d ago
-
Manager, Alpha Analytics and Portfolio Risk USD 225K-225KAPI Integration | Alpha Beta Separation | At risk | Audit Logging | Currency hedgingHealth and well-being benefits | Hybrid workSenior-level Full TimeChicago Office, United States R1d ago
-
Manager, Data & Advanced Analytics USD 106K-152KArtificial Intelligence | Automation | Data Governance | Data Science | Data analytics401k savings plan | Flexible spending accounts | Health and wellness programs | Health savings account | Paid parental leaveMid-level Full TimeUS-Nationwide-FIELD, United States R1d ago
-
Analytics Engineering Manager USD 220K-260KAWS | Cloud Computing | DBT | Data Documentation | Data Modeling401k matching | Flexible time off | Health benefits | Remote-friendlySenior-level Full TimeUnited States (Remote) R1d ago
-
Integrated Services Analytics Product Manager USD 132K-250KAPI | Agile | Artificial Intelligence | BigQuery | Cloud NativeFamily care days | Flexible work schedule | Medical, dental & vision coverage | Paid parental leave | Remote work flexibilityMid-level Full TimeUnited States R1d ago
-
Data & Analytics Engineering Manager USD 185K-200KA/B | A/B Testing | B testing | CI/CD | DBTWellness benefitsSenior-level Full TimeNew York, NY R2d ago
-
API Integration | Agent systems | Agents | Artificial Intelligence | Cloud ComputingHybrid work policy | Visa sponsorship supportMid-level Full TimeSan Francisco, CA | New York … R2d ago
-
Data Analytics Lead USD 150K-250KAI readiness | Data Architecture | Data Modeling | Data integration | Data setsCharitable giving program | Company-Paid Holidays | Dental insurance | Flexible PTO | Life insuranceSenior-level Full TimeMassachusetts - Boston R2d ago
-
Manager of Applied AI Architecture, Commercial USD 315K-380KAgentic Architectures | Artificial Intelligence | Context engineering | Customer Engagement | Developer ToolingSenior-level Full TimeSan Francisco, CA | New York … R2d ago
-
AI Solutions Lead, Customer Onboarding USD 88K-140KAPI Integration | Agent Orchestration | Atlassian Rovo | Compliance Management | CryptographyBonuses | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, United States R2d ago
-
Product Analytics USD 150K-180KAI tools | Analytical Thinking | Customer Analytics | Data Analysis | Language Processing401-k match | Flexible time off | Health benefits | Remote-friendlySenior-level Full TimeUnited States (Remote) R2d ago
-
Analytics & Insights Manager USD 115K-218KAutoml | Cloud Storage | Data Science | Google BigQuery | Machine LearningBack up child care option | Child care support | Employee resource groups | Flexible family care days | Holiday payMid-level Full TimeDearborn, MI, United States R2d ago
-
Sr. Technology Partner Director - AI Code-Gen & Apps USD 178K-245KAs-a-Service | Business Development | CI/CD | Communication | Cross-functionalSenior-level Full TimeRemote - California; San Francisco, California R2d ago
-
Engineering Manager, AI Intake USD 220K-275KAgile | Architecture Decision | Architecture Decision Records | Autogen | Decision records401k retirement plan | Equity incentive plan | Flexible vacation policy | Health Savings Account contribution | Home office stipendMid-level Full TimeU.S. Remote R2d ago
-
Actuarial Engineering Manager USD 93K-150K.NET | API Development | Analytics | Angular | CI/CD401k matching | Annual Incentives | Dental insurance | Employee assistance program | Health insuranceMid-level Full TimeNew York, NY, United States R2d ago
-
Manager, Revenue Analytics USD 94K-102KAnalytics | Business Intelligence | Cloud platform | Data Integrity | Data Visualization401k | Bonus pay | Hybrid work schedule | Medical, dental, and vision insurance | Paid time offMid-level Full TimeGA, United States R2d ago
-
Solution Engineer, Data Engineering (Manager) USD 99K-165KAWS | Airflow | Azure | CI/CD | Data Modeling401k matching | Health benefits | Paid Holidays | Paid caregiver leave | Paid medical leaveMid-level Full TimeUSA - NY - Headquarters, United … R2d ago