Senior Technical Product Manager - AI Agents, Evals & Reliability
Tasks
- Collaborate on system design and evaluation
- Define AI system requirements
- Define evaluation frameworks for offline online human feedback
- Drive execution with specs and prioritization
- Ensure safe reliable shipping with feedback loops
- Make tradeoffs for quality latency cost reliability UX
- Own product quality correctness predictability user trust
- Translate model and evaluation results into decisions
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | Algorithms | Algorithms and Data Structures | B testing | Data Structures | Experiment design | Human Feedback | Language Models | Language Processing | Large Language Models | Machine Learning | Model Evaluation | Natural Language | Natural Language Processing | Prompt engineering | System design
Education
N/A
Roles
Related jobs
-
Senior Technical Product Manager, AI Agents & Systems GBP 120K-190KA/B | A/B Testing | Algorithms | B testing | Bias detectionSenior-level Full TimeUnited Kingdom R1d ago
-
Head of AI Governance GBP 98K-133K800-53 | AI Act | AI Agents | AI Governance | AI RMFCompany Paid Life Cover | EAP | Pension Retirement Plan | Private medical | Wellness benefitsExecutive-level Full TimeRemote - United Kingdom R4d ago
-
Data Science - Manager GBP 58K-75KApache Spark | Causal Inference | Distributed Computing | Experimental Design | HadoopHybrid work | In-office collaboration | Work from home days per weekMid-level Full TimeLondon R5d ago
-
Staff Tech Lead Manager, Machine Learning, Vision Models GBP 150K-162K3D Scene | 3D Scene Understanding | Auto-labeling | Computer Vision | Counterfactual evaluationHealth and wellness benefits | Hybrid work | Inclusive work environment | Learning and developmentSenior-level Full TimeLondon R6d ago
-
Applied AI Architect, Industries GBP 165K-190KClaude API | Cloud Architecture | Data Science | Enterprise Integration | EvaluationSenior-level Full TimeLondon, UK R6d ago
-
Data Product Delivery Lead GBP 78K-93KAWS | Agile | Apache Iceberg | Artificial Intelligence | Azure3 Days Per Week In Office | Hybrid workSenior-level Full TimeGlasgow Campus, United Kingdom R7d ago
-
Manager - Deals Analytics GBP 55K-61KAdvanced Analytics | Alteryx | Artificial Intelligence | BI Analytics | Business IntelligenceFlexible work | Hybrid workMid-level Full TimeLondon, Mayfair R8d ago
-
Sr. Manager, Data Science (Remote, GBR) GBP 90K-117KA/B | A/B Testing | AWS | Agentic AI | B testingAdoption leave | Paid Holidays | Paid parental leave | Paid vacation | Professional development opportunitiesSenior-level Full TimeGBR Remote, United Kingdom R9d ago
-
Senior Manager - Deals Analytics GBP 75K-90KAlteryx | Artificial Intelligence | Dashboards | Data Visualization | DatabricksHybrid work flexibility | Supportive inclusive recruitment processSenior-level Full TimeLondon, Mayfair R14d ago
-
Data Engineering Manager GBP 60K-79KAWS | Amplitude | Apache Airflow | Apache Kafka | Cloud platformFlexible vacation allowance | Hall of Fame benefit programs | Insurance | Mentorship programs | Paid leaveSenior-level Full TimeEdinburgh / Hybrid R15d ago
-
AI Engineer - London, UK GBP 78K-90KAI ethics | Artificial Intelligence | Cloud Computing | Data Privacy | Generative AIAccess to training courses | Hybrid working | Learning and development opportunities | Wellbeing programsMid-level Full TimeLondon, GB R19d ago
-
Director, AI Engagements USD 270K-325KAPI Integration | Agentic Workflows | Artificial Intelligence | Claude API | Data FlowExecutive-level Full TimeRemote, United States; UK - Remote R19d ago
-
Legal AI Program Manager USD 315K-385KArtificial Intelligence | Change Management | Enterprise Architecture | Enterprise Software | EvaluationMid-level Full TimeRemote, United States; UK - Remote R19d ago
-
Artificial Intelligence | Cloud Computing | Data analytics | Database analytics | Deep learningSenior-level Full TimeRemote - UK R20d ago
-
Strategy Analytics Manager GBP 44K-60KAPIs | Analytics engineering | Artificial Intelligence | Causal Inference | DBTEmployee referral program | Flexible working | Hybrid or remote work options | Office meals and snacks | Private health coverMid-level Full TimeLondon R22d ago
-
Product Analytics Manager USD 115K-140KChurn analysis | Customer Journey | Customer Segmentation | Customer journey analysis | Dashboard Design401k match | Dental insurance | Health insurance | Life insurance | Long-term disabilityMid-level Full TimeUnited Kingdom - Remote R26d ago
-
Executive-level Full TimeAmsterdam, Netherlands; Belgrade, Serbia; Berlin, Germany; … R29d ago
-
Senior Data Science Manager (Experimentation) GBP 67K-85KA/B | A/B Testing | B testing | Causal Inference | Cohort AnalysisDonation matching | Employee assistance program | Flexible schedule | Generous referral scheme | Health benefitsSenior-level Full TimeLondon R29d ago
-
Data Analytics & AI Solutions Group Leader GBP 78K-109KAgentic Workflows | Agile | Artificial Intelligence | Automation | Continuous ImprovementAnnual paid leave | Flexible working arrangements | Learning resources | Remote work | Volunteering daysSenior-level Full TimeBasingstoke, Hampshire, United Kingdom R1mo ago
-
A/B | A/B Testing | Amplitude | B testing | CSSAnnual company offsite | Co-working stipend | Learning and development stipend | Social travel stipendMid-level Full TimeUnited Kingdom R1mo ago
-
API Integration | Adversarial Testing | Agent systems | Amazon AWS | BenchmarkingESG initiatives | Employee incentive programs | Flexible vacation | Flexible work arrangements | Headspace app accessSenior-level Full TimeUnited Kingdom, London R1mo ago
-
MLOps Engineering Manager GBP 100K-115KAPI Monitoring | AWS | Access Management | Airflow | AutoscalingDental insurance | EV scheme | Extra festive time off | Family-friendly benefits | Personal learning budgetMid-level Full TimeLondon R1mo ago
-
AI Agents | API Design | Audience Segmentation | CDP Integration | CRM IntegrationHealthcare benefits | Paid time off | Professional development | Remote workSenior-level Full TimeRemote - United Kingdom R1mo ago
-
Staff Product Manager, Machine Learning USD 160K-200KB2B Product | B2B Product Management | Computer Vision | Data Ingestion | Data PipelinesCollaborative work environment | Coworking budget | Flexible remote work | Home office stipend | Ongoing education budgetSenior-level Full TimeRemote: United States | Canada | … R1mo ago
-
Machine Learning Manager, Operations GBP 113K-145KDemand forecasting | Forecasting | Machine Learning | Model Validation | MonitoringFlexible working hours | Learning budget | Relocation support | Visa sponsorship | Work from home supportMid-level Full TimeCardiff, London or Remote (UK) R1mo ago