Senior Technical Product Manager - AI Agents, Evals & Reliability
Tasks
- Collaborate on system design evaluation and iteration
- Create clear specifications and prioritize execution
- Define end to end AI system requirements
- Define evaluation frameworks for offline metrics online experiments and human feedback
- Ensure end to end product quality correctness predictability and user trust
- Make tradeoffs across quality latency cost reliability and user experience
- Translate model capabilities and evaluation results into product decisions
Perks/Benefits
- N/A
Skills/Tech-stack
Algorithms | Data Structures | Experimentation | Human Feedback | Language Models | Language Processing | Large Language Models | Machine Learning | Model Evaluation | Natural Language | Natural Language Processing | Prompt engineering | System design
Education
N/A
Roles
Related jobs
-
Senior Manager, AI Engineering & Platform USD 118K-145KAI-assisted software development | API Integration | AWS | Authentication | Backup and Recovery401k | Continuous professional development | Entrepreneurial culture | Health, dental, and vision | Minimal travelSenior-level Full TimeOverland Park, KS R1d ago
-
Senior Engineering Manager-People Analytics USD 175K-201KAI | Access Controls | Airflow | CI/CD | DBTSenior-level Full TimeUnited States R1d ago
-
Senior Manager, Data Science - AI and Styling Algorithms USD 200K-246KA/B Testing | Artificial Intelligence | B testing | Cloud infrastructure | Data ManagementAnnual bonus | Dental insurance | Equity grants | Medical insurance | Restricted stock unitsSenior-level Full TimeRemote USA R1d ago
-
Senior Manager Delivery Applied AI USD 145K-185KAI Development | AI development lifecycle | AWS | Agentic AI | Artificial IntelligenceSenior-level Full TimeMinneapolis, Minnesota, USA R1d ago
-
Senior Technical Product Manager, AI Agents & Systems USD 175K-240KAlgorithms | Artificial Intelligence | Cost Optimization | Data Structures | Experiment designSenior-level Full TimeUnited States R1d ago
-
Data Analytics Manager USD 86K-138KAlteryx | BI governance | CI/CD | Data Lineage | Data Modeling100 percent remote workSenior-level Full TimeUnited States R1d ago
-
Senior Engineer I, MSEO Automation and Data Analytics USD 106K-129KAzure AI | Batch records | Business Continuity | CAPA | CGMPCareer growth opportunities | Competitive benefits | Flexible work schedule | Training opportunitiesSenior-level Full TimeSummit West - NJ, United States R2d ago
-
Associate Director, Global Data Engineering USD 170K-224KAI Governance | AWS | Airflow | Apache Spark | AthenaMid-level Full TimeNutley, United States R2d ago
-
Business Analytics Manager, Revenue USD 125K-145KAI Agents | Business Intelligence | Data Modeling | Data Warehousing | Data analytics401k match | Dental insurance | Employee stock purchase plan | Flexible time off | Medical insuranceMid-level Full TimeHybrid - Denver R3d ago
-
Manager, Global Sourcing Analytics USD 75K-115KAI-enabled | AI-enabled tools | Amadeus | Analytics automation | Artificial Intelligence401k match | Employee Discounts on Cruises | Employee assistance program | Employee stock purchase plan | Health benefitsMid-level Full TimeMiami, FL, United States R4d ago
-
Technical Product Manager, AI Engineering & Systems USD 130K-170KAPI Design | Artificial Intelligence | Chatbot | Computer use | Data Analysis401k employer matching | Dental insurance | Health insurance | Paid parental leave | Paid time offMid-level Full TimeNew York, New York R4d ago
-
API Design | Agentic AI | Artificial Intelligence | Data Analysis | Experimentation401k matching | Dental insurance | Equity opportunities | Health insurance | Paid parental leaveSenior-level Full TimeNew York, New York R4d ago
-
Sr Technical Account Manager - West USD 150K-200KAWS | Active Directory | Apache Spark | Azure | Cloud platform401k match | Commuter benefits | Dental insurance | Employer paid disability coverage | Flexible spending accountsSenior-level Full TimeUnited States, Remote R4d ago
-
Data & Analytics Engineering Manager USD 185K-200KA/B | A/B Testing | AI tools | Agile | Alert triageSenior-level Full TimeNew York, NY R4d ago
-
Manager Applied AI Architecture, Financial Services USD 315K-380KAI Safety | API Integration | Artificial Intelligence | Consultative Sales | Cross-functionalFlexible working hours | Generous vacation | Optional equity donation matching | Parental leaveSenior-level Full TimeNew York City, NY; San Francisco, … R4d ago
-
AI Adoption Strategy | AI adoption | API Integration | Adoption strategy | Artificial IntelligenceFlexible working hours | Generous vacation | Optional equity donation matching | Parental leave | Visa sponsorship supportSenior-level Full TimeNew York City, NY; San Francisco, … R4d ago
-
Sr. Manager, Field Engineering USD 192K-264KBig Data | Cloud Data | Cloud data platform | Customer Success | Data EngineeringSenior-level Full TimeChicago, Illinois; Connecticut; New Jersey; Remote … R5d ago
-
Head of Applied Machine Learning - Application Fraud USD 210K-260KAmazon Web Services | Applied statistics | Data Governance | Elasticsearch | Experimentation401k match | Company in person events | Employer-paid health insurance | Flexible paid time off | Home office stipendExecutive-level Full TimeUnited States R5d ago
-
Applied Machine Learning Manager - Application Fraud USD 200K-250KAWS | Applied statistics | Code review | Data labeling | Elasticsearch401k match | Company in person events | Employer-paid health insurance | Flexible paid time off | Home office stipendSenior-level Full TimeUnited States R5d ago
-
Manager, Alpha Analytics and Portfolio Risk USD 225K-225KAPI Integration | Alpha Beta Separation | At risk | Audit Logging | Currency hedgingHealth and well-being benefits | Hybrid workSenior-level Full TimeChicago Office, United States R5d ago
-
Data Science Research Manager - Accenture Research USD 80K-293KAPI Integration | Agent Orchestration | Agentic AI | BigQuery | Cloud RunMid-level Full TimeNew York, One Manhattan West, Corp, … R5d ago
-
Integrated Services Analytics Product Manager USD 132K-250KAPI | Agile | Artificial Intelligence | BigQuery | Cloud NativeFamily care days | Flexible work schedule | Medical, dental & vision coverage | Paid parental leave | Remote work flexibilityMid-level Full TimeUnited States R5d ago
-
API Integration | Agent systems | Agents | Artificial Intelligence | Cloud ComputingHybrid work policy | Visa sponsorship supportMid-level Full TimeSan Francisco, CA | New York … R5d ago
-
Data Analytics Lead USD 150K-250KAI readiness | Data Architecture | Data Modeling | Data integration | Data setsCharitable giving program | Company-Paid Holidays | Dental insurance | Flexible PTO | Life insuranceSenior-level Full TimeMassachusetts - Boston R5d ago
-
Manager of Applied AI Architecture, Commercial USD 315K-380KAgentic Architectures | Artificial Intelligence | Context engineering | Customer Engagement | Developer ToolingSenior-level Full TimeSan Francisco, CA | New York … R5d ago