Senior Technical Product Manager - AI Agents, Evals & Reliability

Tasks

• Collaborate on system design evaluation and iteration
• Conduct offline metrics online experiments human feedback evaluation
• Define end to end AI system requirements
• Define evaluation frameworks for AI systems
• Drive execution with clear specifications and prioritization
• Manage trade offs quality latency cost reliability and user experience
• Own product quality end to end
• Translate model capability data constraints evaluation results into product decisions

Perks/Benefits

• N/A

Skills/Tech-stack

Algorithms | Bias detection | Cost Optimization | Data Structures | Experimentation | Human Feedback | Language Models | Language Processing | Large Language Models | Latency optimization | Machine Learning | Model Evaluation | Natural Language | Natural Language Processing | Prompt engineering | Reliability Engineering | System design