Senior Manager, Data Quality & Evaluation

Tasks

• Build reusable quality assets
• Coach and develop quality teams
• Conduct adjudication and error analysis
• Coordinate data collection and annotation quality
• Design quality frameworks
• Develop rubrics and task instructions
• Improve operational workflows and training
• Lead quality program escalations and retrospectives
• Manage quality standards and acceptance criteria
• Monitor quality indicators and root causes
• Perform evaluation and review processes
• Run calibration and QA sampling
• Support multilingual speech audio evaluation
• Track reviewer performance metrics
• Translate customer requirements into quality workflows

Perks/Benefits

• Fully remote
• International team collaboration
• Professional growth opportunities

Skills/Tech-stack

AI | Adjudication | Annotation Guidelines | Audio evaluation | Calibration | Cause analysis | Coding Evaluation | Dashboard | Data Annotation | Data Quality | Error Analysis | Evaluation | Human Preference Evaluation | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance Metrics | Preference Evaluation | QA Sampling | Quality Assurance | Quality Scorecards | RLHF | Reporting | Root Cause Analysis | Root cause | Rubrics | Speech Audio | Speech Audio Evaluation | The Loop | Transcription