Senior Manager, Data Quality & Evaluation

Tasks

• Build QA sampling adjudication and calibration processes
• Coach and develop quality managers and reviewers
• Conduct quality reviews and support customer improvement plans
• Convert quality insights into operational improvements
• Define evaluation frameworks and quality standards
• Design quality frameworks for AI data
• Develop rubrics task instructions and reviewer guidelines
• Identify quality risks and resolve delivery issues
• Lead quality operations for multilingual evaluation and speech and audio review
• Manage data collection annotation evaluation review calibration reporting
• Monitor quality indicators and analyze error trends
• Translate customer requirements into quality workflows

Perks/Benefits

• Fully remote
• International team collaboration
• Operational excellence focus
• Professional growth opportunities

Skills/Tech-stack

Adjudication | Annotation | Audio evaluation | Calibration | Data Quality | Error Analysis | Evaluation rubrics | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance reporting | QA Methodology | Quality Sampling | RLHF | Speech Audio | Speech Audio Evaluation | The Loop | Transcription Evaluation