Senior Manager, Data Quality & Evaluation

Tasks

• Build calibration QA sampling adjudication and reviewer tracking processes
• Coach and develop quality managers leads specialists reviewers and contributors
• Conduct quality reviews and manage customer escalations and improvement plans
• Define quality standards acceptance criteria and review processes
• Design quality frameworks for AI data and evaluation programs
• Develop rubrics task instructions reviewer guidelines and golden datasets
• Drive actionable improvements across instructions training tooling staffing and workflows
• Identify quality risks and resolve issues affecting timelines
• Lead quality operations for multilingual evaluation speech and audio transcription and annotation
• Maintain reusable quality assets for scalable operations
• Manage data collection annotation evaluation review calibration and reporting
• Monitor quality indicators error trends and reviewer agreement
• Translate customer requirements into quality workflows

Perks/Benefits

• Continuous improvement culture
• Flexible work location
• Fully remote
• Innovation-focused workplace
• International team collaboration
• Professional growth opportunities

Skills/Tech-stack

Adjudication | Annotation quality | Audio evaluation | Calibration | Cause analysis | Dashboards | Data Quality | Data scorecards | Error Analysis | Evaluation Frameworks | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Operational excellence | Performance reporting | QA methodologies | Quality Risk Management | Quality dashboards | RLHF | Reviewer agreement | Risk Management | Root Cause Analysis | Root cause | Rubrics | Sampling strategies | Speech evaluation | The Loop | Transcription