Senior Manager, Data Quality & Evaluation

Tasks

• Coach and develop quality team members
• Design quality frameworks for AI data
• Develop evaluation rubrics and reviewer guidelines
• Implement continuous improvement actions
• Lead multilingual evaluation and speech audio review operations
• Manage quality standards and acceptance criteria
• Monitor quality indicators and root causes
• Perform adjudication and reviewer agreement analysis
• Run calibration and QA sampling
• Support customer quality reviews and escalations
• Track reviewer performance and error trends
• Translate customer requirements into quality workflows

Perks/Benefits

• Fully remote
• Innovation-focused workplace
• Professional growth

Skills/Tech-stack

Adjudication | Annotation Guidelines | Audio evaluation | Calibration | Cause analysis | Coding Evaluation | Dashboards | Data Quality | Data workflows | Error Analysis | Evaluation rubrics | Human Preference Evaluation | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance Metrics | Preference Evaluation | QA Sampling | Quality Assurance | RLHF | Reviewer Training | Root Cause Analysis | Root cause | Scorecards | Speech Audio | Speech Audio Evaluation | The Loop | Transcription