Senior Manager, Data Quality & Evaluation

Tasks

• Build calibration and QA sampling processes
• Build reusable quality assets and standardized approaches
• Coach and develop quality teams
• Design quality frameworks for AI data
• Develop rubrics and reviewer guidelines
• Drive improvements to instructions training and tooling
• Identify quality risks and resolve issues
• Lead quality operations for AI evaluation and annotation
• Monitor quality indicators and error trends
• Partner with teams for quality success from planning to production
• Support customer quality reviews and escalations
• Translate customer requirements into quality workflows

Perks/Benefits

• Fully remote
• International team collaboration
• Leadership influence
• Professional growth

Skills/Tech-stack

AI Data | Adjudication | Audio evaluation | Calibration | Coding Evaluation | Dashboards | Data Annotation | Data Operations | Error Analysis | Evaluation | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance Metrics | QA Sampling | Quality Assurance | Quality Auditing | Quality management | RLHF | Reviewer Training | Rubrics | Speech evaluation | The Loop | Transcription