Senior Manager, Data Quality & Evaluation

Tasks

• Build repeatable calibration QA sampling and adjudication processes
• Build reusable quality assets and standardized approaches
• Coach and develop quality operations team
• Design programs for quality success through planning and delivery
• Design quality frameworks for AI data programs
• Develop rubrics task instructions and reviewer guidelines
• Identify quality risks and resolve delivery issues
• Implement operational improvements based on quality insights
• Lead multilingual evaluation and annotation quality operations
• Monitor quality indicators and analyze root causes
• Run quality reviews and improvement plans for customers
• Translate customer requirements into quality workflows

Perks/Benefits

• Fully remote
• International team collaboration
• Professional growth opportunities
• Work flexibility

Skills/Tech-stack

Adjudication | Annotation | Audio evaluation | Calibration | Cause analysis | Coding Evaluation | Dashboards | Data Quality | Error Analysis | Evaluation rubrics | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance reporting | QA methodologies | Quality Assurance | RLHF | Reviewer Training | Root Cause Analysis | Root cause | Sampling | Scorecards | Speech Audio | Speech Audio Evaluation | The Loop | Transcription