Senior Manager, Data Quality & Evaluation

Tasks

• Build calibration and QA sampling processes
• Build reusable quality assets
• Coach and develop quality teams
• Conduct quality reviews and escalations
• Design quality frameworks
• Develop annotation rubrics and guidelines
• Identify quality risks and resolve issues
• Lead multilingual evaluation quality operations
• Manage AI data evaluation programs
• Monitor quality indicators and error trends
• Partner to design programs for quality success
• Translate customer requirements into quality workflows
• Turn quality insights into process improvements

Perks/Benefits

• Fully remote
• International team collaboration
• Professional growth opportunities

Skills/Tech-stack

Adjudication | Annotation Guidelines | Annotation platforms | Audio evaluation | Calibration | Cause analysis | Dashboards | Data Quality | Error Analysis | Evaluation Frameworks | Human Preference Evaluation | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance reporting | Preference Evaluation | QA Sampling | Quality management | Quality metrics | RLHF | Reviewer Training | Root Cause Analysis | Root cause | Rubrics | Sampling strategies | Scorecards | Speech Audio | Speech Audio Evaluation | The Loop | Transcription