Senior Manager, Data Quality & Evaluation

Tasks

• Build scalable quality workflows
• Coach and develop quality team members
• Collaborate with cross-functional teams and customers
• Conduct root cause analysis for quality variation
• Define evaluation frameworks and acceptance criteria
• Design quality frameworks for AI data
• Develop rubrics and reviewer guidelines
• Improve quality processes through training and tooling
• Lead quality operations across AI evaluation and annotation
• Manage QA sampling and adjudication
• Monitor quality indicators and error trends
• Track reviewer performance and agreement

Perks/Benefits

• Fully remote
• International team collaboration
• Professional growth

Skills/Tech-stack

AI Data | AI data workflows | Adjudication | Annotation | Audio evaluation | Calibration | Cause analysis | Coding Evaluation | Dashboards | Data Quality | Data workflows | Error Analysis | Evaluation Frameworks | Human Preference Evaluation | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance reporting | Preference Evaluation | QA Sampling | Quality Assurance | Quality metrics | RLHF | Reviewer Training | Root Cause Analysis | Root cause | Rubrics | Sampling strategies | Scorecards | Speech Audio | Speech Audio Evaluation | The Loop | Transcription