Senior Manager, Data Quality & Evaluation

Tasks

• Define evaluation frameworks and quality standards
• Design AI data quality frameworks
• Develop rubrics and reviewer guidelines
• Drive process improvements across tooling training and staffing
• Lead annotation and evaluation quality operations
• Manage QA sampling and calibration processes
• Monitor quality metrics and error trends
• Perform root cause analysis for quality variation
• Run quality reviews and customer escalations
• Translate customer requirements into quality workflows

Perks/Benefits

• Fully remote
• International team collaboration
• Professional growth

Skills/Tech-stack

Adjudication | Annotation | Audio Review | Calibration | Cause analysis | Dashboards | Data Quality | Error Analysis | Golden datasets | Human Preference Evaluation | Human-in-the-loop | LLM Evaluation | Multilingual Evaluation | Performance reporting | Preference Evaluation | QA methodologies | RLHF | Root Cause Analysis | Root cause | Rubrics | Sampling strategies | Scorecards | Speech evaluation | The Loop | Transcription