Senior Manager, Data Quality & Evaluation
Tasks
- Build golden datasets and scorecards
- Coach and develop quality managers and reviewers
- Coordinate quality escalations and customer improvement plans
- Define evaluation frameworks for AI data
- Design QA sampling and review processes
- Develop rubrics and reviewer guidelines
- Lead quality operations for multilingual evaluation and speech audio review
- Manage calibration and adjudication workflows
- Monitor error trends and root cause quality issues
- Set quality standards and acceptance criteria
- Track reviewer performance and agreement metrics
- Transform quality insights into process improvements
Perks/Benefits
- Fully remote
- International team collaboration
- Opportunity to influence quality strategy
- Professional growth opportunities
Skills/Tech-stack
AI model | AI model assessment | Adjudication | Annotation Guidelines | Annotation platforms | Audio evaluation | Calibration | Cause analysis | Coding Evaluation | Dashboards | Data Annotation | Data Annotation Platforms | Data Quality | Error Analysis | Evaluation Design | Human-in-the-loop | LLM Evaluation | Model Assessment | Multilingual Evaluation | Performance reporting | QA Sampling | Quality Assurance | RLHF | Reviewer Training | Root Cause Analysis | Root cause | Rubrics | Scorecards | Speech Audio | Speech Audio Evaluation | The Loop | Transcription
Education
N/A
Related jobs
-
VP - Senior Data Engineer INR 3715K-4720KAmazon Web Services | Apache Airflow | Artificial Intelligence | Azure | CI/CDComprehensive healthcare | Flexible time off | Hybrid work model | Retirement plan | Support for working parentsSenior-level Full TimeBL4 - KNG Tower 1, Indiqube … R4d ago
-
Principle – Knowledge Governance & GenAI Enablement. INR 3799K-5500KAgent-based | Agent-based systems | Auditability | Collibra | Data GovernanceFlexible work environment | Inclusive workplace | Internal mobility | Volunteering opportunities | Wellbeing programsSenior-level Full TimeKA Remote, India R4d ago
-
Manager, Data Engineering INR 1800K-2500KAI | Azure | Azure Data | Azure Data Lake | Azure Data Lake StorageAccidental insurance | Maternity leave | Medical insurance | Paid time off | Paternity leaveMid-level Full TimeRemote - India R4d ago
-
Automation | Big Data | Business Intelligence | CI/CD | Cloud infrastructureFlexibility to work from anywhere | Opportunity to shape technical strategy | Remote-first work environment | Work with international teams | Work-life balanceExecutive-level Full TimeIndia R6d ago
-
GCP BigQuery,DevOps,Credit Card Domain/Associate Director, Data and Analytics Specialist INR 650K-900KApache Airflow | Artificial Intelligence | BigQuery | CI/CD | Cloud ComposerMid-level Full TimeHyderabad, Telangana, India R6d ago
-
Senior Manager - Data Analytics & Insight INR 3000K-4000KData Modeling | Data Pipelines | Data Quality | Data Visualization | ExcelHybrid work option | Professional developmentSenior-level Full TimeGurugram - DLF Building, India R7d ago
-
Sub Value Stream Technology Lead - Capital and Statistical Regulatory Reporting (CSRR) INR 3000K-4000KAgentic Frameworks | Agile | Automation | Best practices | Cause analysisCareer development | Diverse and inclusive culture | Remote and flexible collaborationSenior-level Full TimePune, Maharashtra, India R18d ago
-
Senior Manager Software Engineering, Data (Employer of Record) INR 7131K-10460KAWS Glue | AWS Lambda | AWS S3 | Agile | Amazon Web ServicesCollaborative team | Flexible scheduling | Performance-based rewards | Remote workSenior-level Full TimeIndia - Field R22d ago
-
IJP - Senior Manager - Data Science INR 2800K-4000KAPI | AWS | Apache Spark | Azure | Cloud DataHybrid work | Inclusive culture | Professional development | Supportive leadershipSenior-level Full TimeMumbai - Hiranandani, India R25d ago
-
Manager, Software Development INR 2000K-3000K.NET | AWS | Agile | C# | Cloud ArchitectureCareer development | Community building | Learning opportunities | Work from anywhereMid-level Full TimeIndia R1mo ago
-
Senior Data Engineering Manager INR 3000K-3880KAgile | Amazon S3 | Amazon Web Services | Apache Spark | Batch ProcessingHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R1mo ago
-
Master Data Manager – Data Governance & AI Enablement INR 2000K-3500KAPI | Cloud Computing | Data Classification | Data Enrichment | Data GovernanceSenior-level Full TimeIndia - Remote R1mo ago
-
Technical Manager, Data Engineering INR 938K-2100KAWS | Agile | Amazon Aurora | Athena | CI/CDFlexible work arrangements | Remote work optionsMid-level Full TimeHyderabad, India R1mo ago