Human Evaluation Data Operations Manager, Google Play

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
  
• 5 years of experience in a technical project management or a customer-facing role.
  
• Experience with working in a operations role.
• Experience working in a human evaluation or data labeling environment.

Preferred qualifications:

• Experience in partnering with engineers, product managers, product specialists or operations managers to drive product and process development.
• Experience in problem solving and tools (e.g., SQL) with the ability to work through datasets to draw insights and make recommendations.
• Ability to adapt, prioritize and manage time effectively in a fast-paced environment.
• Ability to build consensus and collaborate with cross-functional teams and internal stakeholders.
• Excellent communication skills, enabling effective collaboration with international teams.

About the job

The Platforms and Devices team encompasses Google's various computing software platforms across environments (desktop, mobile, applications), as well as our first party devices and services that combine the best of Google AI, software, and hardware. Teams across this area research, design, and develop new technologies to make our user's interaction with computing faster and more seamless, building innovative experiences for our users around the world.

Responsibilities

• Own product data streams, partner with subject matter experts to implement evaluations to production and collect data from teams, measure rater accuracy and agreement and iterate on data collection projects to improve quality or volume.
• Track data quality and operational Key Performance Indicator (KPIs) and perform analysis to drive improvement and troubleshoot issues.
• Work with stakeholders to understand data collection priorities and manage capacity planning and scheduling within budget.
• Work with global vendor raters, vendor managers and vendor trainers to identify gaps or issues in data collection and design and implement solutions to address them.
• Stay up-to-date on the latest data labeling methods, including human rater labeling and machine learning and make recommendations to cross-functional teammates on solutions for applications.