Data Operations Manager, Human Data Operations

Netflix is one of the world's leading entertainment services, with over 300 million paid memberships in over 190 countries enjoying TV series, films and games across a wide variety of genres and languages. Members can play, pause and resume watching as much as they want, anytime, anywhere, and can change their plans at any time.

Now, as Netflix explores a broader world of entertainment—expanding into Games, Ads, and Live—we are looking for a Human Data Operations Manager to help drive the growth of a new business unit at Netflix focused on how we evaluate and train ML and Generative AI models for use in our work.

About the Role 

High-quality human data is a critical part of AI. As a Human Data Operations Manager, you’ll play a key role in standing up and shaping this emerging function from the ground up, helping build a modern, nimble Data Operations function that supports a diverse and fast-moving set of ML initiatives across the business.

The Human Data Operations Manager role and the Human Data Operations (HDO) team are new. We’re building the infrastructure, workflows, and best practices that deliver high-quality, human-in-the-loop (HITL) data to power the company's development, evaluation, and iteration of AI and machine learning features.

You’ll help design and pilot the systems that make this possible—partnering closely with product, research, engineering, and data science teams to translate early-stage needs into operational workflows—and help create a repeatable foundation we can scale over time. Critically, you’ll help define new ways of working across teams—acting as a trusted guide and influencer as you introduce standards, pilot automation, and help partners build confidence in new evaluation tools and approaches. Your work will serve as the foundation for a nimble, modern Data Operations capability inside Netflix—one that matures with the business, adapts to evolving needs, and delivers clarity in a fast-changing space.

You’ll work closely with the Evaluation Program/Project Managers to translate evaluation frameworks into operational workflows that are clear, scalable, and delivered on time.

The Opportunity

This is a unique opportunity to get in on the ground floor of a fast-growing, strategically critical capability. As a Human Data Operations Manager, you’ll help stand up, shape, and scale a new function—one that connects human insight to machine learning progress in tangible, repeatable ways. You’ll work closely with product, research, and infra teams to design annotation workflows, drive workflow automation (including piloting tools like LLM Judges), and improve cost-efficiency, quality, and throughput across a wide range of ML initiatives. If you thrive in ambiguity and operate at the intersection of strategy, systems thinking, and execution, this is your chance to build something from scratch—with real and lasting impact.

The ideal candidate:

• Thrives in ambiguity, enjoys solving complex problems and has a knack for turning chaos into clarity. They are deeply curious, proactive and focused. 

• Can succinctly and broadly communicate what’s important about a problem, potential solutions and why. 

• Is comfortable working in a high-volume, high-quality, deadline-and data-driven environment. They are an excellent cross-functional partner who can relay the importance of this work to colleagues beyond your team, as well as collaborate on unique projects that use tags, technology, metadata, and content insights in new ways. 

• Is flexible, adaptive, resilient and curious in an ever-evolving organizational landscape.

• When presented with a vision/memo/test plan, can effectively and independently break it down into a clear scope, visualize potential solutions, identify critical open questions and proactively collaborate with the appropriate partners to get the clarity needed. They bring people along.

• Has a proven track record of developing smart, scalable solutions and successfully executing and evolving them over time. They are a change management expert. 

• Brings clarity to chaos.
   

Responsibilities:

• Design and manage end-to-end annotation workflows across evolving ML use cases, from early pilots to scalable systems.

• Partner with product managers, researchers, engineers, and data scientists to scope human data needs and turn them into actionable plans

• Establish and monitor benchmarks for throughput, quality, cost-efficiency, and turnaround time.

• Define best practices for tooling, vendor engagement, and human data governance as we grow.

• Pilot and integrate automation tools (e.g., LLM Judges, auto-evals) to increase speed and scalability without sacrificing quality.

• Act as a strategic partner and internal advisor—helping teams across the business adopt and operationalize effective HITL practices.

• Build lightweight documentation and workflows to scale knowledge and consistency across teams.

• Act as a connective tissue across teams to drive alignment and ensure effective execution of human algo evaluation initiatives.

• Responsibility for the intake, scoping, planning and resourcing of our multiple AI/algo eval projects, centralising and developing the playbooks we need as we scale. 

• Develop resourcing and operationalization plans.

• Partner with product managers, engineering, and data science to evaluate the operational impact of algorithmic changes and implement improvements as needed.

• Partner directly with Product Managers to manage HITL (Human-in-the-Loop) evaluation support requests, handling intake, scoping, and resourcing strategy.

Qualifications 

• 4+ years of experience leading human evaluations or structured QA frameworks in an AI/ML environment

• Strong understanding of evaluation design, including guidelines, rubrics, and scoring protocols

• Proven ability to work across disciplines and align partners around shared quality goals

• Comfort working in ambiguous, rapidly evolving environments

• Excellent written and verbal communication skills 

• Experience establishing and driving operational processes in the Data Operations / Algo Evaluation space 

• Experience evaluating LLMs or GenAI outputs (e.g., prompt completion, summarization, content generation)

• Familiarity with responsible AI principles and how to reflect them in evaluation design

• Ability to synthesize noisy feedback into actionable insights for product and research partners

• Prior experience working with human annotation vendors or raters

• Have a strong business mindset with a good understanding of how a well-oiled “business machine” is supposed to run

• Have acted as a strategic and operational partner to senior leaders. Attributes include:

  • Shine in ambiguous situations

  • See around corners (anticipate bottlenecks, escalate effectively, anticipate and make trade-offs)

  • Influence over authority to promote alignment 

  • Strategic thinker

  • Strong organizer & executor

  • Operates effectively across many teams 

• Experience using data and metrics to determine and drive business process improvements

• Experience in driving change management across complex organizational structures
   

Generally, our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top of market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $70,000 - $370,000.

Netflix provides comprehensive benefits including Health Plans, Mental Health support, a 401(k) Retirement Plan with employer match, Stock Option Program, Disability Programs, Health Savings and Flexible Spending Accounts, Family-forming benefits, and Life and Serious Injury Benefits. We also offer paid leave of absence programs.  Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off. Full-time salaried employees are immediately entitled to flexible time off.

See more detail about our Benefits here.

Netflix is a unique culture and environment.  Learn more here.

Inclusion is a Netflix value and we strive to host a meaningful interview experience for all candidates. If you want an accommodation/adjustment for a disability or any other reason during the hiring process, please send a request to your recruiting partner.

We are an equal-opportunity employer and celebrate diversity, recognizing that diversity builds stronger teams. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.

Job is open for no less than 7 days and will be removed when the position is filled.