Search Operation Quality Assurance for AI Model Evaluation

TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and its offices include New York, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Why Join Us
Creation is the core of TikTok's purpose. Our platform is built to help imaginations thrive. This is doubly true of the teams that make TikTok possible.
Together, we inspire creativity and bring joy - a mission we all believe in and aim towards achieving every day.
To us, every challenge, no matter how difficult, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always.
At TikTok, we create together and grow together. That's how we drive impact - for ourselves, our company, and the communities we serve.
Join us.

About Trust & Safety
Building a world where people can safely discover, create and connect. The Trust & Safety (T&S) team at TikTok helps ensure that our global online community is safe and empowered to create and enjoy content across all of our applications. We have invested heavily in human and machine-based moderation to remove harmful content quickly and often before it reaches our general community.

About the Team
The Search Operations team aims to improve search user experience, contribute significant DAU impact to the products and drive for increase in traffic and GMV.

The T&S evaluation and investigation team, working closely with the search team, taking the evaluation standards as the baseline and principles, and manually evaluating the search query and the corresponding search results page to reflect the real user experience of the search function.

The team is committed to providing search and evaluation services for international products, with resource support provided by different countries around the world, and global members conduct evaluations according to unified guidelines. The evaluation scope covers multiple search sections, such as basic search experience, recommended search, search advertising, and search security. As a bridge for business communication domestically and abroad, the team is responsible for key nodes such as standard writing, training Q&A, report output, and human efficiency management to effectively promote the execution and implementation of projects.

It is possible that this role will be exposed to harmful content as part of the core role/as part of project/ in response to escalation requests/by chance.

This may occur in the form of images, video, and text related to every-day life, but it can also include (but is not limited to) bullying; hate speech; child safety; depictions of harm to self and others, and harm to animals.

What will I be doing?
- Complete regular assessments of AI model's quality in accordance with evaluation standard documents, based on business requirements
- Plan, conduct and support policy training and go-live processes for in-house specialists. Coordinate and support in-house specialists' tasks and data delivery
- Ensure overall data delivery quality and improve project performance to achieve established targets:
- Conduct quality inspection of evaluated data and specialists' daily tasks, comb the types of AI model bad cases
- Be responsible for standard document clarifications to ensure better understanding for in-house evaluation specialists
- Conduct calibration meetings with the team to analyze and align on different cases, evaluate performance, offer solutions and suggestions to improve accuracy rates
- Work and interact closely with POC and other stakeholders to provide timely and quality feedback on SOP/guidelines/data quality to optimize existing processes and to ensure overall project success
- Work on various evaluation-related projects outside core responsibilities based on business needs
- Perform other ad-hoc duties on relevant projects as assigned