Large Model Algorithm Researcher(Multimodal & Code AI)

Team introduction:
The TikTok AI Innovation Center is a department focused on building AI infrastructure and driving cutting-edge research in AI. We explore industry-leading AI technologies, including large language models (LLMs) and multimodal large models, with the goal of developing models that can understand multilingual content and vast amounts of video data, ultimately delivering a better content consumption experience for users. In the Code AI domain, we leverage the powerful code understanding and reasoning capabilities of LLMs to enhance program performance and R&D efficiency.

Project Introduction:
Multimodal foundation large models (VLM) represent a research hotspot in the industry and a critical technology for TikTok's business scenario applications. In 2024, TikTok's Innovation Center developed VFM V1, a multimodal large model tailored for TikTok's business scenarios. It matches the performance of the best open-source model Qwen VL on public test sets, while significantly outperforming all other foundation models on TikTok's business test sets. In the future, we aim to continuously develop foundation models with efficient perception and reasoning capabilities, capable of handling multilingual and massive video content understanding algorithms to deliver a better content consumption experience for users.

Project Challenges:
Enhance the multimodal perception encoder: The current encoder uses a fixed frame rate. We need to explore more efficient adaptive frame rates while considering the integration of modalities such as audio and user behavior.
How to fuse multimodal perception and thinking capabilities to promote stronger comprehensive perception and cognitive abilities of the model.