Video-R1: The World's First Video R1 Model from CUHK and Tsinghua University 130

What is Video-R1?

Video-R1 is the world's first R1 model for video, jointly released by the Chinese University of Hong Kong and Tsinghua University. It enhances the video reasoning capabilities of multimodal large language models (MLLMs) through reinforcement learning.

Video-R1 Features

• Time-aware Reasoning: Video-R1 introduces the T-GRPO algorithm, which compares shuffled-input videos against in-order input videos to encourage the model to leverage temporal information for reasoning, thereby better understanding the temporal relationships within videos.

• Mixed Data Training: To address the scarcity of high-quality video reasoning data, Video-R1 is trained on a mixed dataset of images and videos, including Video-R1-COT-165k (used for SFT cold start) and Video-R1-260k (used for RL training), enabling the model to transfer its capabilities from static image reasoning to dynamic video reasoning.

• Multiple Output Types: Video-R1 supports various answer output types, including multiple choice, numerical, OCR, free-form, and regression.

• Self-reflective Reasoning Behavior: The model performs self-reflection during reasoning, adjusting its reasoning direction to reach more accurate conclusions. This "aha moment" indicates that the model is actively engaging in an internal feedback loop.

Video-R1 Technology

• Time-aware Algorithm T-GRPO: This algorithm enhances the model's ability to understand temporal sequences, enabling it to deeply consider the causal relationships between events. Compared with the traditional GRPO algorithm, T-GRPO binds temporal understanding through a reward mechanism, rewarding the model only when it achieves a higher proportion of correct answers on in-order inputs.

• Mixed Image-Video Dataset: The researchers created two high-quality datasets: Video-R1-COT-165k (primarily image-based) and Video-R1-260k (centered on high-quality videos), addressing the scarcity of video reasoning data. This mixed training approach allows the model to evolve from "image captioning" to "video deep thinking," truly achieving multimodal understanding.

• Reinforcement Learning Training Strategy: Through reinforcement learning training, Video-R1 progressively discards suboptimal reasoning patterns during training, forming more efficient and more logical expression pathways.

Video-R1 Applications

• Video Content Analysis: Video-R1 can efficiently parse and understand the dynamic content within videos, and can be used for automatically generating key video segments, behavior recognition, intelligent tag generation, and other functions.

• Video Search and Recommendation: Through deep reasoning, Video-R1 can more accurately understand user needs and provide video content recommendations that better align with user interests.

• Autonomous Driving Visual Understanding: The model is capable of understanding temporal relationships and causal logic in videos, helping autonomous driving systems better perceive and predict road environments.

• Video Surveillance and Security: Video-R1 can be used for real-time analysis of surveillance videos, quickly identifying abnormal behaviors or events, and enhancing public safety.

• Education and Training: In the education sector, Video-R1 can be used in intelligent tutoring systems. By analyzing instructional video content, it provides students with personalized learning recommendations.

• Smart Home: The model can be integrated into smart home systems to achieve more intelligent environmental perception and automated control through video analysis.

GitHub Repository:https://github.com/tulerfeng/Video-R1

Paper:https://arxiv.org/abs/2503.21776