VIMI: Controllable Character Video Generation Model 133

VIMI large model, an AIGC product for controllable character video generation launched by SenseTime, the world's first of its kind. Based on SenseTime's AI foundation model technology, VIMI can drive character images through multiple driving elements such as motion videos, animations, sounds, and text. The visual quality does not degrade or distort over time, thereby generating character videos consistent with the target motion. In addition to controllability, Vimi has also broken through the limitations of other large model AI video generation solutions in terms of character generation stability and video duration, and can stably generate minute-level single-shot character videos.

The VIMI large model not only achieves precise control over character expressions, but also controls the natural body movements of the upper body of characters in photos, and automatically generates matching changes in hair, clothing, and background. At the same time, lighting and shadow variations can also be reasonably generated, making character motions and visual effects smooth and natural, with a harmonious and beautiful画面. It is a stable controllable character video generation product.

Features of the VIMI Large Model:

• Controllable Characters: VIMI can precisely control facial expressions and body movements of characters in videos, generating video content highly consistent with target motions. Whether it's subtle changes in facial expressions or complex full-body movements, VIMI can accurately capture and reproduce them.

• Multiple Control Methods: VIMI provides diverse control input methods, including character videos, animation parameters, speech and text. This enables users to drive video generation through different input methods, creating richer and more diverse videos.

• Minute-Level Single Shot: Supports generating minute-level single-shot character videos

• Reasonable Video Scene Generation: VIMI not only focuses on the performance of the character itself, but can also generate reasonable video scenes, including the completion of elements such as the character's hair, clothing, background, as well as support for lighting and shadow changes. This makes VIMI-generated videos not only naturally performed by characters, but the entire scene is also harmonious and unified.

• Stable Generation of Long Videos: VIMI can generate single-shot character videos up to one minute long, with stable visual effects that do not degrade or distort over time. This is a significant breakthrough.
  

Application Scenarios of the VIMI Large Model:

VIMI is mainly aimed at consumer users and is suitable for various entertainment interaction scenarios such as chatting, singing, and dancing.

• Entertainment Creation: Meets the entertainment creation needs of a wide range of users, such as generating digital avatars, portrait videos, and various fun character memes.

• Film and Television Production: Provides high-quality character video素材 for film and television works, reducing production costs and time.

• Animation Production: Supports animation creators in using skeletons to generate motions, improving animation production efficiency.

The Vimi large model was released at the 2024 World Artificial Intelligence Conference (WAIC) and was selected as the conference's highest exhibition honor, the "Treasure of the Museum". The Vimi large model can generate single-shot character videos over 1 minute long, with visual effects that do not degrade or distort over time. It can also adjust environmental scenes based on character motions, simulate changes in camera angles and hair swaying, providing realistic visual effects.

Vimi will be fully opened to all users. Users only need to upload high-definition character photos from different angles to automatically generate digital avatars and portrait short videos in different styles. For users who love taking selfies, Vimi supports various entertainment interaction scenarios such as chatting, singing, and dancing.