UniVG: Video Generation Model by Baidu 174

Baidu UniVG is a unified model aivideo generation system capable of performing video generation using any combination of text and image as input conditions. The UniVG system was developed by the Baidu team to address the limitations of existing video generation models when handling a single task or a single objective.

UniVG open source address:https://univg-baidu.github.io/

Main features and characteristics of UniVG:

• High-freedom video generation: Uses a multi-condition cross-attention mechanism to generate semantically consistent videos based on input images or text.

• Low-freedom video generation: Introduces biased Gaussian noise to replace fully random Gaussian noise, in order to better preserve the content of input conditions.

• Multi-task processing: The video generation models within the system are reclassified into high-freedom generation and low-freedom generation, suitable for different generation tasks respectively.

• Diffusion model: A diffusion-based video generation approach that has achieved remarkable success in both academia and industry.

Application scenarios and advantages of UniVG:

• Flexible input: Users can flexibly input image and text conditions, used individually or in combination, to meet the needs of real-world application scenarios.

• High-quality output: Achieves the lowest FVD (Fréchet Video Distance) on the MSR-VTT database, surpasses current open-source methods in human evaluation, and is comparable to the existing closed-source method Gen2.

• Diverse tasks: Supports multiple video generation tasks including text-to-video, image-to-video, and combined text and image to video.

As a powerful tool, UniVG drives video generation technology toward greater flexibility and higher quality by integrating text and image inputs, providing broader ideas for promotion across multiple fields such as content creation, advertising, and entertainment.