ByteDance OmniHuman Model Now in Internal Test on Jimeng 200

ByteDance's newly launched OmniHuman model has started internal testing on Jimeng AI. This model adopts a brand-new multimodal video generation technology. Users only need to input a single image and a piece of audio to generate highly realistic human video animations, covering everything from facial expressions to full-body movements, whether speaking, singing, dancing, etc.

Main Features of the OmniHuman Model:

• Video Generation: Capable of transforming an input single photo into a realistic full-body dynamic video, and can also enable digital humans to perform various natural movements such as speaking and singing.

• Multimodal Input: Supports image inputs of different sizes including portraits, half-body, and full-body at the same time. Combined with audio content and well-trained data, it can generate high-quality animated videos.

The OmniHuman model not only supports portrait and full-body video generation but also delivers satisfactory performance with different types of images, such as non-real-person inputs like anime and 3D cartoons. The person in charge of Jimeng AI stated that although OmniHuman has already shown great results, there is still room for improvement in generating cinematic-level real videos. OmniHuman will also be gradually opened up for use, currently undergoing small-scale internal testing with strict security review mechanisms in place.

Where is OmniHuman?

Log in to Jimeng AI, and select Video Generation from the left sidebar.

I am currently applying for internal testing... but some users say the lip-syncing effect is not very good, while others say it's amazing.