Hotshot: Generate Up to 10-Second 720P Videos 209

Hotshot has launched an ai model capable of generating 10-second clips at a resolution of 1280 x 720 pixels.

The startup, which launched the model, also known as Hotshot, into public preview on Monday, joins a growing roster of artificial intelligence video generators on the market. OpenAI, Runway ML Inc., and many other startups similarly offer models capable of generating short clips based on user prompts.

Hotshot launched a consumer-facing artificial intelligence image generation application last year. According to VentureBeat, the company appears to have shut down that service to focus on the emerging field of video generation. Hotshot is reportedly backed by SV Angel, angel investor Lachy Groom, and Reddit Inc. co-founder Alexis Ohanian.

The company spent several months developing its latest video generation model. As part of the project, it created three different neural networks: the core Hotshot model and two additional neural networks that helped prepare the training dataset for the AI video generator.

Key features of Hotshot:

• Text-to-video conversion: Users can input text descriptions, such as scenes, characters, or actions, and Hotshot will automatically generate 3-second videos.

• Generate HD videos up to 10 seconds: The tool is capable of generating 10-second clips at a resolution of 1280 x 720 pixels.

• Generate GIF animations: Using the latest image generation technology, Hotshot can convert complex text descriptions into realistic GIF animations.

• Watermark-free video output: Videos generated by free users have no watermarks, giving creators more freedom.
  

In the first phase of the plan, Hotshot curated a repository containing 600 million clips with captions describing their content. It combined these clips with one billion images to create the training dataset for the video generation model. "We knew we wanted to train the model jointly on images and video in order to take advantage of the much richer pool of publicly accessible image data than video," wrote Hotshot team members John Mullan, Duncan Crawbuck, Chaitu Aluru, and Aakash Sastry in a blog post.

In the next phase of the project, the company developed an AI model to generate captions for the videos in its training dataset. Captions can help neural networks better understand the clips being trained on. The additional knowledge gathered by the AI thus improved the quality of its output.

Hotshot found that existing caption generation models on the market did not meet its requirements. In response, the company took one of the existing models and customized it on a training dataset containing 300,000 clips with manually created captions. "Within a few weeks, we had a video captioner that we were thrilled to use to annotate our hundreds of millions of video samples," the Hotshot team detailed.

The second auxiliary AI model the company built to support the development of the video generator was an autoencoder. This algorithm can take a piece of data (in this case, a video) and remove details that aren't needed for AI training. Removing redundant information reduces storage requirements, thereby lowering costs.

After preparing the autoencoder and caption models, Hotshot spent four months training its AI video generator. The company used thousands of Nvidia H100 graphics processing units, accumulating millions of processing hours during the project.

Hotshot's engineers applied several optimizations to reduce the infrastructure requirements of the training runs. The company stored many of the files used in the project in bfloat16 format, which can compress 32-bit data to 16 bits to save storage space. Additionally, it performed some calculations in advance that are typically executed during the training process, to better leverage the processing power of Nvidia chips.

Hotshot's new AI is accessible as part of its free video generation service on its website. The company also plans to make the model available to developers through an application programming interface.

The Hotshot AI video generation model has now entered the public "preview" phase. Users can experience this model for free on Hotshot's official website, though it's worth noting that the daily generation limit is capped at two videos.

Hotshot official website: https://hotshot.co/