DreamActor-H1: Generate Videos of People Presenting Products, Ideal for Promotional Clips 71

DreamActor-H1 is a video generation framework launched by ByteDance in June 2025. It is based on diffusion transformer (DiT) technology and primarily targets the e-commerce and digital marketing sectors, where it can be used to generate product promotion videos.

This framework can pair human body and product images to generate high-quality demonstration videos of people interacting with products. It has been trained with multi-category augmentation on a large-scale mixed dataset, outperforming existing methods in preserving the identity information of both people and products and in generating physically plausible demonstration actions. It is well suited for personalized e-commerce advertising and interactive media. It leverages 3D body mesh templates and product bounding boxes to provide precise motion guidance. Combined with VAE-encoded appearance guidance, structured text-encoded category-level semantics, as well as mechanisms such as reference attention and object attention, it addresses the challenges that existing frameworks face in identity preservation and spatial relationship understanding.

Core Advantages of DreamActor-H1:

• Human-Object Dynamic Alignment: It automatically matches gestures with the position of the product. For example, when demonstrating "picking up a cosmetic item," the contact points between the fingers and the bottle can naturally align, without issues such as clipping or misalignment. In addition, it supports joint guidance using 3D human body mesh templates and product bounding boxes to ensure that actions follow physical rules such as gravity and grip angles.

• Multi-Dimensional Detail Fidelity: In maintaining human identity consistency, it can preserve features such as hairstyle and skin tone; for products, it can achieve high-fidelity reproduction, accurately presenting details such as material textures and logos.

• Semantically Enhanced 3D Coherence: Combined with category-level text descriptions such as "rotate to display lipstick," it enables smoother visuals during small rotations, avoiding product jitter or deformation.

Technical Principles:

• Data Processing Stage: A vision-language model (VLM) parses text instructions, such as "hold the phone and flip it." Then, pose estimation is used to generate the human skeleton, which is combined with product bounding boxes to construct spatial relationships, thereby extracting motion features.

• Model Architecture Design: Global attention controls the temporal coherence of the overall frame; reference attention injects features from reference images of people or products; object attention specifically handles the latent features of the product to ensure no loss of detail. The diffusion transformer (DiT) with a triple attention mechanism is built on the 7-billion-parameter foundation model Seaweed-7B, and fuses human and object information through masked cross-attention.

• Inference Optimization: It dynamically retrieves a predefined library of motion templates and adjusts the scaling of the object bounding box—for example, the bounding box of a small-sized product can be enlarged.

Application Scenarios of DreamActor-H1:

• E-commerce Livestreaming: It can automatically generate demonstration videos of virtual hosts holding products, thereby reducing livestreaming costs and improving livestreaming efficiency.

• Advertising and Marketing: It can produce low-cost multi-angle product usage animations, such as rotating to display shoe styles, capturing the attention of consumers.

Limitations of DreamActor-H1:

• Product Limitations: Currently, only small and medium-sized items are supported; large appliances and similar products cannot be handled.

• Dependency on Motion Templates: Actions need to be predefined, such as "pick up an item on the table," and the framework depends on matching with the initial human pose.

• Dynamic Artifacts: During fast motion, the texture of the product may exhibit flickering.

Workflow for Generating Product Promotion Videos with DreamActor-H1:

1. Material Preparation: Prepare one image of a person and one image of a product. For the person image, a front-facing half-body shot is recommended, with a resolution greater than 800*800 to ensure that the person's features are clear in the generated video. The product image should clearly display every detail of the product, such as logos and textures.

2. Text Instruction Input: Based on the requirements of the product promotion, enter relevant category-level text descriptions, such as "hold the phone and flip it" or "showcase lipstick details," to provide specific action and display requirements for video generation.

3. Video Generation: Input the prepared materials and text instructions into the DreamActor-H1 system. The system will process them based on its multi-stage collaborative architecture and hybrid attention mechanism. In the data processing stage, the vision-language model (VLM) parses the text instructions, and the motion feature extraction module generates the human skeleton through pose estimation and constructs spatial relationships combined with product bounding boxes; within the model architecture, global attention, reference attention, and object attention work in coordination, ultimately producing a high-fidelity demonstration video of human-product interaction.

The project links are as follows:

Project Page:https://submit2025-dream.github.io/DreamActor-H1/

Technical Paper:https://arxiv.org/abs/2506.10568