LHM: A 3D Human Model Reconstructed from a Single Image with Animation Support, by Alibaba 198

小编 · 2025-03-22

LHM is a technology proposed by Alibaba's Tongyi Lab for reconstructing animatable 3D human models from a single image. It addresses the decoupling of geometry, appearance, and deformation in 3D human reconstruction from a single image. In simple terms, you only need to provide one photo, and within seconds it creates a 3D human model capable of performing various actions while preserving fine details of clothing, hair, and face with highly realistic results.

LHM: A 3D Human Model Reconstructed from a Single Image with Animation Support, by Alibaba

What is LHM?

LHM is a technology proposed by Alibaba's Tongyi Lab for reconstructing animatable 3D human body models from a single image. It addresses the issues of geometry, appearance, and deformation decoupling when reconstructing 3D human bodies from a single image. In simple terms, you only need to provide one photo, and within seconds you can create a 3D human body model that can perform various actions. Details such as clothing, hair, and facial features are well preserved, and the result looks very realistic.

LHM：阿里推出的单张图像重建出可动画化的3D数字人模型.webp

Technical Highlights of LHM

Multimodal Transformer Architecture:

Leverages a multimodal Transformer architecture to effectively encode human body positional features and image features.
Preserves details through the attention mechanism, especially the geometric shapes and textures of clothing.

Head Feature Pyramid Encoding Scheme:

Proposes a head feature pyramid encoding scheme that aggregates multi-scale features of the head region.
Enhances facial identity preservation and detail restoration capabilities.

3D Gaussian Splatting:

Represents the 3D human body as Gaussian splats, enabling real-time, photorealistic rendering and pose-controlled animation.

LHM Method Workflow

Input Image: Extract body and head image tokens from the input image.
Multimodal Transformer Fusion: Use the proposed Multimodal Body-Head Transformer (MBHT) to fuse 3D geometric body tokens with image tokens.
Decode into Gaussian Parameters: After the attention-based fusion process, the geometric body tokens are decoded into Gaussian parameters.

LHM Experimental Results

Rapid Reconstruction: LHM can reconstruct animatable 3D human body models from a single image within seconds, without the need for post-processing.
High Accuracy and Generalization: Experiments on synthetic and real-world datasets show that LHM achieves industry-leading performance in reconstruction accuracy, generalization, and animation consistency.
Facial and Hand Details: LHM generates realistic facial and hand details, outperforming existing methods.

LHM：阿里推出的单张图像重建出可动画化的3D数字人模型.jpg

Features of LHM

Rapid Reconstruction: LHM can reconstruct animatable 3D human body models from a single image within seconds, without the need for post-processing.
High Accuracy and Generalization: Experiments on synthetic and real-world datasets show that LHM achieves industry-leading performance in reconstruction accuracy, generalization, and animation consistency.
Real-time Rendering and Animation Control: The reconstructed model supports real-time rendering and pose-controlled animation, making it suitable for immersive AR/VR applications.

Application Scenarios of LHM

Metaverse: Provides users with personalized 3D avatars, enhancing the immersion of virtual worlds.
Film and TV Production: Quickly generates high-quality 3D character models, improving production efficiency.
Game Development: Creates realistic 3D game characters, enhancing the visual effects of games.