MultiGO: A Breakthrough in High-Fidelity 3D Human Models from a Single Image 52

小编 · 2025-05-06

MultiGO, from the HKUST(GZ) team, uses hierarchical modeling to generate high-fidelity 3D human models from a single image, refining the body from base shape to detailed textures layer by layer, addressing the lack of detail in traditional 3D reconstruction. Accepted at CVPR 2025, it can be applied to virtual try-on, game character generation, and VFX. Through three-level geometry learning it enhances robustness to depth errors, restoring surface details down to sub-millimeter levels, offering strong practical value.

MultiGO: A Breakthrough in High-Fidelity 3D Human Models from a Single Image

What is MultiGO?

The MultiGO solution from the HKUST(GZ) team uses a hierarchical modeling approach to generate high-fidelity 3D human body models from a single image, refining the human body layer by layer from basic body shape to detailed textures, solving the problem of insufficient detail in traditional 3D reconstruction. This achievement was accepted at CVPR 2025 and can also be applied in fields such as virtual try-on, game character generation, and film special effects. It enhances resistance to depth errors through three-level geometric learning and can restore surface details down to the sub-millimeter level, offering great practical value.

MultiGO：实现了从单张图像生成高保真3D人体模型的突破.webp

MultiGO Technology

Three-Level Geometric Learning Framework

Skeleton Enhancement Module: Enhances human skeleton modeling by projecting 3D Fourier features into 2D space and combining SMPL-X human body meshes as geometric priors. Fourier spatial position encoding improves the semantic alignment capability between 3D models and 2D images, enabling the model to more accurately capture human pose features.
Joint Enhancement Strategy: Applies perturbations to joint positions during training to improve the model's robustness to depth estimation errors. This strategy avoids inaccurate limb position reconstruction caused by depth ambiguity by focusing on adjusting parameters that affect depth perception.
Wrinkle Optimization Module: Uses a diffusion-model-like denoising method, treating surface wrinkles as optimizable noise patterns, and progressively recovers finer high-frequency details. This innovative method enables MultiGO to generate sub-millimeter surface details, effectively compensating for the shortcomings of traditional methods in capturing high-frequency details such as clothing folds.

High-Precision Detail Restoration

Through a three-level hierarchical modeling strategy, MultiGO achieves multi-scale refined reconstruction from macroscopic pose to microscopic details. Experimental results show that MultiGO achieves industry-leading performance metrics on both the CustomHuman and THuman3.0 test sets.

How It Works

The SLE module projects 3D Fourier features into the 2D space of the input image, allowing the Gaussian reconstruction model to leverage human body shape prior knowledge and accurately capture human poses; the JLA strategy applies controlled perturbations during training to improve the model's robustness to depth errors during inference and enhance depth estimation of human joints; the WLR module treats the coarse mesh as Gaussian noise and refines wrinkles conditioned on the high-quality texture of the reconstructed Gaussian, optimizing geometric details.

Advantages of MultiGO

High-Fidelity 3D Human Reconstruction: Capable of generating high-fidelity 3D human models from monocular images, supporting fine reconstruction of details such as clothing folds and skin textures.
Rapid Generation and Adjustment: Supports rapid generation of personalized 3D virtual avatars and allows flexible adjustment of details, significantly reducing art asset production costs.
Multi-Scenario Adaptability: Suitable for application scenarios that require fast generation and high demands on realism and multi-scale detail restoration.

Application Scenarios of MultiGO

Virtual Try-On and Fashion E-Commerce: Consumers only need to upload a full-body photo to generate a high-fidelity 3D human model, and the system can automatically simulate the wearing effect of different clothing.
Gaming and the Metaverse: Quickly create personalized 3D virtual avatars from a single photo, with the hierarchical structure supporting flexible adjustments.
Film Special Effects and Virtual Production: In special effects production, a single-view photo of an actor is sufficient to reconstruct a high-precision 3D model for motion capture or stunt double synthesis, and the layered design allows specific layers to be edited separately in post-production.