MagicTryOn: An Open-Source Video Virtual Try-On Project by Zhejiang University and vivo 67

小编 · 2025-06-04

MagicTryOn is an open-source video virtual try-on project jointly released by Zhejiang University and vivo. Built on a video diffusion Transformer (DiT) architecture, it produces more realistic and stable clothing try-on results. By relying on a full self-attention mechanism, it ensures consistency across video frames, avoiding the flickering and jittering common in older methods. Key features include precise garment detail preservation, multi-condition guidance for more realistic results, and a mask-aware loss to improve generation quality. MagicTryOn achieves strong results across multiple datasets, with notable visual quality.

MagicTryOn: An Open-Source Video Virtual Try-On Project by Zhejiang University and vivo

What is MagicTryOn?

MagicTryOn is an open-source video virtual try-on project jointly launched by Zhejiang University and vivo. It uses the DiT architecture of the video diffusion Transformer to make clothing try-on look more realistic and stable. The project relies on a full self-attention mechanism to ensure consistency across video frames, avoiding the flickering and jittering issues common in older methods. Its main features include: precise preservation of clothing details, more realistic try-on results through multi-condition guidance, and improved generation quality using mask-aware loss. MagicTryOn performs well across multiple datasets, with strong visual quality and generalization capability.

MagicTryOn：浙江大学和vivo推出的开源视频虚拟试穿项目.webp

MagicTryOn Technical Principles

Diffusion Transformer (DiT) Architecture: Its modular design allows more flexible injection of condition information, enabling multi-level garment features to be effectively integrated into the denoising process. The built-in full self-attention mechanism jointly models spatiotemporal consistency in the video, capturing both intra-frame local details and inter-frame dynamic changes.
Coarse-to-Fine Garment Preservation Strategy: The coarse strategy injects garment tokens into the input token sequence during the embedding stage and extends the grid size of the rotary positional encoding, allowing garment tokens and input tokens to share consistent positional encoding. The fine strategy introduces semantic-guided cross-attention and feature-guided cross-attention modules during the denoising stage, providing fine-grained garment detail guidance.
Mask-Aware Loss: Based on this loss, the model can focus more on generating garment regions, improving the detail fidelity of garment areas and the realism of the overall composite result.

MagicTryOn技术原理.webp

MagicTryOn Main Features

Garment Detail Preservation: Accurately simulates garment textures, patterns, and silhouettes, maintaining realism and stability during human motion.
Spatiotemporal Consistency Modeling: Ensures coherence between frames in the video, avoiding garment flickering and jittering.
Multi-Condition Guidance: Generates more realistic and detailed try-on results based on multiple conditions, including text, image features, garment tokens, and contour tokens.

MagicTryOn应用场景.webp

MagicTryOn Application Scenarios

Online Shopping: Users can try on different garments online.
Fashion Design: Designers can quickly preview how garments will look.
Virtual Fitting Rooms: Provides virtual try-on services for physical stores.
Advertising and Marketing: Brands can create personalized try-on advertisements.
Gaming and Entertainment: Real-time try-on of virtual garments in games.