PartCrafter: A 3D Generation Model That Creates Multiple Parts and Objects from a Single Image 65

小编 · 2025-06-12

PartCrafter is a novel 3D generation tool jointly developed by Peking University, ByteDance, and Carnegie Mellon University. It can directly generate multiple 3D mesh parts with clear structures and shapes from a single ordinary RGB image.

PartCrafter: A 3D Generation Model That Creates Multiple Parts and Objects from a Single Image

What is PartCrafter?

PartCrafter is a new 3D generation tool jointly developed by Peking University, ByteDance, and Carnegie Mellon University. It can directly generate multiple 3D mesh parts with explicit structure and shape from a single ordinary RGB image.

PartCrafter：单张图像生成多个部件和对象的3D生成模型.webp

Core Features

Multi-part generation: PartCrafter can generate multiple independent 3D parts from a single image at once, without needing to segment the image first.
Part-level manipulation: The generated 3D models support editing of each part individually, such as adjusting position, rotation angle, or scale.
One-click generation: Users only need to provide one image, and PartCrafter can directly generate complex 3D scenes.
Efficient generation: Powered by a pretrained 3D mesh diffusion Transformer (DiT), PartCrafter delivers faster generation speed.

Technical Principles

Compositional latent space: Each 3D part is represented by a set of independent latent tokens. This allows parts to develop independently during generation while preserving part-level details.
Hierarchical attention mechanism: This mechanism enables information to flow in an orderly manner within each individual part as well as between parts, ensuring overall consistency in the generated output.
Based on a pretrained 3D mesh diffusion Transformer (DiT): It inherits pretrained weights, encoders, and decoders, all of which further enhance the model's generation capabilities.
End-to-end part-aware generation: Given a single image, it can denoise multiple 3D parts simultaneously, enabling direct generation from a single object to complex multi-object scenes.

Data and Training

Dataset: Part-level annotations were extracted from large-scale 3D datasets to build a dataset containing approximately 50,000 annotated objects and 300,000 independent parts.
Training method: A curriculum learning strategy is adopted to progressively optimize model performance.

Application Scenarios

Game development: It can quickly generate high-quality 3D game assets such as characters, props, and scenes. Developers only need to provide an image or a simple description to quickly obtain a large number of 3D models, significantly accelerating game development.
Architecture and interior design: It can rapidly generate architectural models and interior decoration schemes. Designers upload architectural drawings or design sketches and instantly get 3D models, making it easier to present designs and helping clients understand design ideas more clearly.
Film and television production: With its ultra-high-definition geometric detail modeling capabilities and the ability to generate PBR models from multi-view input, it can provide high-quality 3D model assets for film and television production teams.

Performance

High efficiency: Generating a structured 3D model from a single image takes only about 40 seconds.
High accuracy: Part generation accuracy and scene consistency are improved by 32% and 28% respectively compared with existing methods.