CADCrafter: A Framework for Generating Parametric CAD Models from a Single Image 139

What is CADCrafter?

CADCrafter is an image-to-parametric CAD model generation framework proposed by researchers from Modai Technology, Nanyang Technological University in Singapore, and other institutions. It can directly generate high-quality, editable CAD models from a single image. These models can be compiled by a CAD compiler into production-ready 3D files, solving the problem of converting from images to editable CAD models.

Core Features of CADCrafter

• Geometry-based Generation: CADCrafter uses a geometry encoder to accurately capture geometric features in images, such as depth and normal maps. These features not only enhance the model's geometric representation capability but also enable the model to better generalize from synthetic data to real-world images.

• Direct Preference Optimization (DPO): Since compiling CAD parameter sequences into explicit CAD models is a non-differentiable process, CADCrafter uses the DPO method to leverage feedback from the CAD compiler to optimize the model, ensuring that the generated CAD instructions have higher compilability and geometric accuracy.

• Multi-view to Single-view Knowledge Distillation: CADCrafter distills knowledge from a multi-view geometry encoder into a single-view encoder, improving the accuracy and robustness of single-view input.

• Dataset Contribution: The research team also collected a real-world dataset called RealCAD, which contains multi-view images and corresponding CAD instruction sequences, used to validate the model's performance.

Research Method of CADCrafter

CADCrafter achieves the conversion from image to CAD model through the following steps:

• CAD Command Sequence Encoding: Encode CAD command sequences into discrete and continuous parameter sequences, and process them through an embedding space.

• Geometric Condition Encoder: Extract depth and normal maps from the input image, and use these geometric features to enhance the model's understanding of geometric structures and reduce the domain gap between synthetic data and real images.

• Denoising CAD Latent Vector: Use a diffusion-model-based architecture to denoise the latent CAD code, which iteratively learns to recover the original latent vector.

• Multi-view to Single-view Knowledge Distillation: Improve the accuracy and robustness of single-view input by distilling knowledge from a multi-view geometry encoder into a single-view geometry encoder.

• Direct Preference Optimization (DPO): Optimize the model through feedback from the CAD compiler, ensuring that the generated CAD instructions have higher compilability and geometric accuracy.

Experiments

• Dataset: The DeepCAD dataset was used for training, and a real-world dataset called RealCAD was collected for testing.

• Evaluation Metrics: Command accuracy (Acccmd), parameter accuracy (Accpara), median Chamfer Distance (Med CD), and Invalid Rate (IR) are used to evaluate model performance.

• Results:

• On the DeepCAD dataset, CADCrafter performed excellently in both multi-view and single-view tasks, significantly reducing the invalid rate.

• On the RealCAD dataset, although the model was only trained on synthetic data, it showed good generalization capability on real-world data, maintaining high accuracy and a low invalid rate.

• Compared with existing image-to-3D model generation methods (such as One-2-3-45, Wonder3D, and TripoSR), CADCrafter performs better in geometric accuracy.

Application Scenarios of CADCrafter

• Industrial Design: Quickly generate editable CAD models, accelerating prototyping and part reconstruction.

• Everyday Object Modeling: Directly generate engineering files usable for design and manufacturing by photographing everyday objects.

CADCrafter Paper:https://arxiv.org/pdf/2504.04753