StarVector: An Open-Source Multimodal Vision Model for Generating Scalable Vector Graphics 197

小编 · 2025-03-22

StarVector is a new open-source multimodal vision-language model jointly developed by ServiceNow Research, Mila - Quebec AI Institute, and ETS Montreal, specifically designed for generating Scalable Vector Graphics (SVG).

StarVector: An Open-Source Multimodal Vision Model for Generating Scalable Vector Graphics

What is StarVector?

StarVector is a new open-source multimodal vision-language model jointly developed by ServiceNow Research, Mila - Quebec ai Institute, and ETS Montreal, specifically designed for generating scalable vector graphics (SVG).

StarVector: An Open-Source Multimodal Vision Model for Generating Scalable Vector Graphics.webp

StarVector Core Features

Image-to-SVG Conversion: Capable of converting images directly into SVG code, achieving vectorization of images.
Text-to-SVG Generation: Generates corresponding SVG graphics based on text instructions.

StarVector Technical Features

Multimodal Architecture: StarVector adopts a multimodal architecture that can simultaneously process image and text information, seamlessly integrating vision and language models.
Direct Operation in SVG Code Space: Unlike some intermediate representation methods, StarVector operates directly in the SVG code space, generating standard, editable SVG code.
Large-Scale Dataset Training: StarVector is trained on the SVG-Stack dataset containing over 2 million SVG samples, ensuring that the model can generalize to various vectorization tasks.
Two Model Sizes: Provides two models—StarVector-1B (with 1 billion parameters) and StarVector-8B (with 8 billion parameters)—to meet different computational resource and performance requirements.

StarVector Architecture and Principles

Image Encoder: Uses Vision Transformer (ViT) to process images, dividing them into small patches and processing them sequentially.
LLM Adapter: Non-linearly projects the embedding vectors generated by the image encoder into visual tokens for integration with the language model.
Transformer Decoder Architecture: Maps sequences of visual tokens or text tokens to SVG code.

StarVector Performance

In image-to-SVG and text-to-SVG tasks, StarVector outperforms existing models. For example, on the SVG-Bench benchmark, StarVector-8B outperforms models such as GPT-4 Vision (2023) and Potrace on multiple metrics.

On the image vectorization task of the SVG-Diagrams dataset, StarVector-8B performs excellently on metrics such as DinoScore and LPIPS.

StarVector Application Scenarios

Design and Creative Fields: Helps designers quickly generate SVG files, improving design efficiency.
Technical Drawing and Chart Generation: Capable of generating complex technical diagrams and graphics, suitable for fields such as engineering and science.