DeTikZify: Automatically Generate TikZ Graphics Code from Hand-Drawn Sketches or Figures 206

小编 · 2025-03-22

DeTikZify is a tool based on multimodal language models that automatically generates TikZ graphics programs for scientific diagrams. It can convert hand-drawn sketches or existing scientific figures into semantically faithful TikZ code, useful for drawing complex scientific diagrams.

DeTikZify: Automatically Generate TikZ Graphics Code from Hand-Drawn Sketches or Figures

What is DeTikZify?

DeTikZify is a tool based on a multimodal language model that can automatically generate TikZ graphics programs for scientific diagrams. It can convert hand-drawn sketches or existing scientific diagrams into semantics-preserving TikZ code, which can be used to draw complex scientific illustrations.

DeTikZify：自动从手绘草图或图形生成TikZ图形代码.webp

DeTikZify Core Features

Automatic Diagram Generation: Converts hand-drawn sketches or existing diagrams into TikZ graphics programs. The generated code can be directly used in LaTeX documents.
Semantics Preservation: The generated TikZ graphics are not only visually similar to the input sketch, but also retain the semantic information of the diagram, making subsequent editing and modification easier.
Multimodal Input Support: Supports multiple input methods, including hand-drawn sketches and existing diagrams, suitable for different use cases.

DeTikZify Technical Features

Based on Multimodal Language Model: Leverages advanced multimodal language model technology to understand image and text information, thereby generating accurate TikZ code.
MCTS-Based Reasoning Algorithm: Uses the Monte Carlo Tree Search (MCTS) algorithm to iteratively refine the generated TikZ programs without requiring additional training.

DeTikZify Use Cases

Rapid Diagram Generation: Researchers and students can use DeTikZify to quickly convert hand-drawn sketches into formal scientific diagrams, saving the time spent manually writing TikZ code.
Reproducing Existing Diagrams: For existing diagrams that have no stored semantic information, DeTikZify can help regenerate the diagrams, making further editing and modification easier.

DeTikZify Advantages

High-Quality Output: The generated TikZ graphics programs retain the semantic information of the diagram, ensuring the accuracy and readability of the figure.
Efficiency: Through the automated diagram generation process, the time spent manually writing and adjusting TikZ code is significantly reduced.
Flexibility: Supports multiple input methods, suitable for different use cases.

DeTikZify Installation and Usage

Installation: DeTikZify provides a Python package that can be installed via pip. In addition, it also provides online demo environments on Hugging Face Space and Google Colab.
Usage: Users can launch the DeTikZify Web UI via the command line, or directly invoke its features (including compiling, rendering, and saving TikZ graphics) through the programming interface.

Models and Datasets

Model Weights: DeTikZify's model weights and related model versions (such as DeTikZify-v2-8b) are hosted on the Hugging Face Hub.
Datasets: Although some TikZ drawings from arXiv have been removed from the public version of the DaTiZv2 dataset, the project provides a dataset creation script for users to rebuild the complete dataset on their own.

DeTikZify Target Users

Researchers: Researchers who need to quickly generate high-quality scientific diagrams.
Students: Students who need to create scientific diagrams.
LaTeX Users: Users who wish to insert high-quality diagrams into LaTeX documents.