Open R1: A Fully Open-Source Reproduction Project of DeepSeek-R1 262

Open R1 is a fully open-source reproduction project of DeepSeek-R1 initiated by Hugging Face , aiming to fully reproduce the training process of the DeepSeek R1 model. The project's goal is to build the missing parts of the DeepSeek-R1 pipeline and provide researchers and the industry with a complete reasoning-optimized training pipeline.

Project Goals

• Reproduce the DeepSeek-R1 training pipeline: Through an open-source approach, it details how to progressively reproduce the DeepSeek-R1 training process, from knowledge distillation to reinforcement learning, and then to multi-stage training.

• Provide a complete implementation: The project includes a complete reasoning-optimized training pipeline, ensuring that users can easily use and modify the model.

Project Components

• Training data and scripts: Provides open-source datasets and training scripts for training, making it convenient for users to conduct experiments and verification.

• Documentation and examples: Detailed documentation and example code to help users get started quickly and understand how to use the model.

The Open R1 project also pays special attention to the following aspects:

Data collection and training code: Although the model weights of DeepSeek-R1 are open source, the datasets and code used for its training have not been made public. Open R1 plans to systematically rebuild DeepSeek-R1's data and training pipeline to verify its claimed effects and push the boundaries of open reasoning models.

Multi-stage training: The project's plan includes a multi-stage training process from a base model to supervised fine-tuning (SFT) and then to reinforcement learning (RL), aiming to demonstrate how to fine-tune existing or new models through synthetic datasets in order to transform them into reasoning models.

Cross-domain applications: In addition to mathematics datasets, Open R1 also hopes to explore potential applications in other fields, such as code and medical sciences, where reasoning models could have a significant impact.

How does the Open R1 project work?

Step 1: Distill a high-quality corpus using DeepSeek-R1 to replicate the R1-Distill model, and see whether the results are as good as DeepSeek claims.

Step 2: Replicate the pure reinforcement learning (RL) pipeline that DeepSeek used to build R1-Zero.

Step 3: Transition from a base model to an RL version through multi-stage training.

Open R1 project:https://huggingface.co/blog/open-r1

Open R1 code:https://github.com/huggingface/open-r1