DeepSeek-V3/R1-671B Full-Parameter Fine-Tuning Guide and Tutorial 131

What is the DeepSeek-V3/R1-671B Full-Parameter Fine-Tuning Guide?

The DeepSeek-V3/R1-671B Full-Parameter Fine-Tuning Guide is an open-source guide for full-parameter fine-tuning of the DeepSeek-V3/R1-671B model, jointly released by the Institute of Automation, Chinese Academy of Sciences and SCDM. The project provides complete code and scripts from training to inference, and summarizes the experiences and problem solutions encountered during training and deployment.

Features

• Full-parameter fine-tuning support: Provides complete code and scripts from training to inference, supporting data parallelism and sequence parallelism.

• Hardware optimization: Provides detailed hardware configuration recommendations and experimental results to help efficiently utilize computational resources.

• Data format adaptation: Supports the Openai standard data format, and extends compatibility with reasoning data.

• Inference deployment: Supports converting model weights to the Hugging Face format, and performing inference deployment through frameworks such as vLLM.

Use Cases of the DeepSeek-V3/R1-671B Full-Parameter Fine-Tuning Guide

• Natural language processing: Suitable for tasks such as text generation, machine translation, and sentiment analysis.

• Complex scenario reasoning: Capable of handling long documents and multi-domain tasks, such as medical-assisted diagnosis and financial analysis.

• Scientific research and technical exploration: Serves as a technical experimentation platform to explore potential paths toward Artificial General Intelligence (AGI).

• Enterprise-level applications: Can be used in scenarios such as intelligent customer service and document analysis to improve enterprise efficiency and decision-making capabilities.

How to Use the DeepSeek-V3/R1-671B Full-Parameter Fine-Tuning Guide

1. Hardware Configuration

• Requires 32 servers, each configured with:

• 8 x NVIDIA H100 80GB GPU

• 96-core CPU

• 2TB RAM

• 100TB shared storage

• InfiniBand 400G network

2. Environment Configuration

conda create -n ds_env python=3.10
conda activate ds_env
pip install -r requirements.txt
cp -r ./code/xtuner /path/to/env/site-packages

3. Data Preparation

Data format example:

{
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "用户问题"},
{"role": "assistant", "content": "思考过程\n最终回答", "loss": true}
]
}

4. Starting Training

Use pdsh or slurm to start training:

pdsh -R ssh -w node[0-31] 'bash ./code/scripts/sft_deepseek.sh'

5. Model Weight Conversion

Convert the training results to the Hugging Face format:

bash ./code/scripts/convert_pth_to_hf.sh

6. Inference Deployment

Deploy using vLLM:

# Start Ray
pdsh -R ssh -w node0 "conda activate vllm && ray start --head"
pdsh -R ssh -w node1,node2,node3 "conda activate vllm && ray start --address='node0:6379'"
# Start vLLM
pdsh -R ssh -w node0 "vllm serve /path/to/model --tensor-parallel-size 8 --pipeline-parallel-size 4"

GitHub Repository:https://github.com/ScienceOne-AI/DeepSeek-671B-SFT-Guide