DeepSeek Open Source Week Day 3: DeepGemm 247

小编 · 2025-02-26

DeepGEMM is a library designed for efficient FP8 general matrix multiplication (GEMM) with fine-grained scaling support, enabling both regular and Mixture-of-Experts (MoE) grouped GEMM.

DeepSeek Open Source Week Day 3: DeepGemm

What is DeepGemm?

DeepGEMM is a library specifically designed for efficient FP8 general matrix multiplication (GEMM), featuring fine-grained scaling capabilities and supporting both regular and Mixture of Experts (MoE) grouped GEMM.

Technical Features of DeepGemm

Written in CUDA: DeepGEMM is written in CUDA, requires no compilation, and uses lightweight Just-In-Time (JIT) modules to compile all kernels at runtime.
Fine-Grained Scaling: Supports both regular and Mixture of Experts (MoE) grouped GEMM, suitable for different matrix shapes and computational needs.
Performance Optimization: Employs a two-level accumulation (promotion) approach with CUDA cores to address the issue of imprecise FP8 tensor core accumulation, and leverages Hopper TMA features to accelerate data movement.

Main Advantages of DeepGemm

Excellent Performance: Despite its lightweight design, its performance rivals or even surpasses that of expert-tuned libraries for various matrix shapes.
Ease of Access: As a clean and easily accessible resource, DeepGEMM is an ideal choice for learning Hopper FP8 matrix multiplication and optimization techniques.
Flexibility: Supports automatic selection of block sizes, number of warpgroups, optimal pipeline stages, and TMA cluster sizes to adapt to different computational requirements.

Application Scenarios of DeepGemm

DeepGEMM is suitable for scenarios that require efficient matrix multiplication operations, particularly in deep learning ai model training and inference, where it can significantly improve computational efficiency and performance.

github:https://github.com/deepseek-ai/DeepGEMM