CogView4: The World's First Open-Source Text-to-Image Model Supporting Chinese Characters 188

What is CogView4?

CogView4 is the first open-source text-to-image AI model that supports bilingual (Chinese-English) prompts, released by Zhipu AI on March 4, 2025. It is also the world's first open-source text-to-image model that supports generating Chinese characters. It is particularly skilled at understanding and following Chinese prompts and can generate Chinese characters within images, making it ideal for advertising, short video creation, and more.

Main Features of CogView4

• Bilingual Chinese-English Input: CogView4 can process Chinese and English prompts of any length and generate high-quality images.

• Chinese Character Generation: The model can naturally integrate Chinese characters into images, making it especially suitable for creative fields such as advertising and short videos.

• Arbitrary Resolution Image Generation: Supports generating images at any resolution within the range of 512×512 to 2048×2048.

• Strong Semantic Alignment Capability: In the DPG-Bench benchmark test, CogView4 ranked first in the overall score, demonstrating its excellent performance in complex semantic alignment and instruction following.

• Technical Optimization: Employs 2D Rotary Position Embedding (2D RoPE) and multi-stage training strategies, including base resolution training, general resolution training, high-quality data fine-tuning, and human preference alignment training.

• Memory Optimization and Efficient Inference: Significantly reduces memory usage through techniques such as model CPU offloading and text encoder quantization.

• Open Source and Ecosystem Support: Licensed under Apache 2.0, with future support for ecosystem integrations such as ControlNet and ComfyUI, as well as a complete fine-tuning tool package.

CogView4 Technology

• Architecture: Based on a diffusion model combined with a Transformer architecture, using 6 billion parameters.

• Text Encoder: Uses a bilingual GLM-4 encoder, trained on Chinese-English image-text pairs.

• Memory Optimization: Reduces memory usage and improves inference efficiency through techniques such as model CPU offloading and text encoder quantization.

CogView4 Application Scenarios

CogView4 is particularly suitable for fields requiring Chinese creativity, such as advertising design, short video production, and poster creation, capable of naturally integrating Chinese and English characters into images.

• Advertising Creativity: Generate posters with specific text, copy illustrations, etc.

• Short Video Production: Generate corresponding visuals based on video scripts or creative descriptions.

• Artistic Creation: Assist artists and designers in generating images with specific styles and artistic conception.

• Education: Generate images related to teaching content, such as artistic conception images for ancient poetry.

• Game Development: Generate game scenes and character images based on game plots and character settings.

• Other Creative Fields: Such as comic creation, illustration design, brand promotion, etc.

Using CogView4

1. Online Experience

• HuggingFace: https://huggingface.co/spaces/THUDM-HF-SPACE/CogView4

• ModelScope: https://modelscope.cn/models/ZhipuAI/CogView4-6B
  

2. Local Deployment

If you need to use CogView4 locally, you can deploy it through the following steps:

• Get the Model Code: Visit the CogView4 GitHub repository at: https://github.com/THUDM/CogView4.

• Hardware Requirements: The CogView4 model requires a significant amount of memory, with a minimum recommended GPU of 12GB VRAM.

• Install Dependencies: Install the necessary Python dependencies and environment according to the instructions in the GitHub repository.

• Run the Model: Follow the guidelines in the repository to launch the model and input prompts to generate images.

3. Model Fine-tuning

Zhipu plans to release a fine-tuning toolkit for CogView4, allowing users to perform customized training according to their own needs. For example, fine-tuning can be performed for specific advertising styles or brand requirements to generate images that better meet the needs.

4. Using via Hugging Face

Hugging Face Model Library URL: https://huggingface.co/THUDM/CogView4-6B.