YAYI2 (Yayi): Open-Source LLM by Zhongke Wenge 167

YAYI2 is a new-generation open-source large language model developed by Wenge Research, with the Chinese name "雅意" (Yayi). The model includes Base and Chat versions, with a parameter scale reaching 30B, built on the Transformer architecture.

YAYI2 was pre-trained on over 2 trillion tokens of high-quality, multilingual corpus. For general and domain-specific application scenarios, we used millions of instructions for fine-tuning, while leveraging reinforcement learning from human feedback methods to better align the model with human values.

YAYI2 Core Capabilities

• Chinese Knowledge Q&A: Ranks first in AGIEval and CMMLU evaluations, second in MMLU

• Information Extraction: Zero-shot Chinese entity extraction achieves SOTA, comparable to ChatGPT in English

• Multimodal Interaction: Image-text mutual generation, PDF parsing, real-time web understanding

• Role Playing: Supports tens of thousands of custom roles with ultra-long conversation context association

YAYI2 Technical Architecture

• 30 billion parameter scale, supporting 128k ultra-long context (approximately 200,000 characters of text processing)

• Pre-trained from scratch with full independent intellectual property rights, compatible with domestic GPUs such as Huawei Ascend

• Training data contains 2.65 trillion tokens, derived from 200TB of raw data refined through over 1,000 cleaning procedures

• Supports 10+ languages including Chinese and English, with industry-leading Chinese Token compression rate

YAYI2 Industry Applications

YAYI 2 demonstrates exceptional capabilities across multiple vertical domains, including media publicity, public opinion analysis, government governance, and financial analysis. In addition, it has been applied in education, finance and taxation, traditional Chinese medicine, finance and other fields, providing enterprises with AI-assisted tools, knowledge base Q&A systems, AI creative bots and other solutions.

• Government Affairs: Secure information extraction, public opinion special report generation, intelligent legal approval

• Financial Scenarios: Financial report parsing, investment compliance management, macroeconomic policy interpretation

• Media Creation: Multilingual news writing, video understanding, virtual anchor generation

• Professional Tools: Intelligent TCM consultation, enterprise knowledge base Q&A, programming language support

Compared to the Llama 2 open-source model, YAYI 2 performs better in Chinese scenarios, and can invoke tools such as calculators and weather queries through plugins. Its security architecture reinforces value alignment, and output content undergoes real-time review, making it suitable for sensitive government and enterprise scenarios. YAYI has already been deployed in national ministries and financial institutions, establishing ecosystem partnerships with Huawei Cloud and Bank of Beijing, among others.

YAYI 2 provides two versions, Base and Chat. The Base version is open-source, supporting developers in accessing code, models, and training data on GitHub. In addition, the model follows the Apache-2.0 license, allowing commercial use and secondary development.

GitHub Repository: https://github.com/wenge-research/YAYI2

HuggingFace: https://huggingface.co/wenge-research/yayi2-30b