MOSS-TTSD: Open-Source Bilingual Dialogue Speech Synthesis Model 72

MOSS-TTSD is an open-source bilingual conversational speech synthesis model developed by Tsinghua University's Speech and Language Lab in collaboration with Tencent AI Lab and other institutions. It supports Chinese and English, and can convert dialogue scripts between two speakers into natural, expressive conversational speech. The model is built by continuing training on top of the Qwen3-1.7B-base model, using discretized speech sequence modeling. It compresses speech to 1kbps via XY-Tokenizer while preserving both semantic and acoustic information.

The MOSS-TTSD model supports both Chinese and English, enables zero-shot cloning of multiple speakers' voices, and can generate speech up to 960 seconds long, making it suitable for AI podcasts, film and television dubbing, long-form interviews, and other scenarios. Its model weights, inference code, and API interfaces are all open-source, and it also supports free commercial use.

Key Features

✅ Highly expressive conversational speech: Built on a unified semantic-acoustic neural audio codec, a pre-trained large language model, millions of hours of TTS data, and 400,000 hours of synthetic and real conversational speech, it can generate human-like conversational speech with natural dialogue prosody and strong expressiveness.

✅ Dual-speaker voice cloning: Supports zero-shot dual-speaker voice cloning, accurately switches between speakers based on the dialogue script to generate conversational speech.

✅ Chinese-English bilingual support: Can generate highly expressive speech in both Chinese and English.

✅ Long-form speech generation: With a low-bitrate codec and optimized training framework, it is specifically trained for long-form speech generation.

✅ Fully open-source and commercially usable: MOSS-TTSD and its future updates will be fully open-source and support free commercial use.

MOSS-TTSD Technical Principles

✅ Model architecture: Fine-tuned on top of the Qwen3-1.7B-base model, employing discretized speech modeling. The speech signal is converted into a discrete token sequence through 8 layers of RVQ (Residual Vector Quantization), then generated autoregressively combined with Delay Pattern, and finally restored to speech by a decoder.

✅ Core innovations: XY-Tokenizer features a two-stage multi-task learning process. The first stage simultaneously trains Automatic Speech Recognition (ASR) and speech reconstruction tasks, fusing semantic and coarse-grained acoustic information; the second stage fixes the encoder and quantizer, trains only the decoder, and adds reconstruction loss and GAN loss to enhance detail representation.

✅ Data scale and pre-training: Trained with approximately 1 million hours of single-speaker speech data and 400,000 hours of conversational speech data. The data underwent strict filtering and annotation, along with Chinese and English TTS pre-training.

Application Scenarios

✅ AI podcast production: MOSS-TTSD can automatically convert text into multi-speaker conversational podcast audio, supports zero-shot voice cloning and long-form speech generation, reducing production costs.

✅ Film dubbing and animation: Generates natural conversational speech for films and animations, supports multilingual switching, improving dubbing efficiency.

✅ Long-form interviews and meeting records: Converts interview or meeting text into speech, preserving dialogue prosody and emotion for easy archiving and sharing.

Advantages and Features

✅ Open-source and commercially friendly: MOSS-TTSD's model weights, inference code, and API interfaces are all open-source, allowing users free commercial use.

✅ Naturalness and expressiveness: Voice naturalness and expressiveness reach industry-leading levels.

✅ Supports sound event control: Non-verbal sounds such as laughter can be added, making the speech more expressive.

Open-source address: https://github.com/OpenMOSS/MOSS-TTSD

MOSS-TTSD official website: https://www.open-moss.com/cn/