Kyutai TTS: An Open-Source Text-to-Speech Tool Built for Real-Time Use 76

小编 · 2025-07-05

Kyutai TTS is an open-source text-to-speech model developed by France's Kyutai Labs, designed for real-time applications. With 1.6B parameters, it achieves word error rates of 2.82% in English and 3.29% in French, with a voice generation latency of just 220 ms, making it ideal for livestreaming and voice assistants. Its streaming and low-latency modeling techniques enable more efficient text-to-speech generation.

Kyutai TTS: An Open-Source Text-to-Speech Tool Built for Real-Time Use

Kyutai TTS is an open-source text-to-speech model developed by France's Kyutai Labs, designed for real-time applications. It has 1.6B parameters, with word error rates of 2.82% for English and 3.29% for French, and a voice generation latency of only 220 milliseconds, making it suitable for scenarios like live streaming and voice assistants. Its streaming processing and delayed stream modeling technology make text and voice generation more efficient.

Kyutai TTS: An open-source text-to-speech tool designed for real-time scenarios

Technical Features:

Streaming Processing and Low Latency: Kyutai TTS can generate audio immediately after receiving the first text token, with a latency of only 220 milliseconds. When processing 32 requests on an NVIDIA L40S GPU, the latency is approximately 350 milliseconds.

High-Precision Voice Output: The word error rates for English and French are 2.82% and 3.29% respectively, with speaker similarity reaching 77.1% and 78.7%. The voice is natural and close to the original samples.

Long-Form Content Generation: Capable of handling long articles, breaking through the 30-second limit of traditional TTS models.

Voice Cloning: Users provide a 10-second audio sample, and the model can match the speaker's timbre, intonation, and more. To ensure legality, Kyutai TTS offers a voice library based on public datasets and does not directly release voice embedding models.

Word Timestamps: Each word in the generated audio has precise timestamps, facilitating real-time subtitle generation or interactive applications.

Architecture and Training:

Delayed Stream Modeling (DSM): The model can generate voice in real-time as it receives text, supports streaming text input, and can handle multiple requests simultaneously.
Training Data and Hardware: Trained on 2.5 million hours of public audio data, with transcriptions generated by Whisper. The training process utilized 32 H100 GPUs.

Application Scenarios:

Real-Time Interaction: Suitable for virtual assistants, online education platforms, and real-time subtitle generation.
Content Creation: Can be used to generate podcasts, audiobooks, and other long-form content.
Assistive Tools: Provides high-quality text-to-speech services for the visually impaired.

Comparison with Other TTS:

Compared with other TTS models on the market, Kyutai TTS performs exceptionally well in terms of word error rate and speaker similarity, especially excelling in real-time interaction scenarios.

Kyutai TTS is open-sourced under the CC-BY-4.0 license, with source code and weights available on GitHub and Hugging Face. Developers can help expand the model's voice styles and language support by donating voice data.

Project URL: https://kyutai.org/next/tts