Seed-Music: A Music Generation Model by ByteDance 120

Seed-Music is a music generation model developed by ByteDance. Users can generate music by inputting multimodal data (such as text descriptions, audio references, sheet music, voice prompts, etc.), and it provides convenient post-editing features, such as modifying lyrics or melodies.

Seed-Music combines autoregressive language models with diffusion models to provide precise control over the generated music while maintaining its quality.

Seed-Music also supports users uploading short voice clips, which the system will convert into complete songs.

In addition, Seed-Music supports not only vocal and instrumental generation, but also singing voice synthesis, singing voice conversion, music editing, and other features, making it suitable for different user groups.

Seed-Music Main Features:

• High-quality music generation

• Supports the generation of vocal and instrumental works. Users can input via text, audio, and other means to achieve diversified music creation.

• Controlled music generation

• Provides fine-grained music control, allowing users to generate music that meets their needs based on lyrics, style descriptions, reference audio, sheet music, and more.

• Multimodal input: Seed-Music supports various input methods, such as lyrics, style descriptions, reference audio, sheet music, and voice prompts, enabling fine-grained control.

• Style control: Users can specify the style, rhythm, melody, etc. of the music through text or audio references to generate works that meet their needs.

Voice Synthesis and Conversion

• Vocal synthesis: Generates natural and expressive singing in multiple languages.

• Zero-shot singing conversion: Only a 10-second voice or singing recording is needed to convert it into music of different styles.

• Lyrics2Song: Converts input lyrics into accompanied vocals, supporting both short and long music generation.

• Audio prompting and style transfer: Supports audio continuation and style transfer, generating new music in a similar style based on existing audio.

• Instrumental generation: Generates high-quality pure instrumentals, suitable for scenarios without lyrics.

Music Post-Editing

• Supports modification of lyrics and melody, allowing users to directly edit and adjust on the generated audio.

• Lyrics and melody editing: Seed-Music provides interactive tools that allow users to directly edit lyrics and melody in the generated audio, facilitating post-production adjustments.

• Music mixing and arrangement: The system can not only generate complete songs but also supports modifying the generated songs, such as adjusting instrumental parts, mixing effects, and more.

Multi-Style, Multi-Language Support

• Seed-Music can generate works covering a variety of music styles (such as pop, classical, jazz, electronic, etc.) and supports multilingual singing generation, suitable for global users.

• Real-time generation and streaming support

• Supports real-time music generation and streaming output, improving user interactivity and creative efficiency.

Seed-Music Applications:

• Lyrics-to-song: Users can input lyrics, and Seed-Music can automatically generate a complete song. This is very helpful for users who can write lyrics but lack composition skills.

• Music editing: Seed-Music provides highly flexible music editing features. Users can add or remove tracks or change styles through a lead sheet to obtain different versions of music works.

• Cross-language vocal conversion: The system supports converting Chinese vocals into English vocal output, expanding the possibilities and diversity of creation.

• Generating complete songs from audio clips: Even short audio clips can be expanded by Seed-Music into complete songs, which is very practical for users who need to quickly generate music content.

• Multimodal input for music generation: Supports generating music through text descriptions, audio references, sheet music, and other input methods, meeting the creative needs of different users.

In addition, Seed-Music also features the capability to clone human voices for song generation, offering fine-grained note-level adjustments, suitable for multilingual music creation.

Related News:

What products does ByteDance have