Aero-1-Audio: A Compact Audio Model by LMMs-Lab 58

What is Aero-1-Audio?

Aero-1-Audio is a compact audio model developed by LMMs-Lab, built on the Qwen-2.5-1.5B language model. It has 2.42 billion parameters and is released under the MIT license. It can efficiently handle a variety of audio tasks, making it suitable for scenarios where high-quality audio processing capabilities need to be deployed under resource-constrained conditions, such as mobile phone voice assistants, meeting recording and transcription systems, or real-time classroom speech transcription.

Features of the Aero-1-Audio Model

• High efficiency: Aero-1-Audio is built on Qwen-2.5-1.5B. Despite its smaller parameter size, it performs excellently on multiple audio benchmarks, even surpassing many larger ai models such as Whisper, Qwen-2-Audio, and ElevenLabs/Scribe.

• High training efficiency: The Aero-1-Audio model was trained in just one day using only 16 H100 GPUs, with about 5 billion tokens (equivalent to 50,000 hours of audio) of high-quality filtered data.

• Strong long audio processing capability: Aero-1-Audio can process continuous audio input of up to 15 minutes without segmentation, which is relatively rare among current audio models.

Performance Evaluation

• Speech recognition tasks: On datasets such as AMI, LibriSpeech, and SPGISpeech, Aero-1-Audio achieves the lowest Word Error Rate (WER). In long-form speech ASR tests, it shows the smallest performance degradation.

• Audio understanding tasks: Across multiple dimensions including audio analysis and understanding, voice command following, and audio scene understanding, Aero-1-Audio performs excellently, outperforming or matching other large-scale models.

Technical Advantages

• Dynamic batch size: Aero-1-Audio uses a dynamic batching strategy based on token length, improving compute resource utilization.

• Sequence packing: By combining sequence packing techniques with Liger kernel fusion, the average Model FLOP Utilization (MFU) increased from 0.03 to 0.34, significantly improving training efficiency.

Aero-1-Audio Application Scenarios

Aero-1-Audio can be applied in fields such as education, healthcare, and entertainment — for example, automatically generating lecture notes, assisting doctors in recording medical records, and generating real-time video subtitles.

How to Use Aero-1-Audio

• Install dependencies: You need to install a specific version of the transformers library with the command python3 -m pip install transformers@git+https://github.com/huggingface/[email protected].

• Simple demo: By importing the relevant libraries and functions, loading audio, and using the AutoProcessor and AutoModelForCausalLM classes, you can construct inputs and generate outputs to perform audio processing, such as transcribing audio.

• Batch inference: Similar to the simple demo, this can handle multiple audio files. You need to adjust the input data format and parameter settings, such as setting the padding direction.

Frequently Asked Questions

Q: What are Aero-1-Audio's advantages compared with other large audio models?

A: Aero-1-Audio uses parameters more efficiently. Its training data volume is more than 100 times smaller than models like Qwen-Omni and Phi-4, yet its performance remains competitive. It can also accurately handle continuous audio input of up to 15 minutes, which is rare among other models.

Q: What preparations need to be made before performing inference with the Aero-1-Audio model?

A: You need to install a specific version of the transformers library with the command python3 -m pip install transformers@git+https://github.com/huggingface/[email protected]. To improve performance, you can install flash-attn using the command pip install --no-build-isolation flash-attn. If you don't want to use flash-attn, you can also choose sdpa or eager.

Q: What are the characteristics of Aero-1-Audio's training data?

A: The training data comes from over 20 public datasets, with about 5 billion tokens, corresponding to approximately 50,000 hours of audio data. Compared with other models, its training data volume is smaller but sample efficiency is higher, allowing the model to achieve competitive performance with a smaller amount of data.

HuggingFace:https://huggingface.co/lmms-lab/Aero-1-Audio