Ai2 launches OLMoASR, open speech models

Ai2 has released OLMoASR, a family of open automatic speech recognition models trained from scratch, with the data, code, and weights available to anyone. Its goal is to offer a reproducible alternative to proprietary systems such as Whisper, especially for transcribing English audio without task-specific training.

The proposal stands out for its transparency. Ai2 has published the models, the complete training process, the filters applied to the data, and the code used to evaluate the results. That lets you see how the system was built, reproduce the experiments, and modify it for new research.

Six models for different levels of compute

The first family includes six versions, from a lightweight model for devices with fewer resources to two large models with 1.5 billion parameters:

• OLMoASR-tiny.en: 39 million parameters.
• OLMoASR-base.en: 74 million.
• OLMoASR-small.en: 244 million.
• OLMoASR-medium.en: 769 million.
• OLMoASR-large.en-v1: 1.5 billion, trained on 440,000 hours of audio per epoch.
• OLMoASR-large.en-v2: 1.5 billion, trained on 680,000 hours per epoch.

The evaluation covered 21 test sets that had not been used during training: 14 with short audio and 7 with long audio. They included audiobooks, calls, meetings, and classes, with different accents, durations, and types of conversation.

Results close to Whisper

Ai2 measures performance using word error rate, or WER: the lower the percentage, the fewer errors the transcription contains. Across most model sizes, OLMoASR matches or exceeds Whisper's results in tests without prior adaptation, known as zero-shot evaluation.

The OLMoASR-medium.en model achieved 12.8% WER on short audio and 11.0% on long audio. The equivalent-sized Whisper-medium.en model recorded 12.4% and 10.5%, respectively. The results are very close, although Whisper retains a slight advantage in that comparison.

Among the larger models, OLMoASR-large.en-v1 reached 13.0% WER on short audio, compared with 12.2% for Whisper-large-v1. When Ai2 trained a second version using the same 680,000 hours per epoch used in that comparison, the difference fell to around 0.4 percentage points.

The smaller models also come close to their Whisper equivalents. OLMoASR-tiny.en and OLMoASR-base.en roughly match Whisper's results on short audio and outperform it on long audio, while OLMoASR-small.en delivers similar performance in both formats.

Data quality matters as much as quantity

Ai2 started with OLMoASR-Pool, a collection of 3 million hours of English audio and 17 million transcripts gathered from the public web. It then applied several filters to create OLMoASR-Mix, a dataset of around 1 million hours containing selected audio and text pairs.

The process removes, among other problems:

• Audio and transcripts that do not match in language.
• Text in all caps or with repeated lines, which are common in automatically generated transcripts with noise.
• Segments whose transcript differs too much from the audio.
• Near-duplicates, even when they are not identical word for word.

The central idea is simple: adding more hours does not guarantee better results if the text is poorly aligned or contains errors. According to Ai2's experiments, filtering improves the model's generalization across all sizes.

For you, this means there is an open option for building subtitling tools, meeting transcription systems, or search tools for recordings without depending entirely on a closed API. OLMoASR is designed for English, and its results do not eliminate all the differences with Whisper. They do reinforce an important trend: in speech recognition, transparent data can be just as useful as the final model.