IBM's 470M-parameter English models trade translation and keyword biasing for lower latency, smaller memory needs and edge deployment.
By RuntimeWire Staff · Published
Primary source: Hugging Face Newsroom
Why it matters #
IBM's release gives developers a permissively licensed alternative for high-volume English transcription, built by stripping away costly language-model features instead of adding more parameters.
Brian Kingsbury, George Saon and six other IBM researchers released two compact speech-recognition models on August 25 after making a blunt engineering choice: they removed the language-model decoder that gave earlier Granite Speech releases broader capabilities.
The resulting Granite Speech 5.0 Turbo CTC models contain 470 million parameters each and focus solely on turning spoken English into text. IBM says the encoder-only design delivers over 20 times the throughput of earlier Granite Speech models, reaching more than 12,600 real-time factors on one Nvidia H200 during batched inference. At that rate, the system processes more than 3.5 hours of recorded audio per second.
That number comes from IBM's benchmark setup, rather than a third-party production test. IBM labels the OpenASR charts unofficial and says they were generated with Hugging Face Jobs and the leaderboard's scoring tools. The Apache model reports a 5.00% aggregate word error rate, while the noncommercial version reports 4.85% on those public English short-form tests. The results were measured as of August 21, 2026.
IBM's larger bet is easier to assess. Speech transcription still has workloads where raw throughput, predictable latency and local deployment matter more than attaching an audio interface to a general-purpose language model. Kingsbury and Saon's team built Granite Speech 5.0 for that narrower job.
A veteran speech team chooses the smaller machine
The release comes from researchers who were working on speech recognition long before foundation models absorbed the field's attention. Kingsbury, an IBM distinguished research scientist, joined IBM in 1999 and focuses on acoustic modeling and robust large-vocabulary speech recognition. He completed doctoral work at the International Computer Science Institute and the University of California, Berkeley, after studying electrical engineering at Michigan State University.
Saon, IBM's speech-strategy lead and a distinguished research scientist, has worked at IBM Research since 1998. Samuel Thomas, a senior research scientist and IBM Master Inventor, has worked across speech recognition, spoken-language understanding, synthesis and speaker recognition. Takashi Fukuda is a senior technical staff member for audio, speech and language processing at IBM Research-Tokyo and has held technical leadership roles on global speech projects.
Vishal Sunder, Jeff Kuo, Masayuki Suzuki and Madison Lee round out the credited release team. Their work follows IBM's earlier Granite Speech research, which connected a Conformer acoustic encoder to 2-billion- and 8-billion-parameter Granite language models for transcription and speech translation.
Granite Speech 5.0 reverses some of that architectural expansion. Each new model uses 16 Conformer blocks, block attention, self-conditioned connectionist temporal classification and eightfold temporal subsampling. Its standard Conformer block follows the lucidrains implementation, with chunkwise attention and PyTorch's scaled dot-product attention replacing parts of that design. The system reduces audio features from 100 frames per second to 12.5 output tokens per second, then performs non-autoregressive greedy decoding.
Removing the language-model component cuts memory use and latency. It also removes speech translation and keyword biasing, two features available in earlier Granite Speech models. IBM is betting that developers building call transcription, meeting records, voice interfaces and large offline processing jobs will accept that trade.
Two licenses divide research from commercial use
IBM released a pair of closely related models with different training sets and licenses. The Apache 2.0 model is intended for commercial applications and was trained on about 60,000 hours of public English audio, plus synthetic material. IBM reports a 5.00% aggregate word error rate across the public English short-form tests used by the OpenASR leaderboard, which it labels an unofficial result.
The noncommercial version adds GigaSpeech and SPGI Speech data, bringing its natural-audio training set to roughly 75,000 hours. It reports a slightly lower aggregate word error rate of 4.85%, also unofficial, but its CC-BY-NC-SA-4.0 license restricts commercial use.
The licensing split gives researchers access to IBM's stronger reported model while steering production users toward the version trained on a narrower set of permissively usable data. The accuracy gap is 0.15 percentage points on IBM's aggregate measure. For many commercial deployments, reproducibility, latency and error patterns on a company's own audio will carry more weight than that leaderboard difference.
Both models also use additional synthetic training data.
IBM says training the commercial model took 10 days on eight H100 GPUs in its Blue Vela cluster. That is a modest training footprint by current foundation-model standards, consistent with a release built around operational efficiency rather than scale for its own sake.
The speed claim still needs real workloads
The release provides two different views of performance. On short-form OpenASR tests, IBM reports low aggregate word error rates and H200 throughput above 12,600 real-time factors. On the FFASR leaderboard, which tests far-field, noisy and reverberant speech, IBM said the commercial model ranked ninth for accuracy and the noncommercial version ranked fifth on August 25. IBM said they were also the two fastest entries.
Those rankings give the release more substance than a speed number alone, although they do not settle how Granite Speech 5.0 handles accented speakers, overlapping conversation or audio captured by poor microphones in production. The 12,600 figure also reflects batched inference on an H200, a setup designed to maximize aggregate throughput rather than represent a single live stream running on a laptop.
IBM has provided a streaming WebGPU demonstration that runs in Chrome and Edge. The model card describes the Apache release as suitable for laptops, smartphones and other edge devices, but actual performance will depend on hardware, memory and quantization.
That practical testing is where IBM's narrower design should earn its place. Kingsbury, Saon and their colleagues have spent decades reducing speech errors under difficult conditions. Granite Speech 5.0 applies the same discipline to a market newly crowded with much larger models: remove the parts a transcription job does not need, publish the weights, and make developers decide whether a specialist can beat the generalists.