Automatic Speech Recognition • 0.5B • Updated • 260 • 3
TL;DR
Today we're delighted to announce the release of two new models in the Granite Speech family: compact, 470M-parameter English speech recognition models that pair strong accuracy with unprecedented speed — over 12,600 RTFx on an NVIDIA H200 GPU, meaning they can transcribe more than 3.5 hours of speech in one second using batched inference.
To get a sense of the models' responsiveness, check out our WebGPU demo of streaming speech recognition. Note that the demo only runs on Chrome or Edge browsers.
The two models differ mainly in training data and licensing: granite-speech-5.0-470m-turboctc-nc
trains on additional data and carries a CC-BY-NC-SA-4.0 license, while granite-speech-5.0-470m-turboctc
trains on a smaller set and is Apache 2.0 licensed.
Performance #
We report unofficial results on the public, English short-form test sets from the OpenASR Leaderboard below. Please see the leaderboard for official results that include the private datasets.
We provide bar charts showing the performance of ibm-granite/granite-speech-5.0-470m-turboctc-nc
and ibm-granite/granite-speech-5.0-470m-turboctc
as well as Pareto plots for the speed vs. accuracy trade-off and the model size vs. accuracy trade-off. Both models offer high accuracy, with the noncommercial model scoring an aggregate 4.85% WER and the Apache 2.0 model scoring 5.00% WER, and unprecedented aggregate throughput in excess of 12,600 RTFx. The noncommercial model is slightly more accurate than the Apache 2.0 model on most test sets, with a more noticeable advantage on SPGI Speech and a noticeable disadvantage on the new chunked Earnings22 test.
Figure 1: Unofficial results for the public test sets used in the OpenASR leaderboard as of 21 August 2026. Inference was run using HF Jobs and scoring was performed using the OpenASR Leaderboard tooling, so the WER results and RTFx figures are expected to match the official ones.
Figure 2: Unofficial Pareto plots (speed vs. accuracy and model size vs. accuracy) based on results for the OpenASR leaderboard public test sets as of 21 August 2026.
Both models also perform well on far-field speech recognition as measured by the FFASR Leaderboard. As of 25 August 2026, ibm-granite/granite-speech-5.0-470m-turboctc
ranked ninth in accuracy and ibm-granite/granite-speech-5.0-470m-turboctc-nc
ranked fifth while also being the fastest two models. Note that these are offical results.
Model Architecture #
These new models are encoder-only models, unlike prior Granite Speech models which comprise an acoustic encoder, projector, and Granite LM with LoRA adapters. The encoder-only design provides strong transcription performance, a small memory footprint of only 470M parameters, and over 20x faster throughput than previous Granite Speech models. The new models give up some of the capabilities of the LM-equipped models, such as speech translation and keyword biasing, but they are ideal for speech-to-text tasks on edge devices.
The Granite 5.0 Speech models have many things in common with the encoders used in previous Granite Speech models. They all
- comprise a stack of 16 Conformerblocks, - use self-conditioningat the output of the 8th block, - employ chunkwiseattention to avoid the quadratic scaling with sequence length of standard dot product attention, and - optimize the connectionist temporal classification (CTC)loss during training.
What's new about the Granite 5.0 Speech models is that they operate at a much lower token rate than our previous encoders and use a different output tokenization. While our previous encoders generated 50 characters per second, the Granite 5.0 models generate 12.5 tokens per second, with ibm-granite/granite-speech-5.0-470m-turboctc-nc
using a SentencePiece tokenization and ibm-granite/granite-speech-5.0-470m-turboctc
using a BPE tokenization. In both cases, the tokenizers are trained on speech transcripts.
To get from the 100 frames per second rate of the log Mel spectrogram front end down to a rate of 12.5 tokens per second, we rely on three stages of 2x subsampling, as shown below. The first stage, which is also used in the encoders of previous Granite Speech models, stacks successive log Mel feature vectors together via a simple reshape()
operation. The second and third stages are built into the first two Conformer blocks in the stack, which use strided convolutions to perform the temporal downsampling.
Figure 3: Placement of the subsampling operations in the stack of 16 Conformer blocks making up the encoder. Note that the self-conditioning is omitted in this figure for clarity.
The figure below illustrates the implementation of the Conformer block that performs temporal subsampling in comparison to a standard Conformer block. The two changes are that (1) the convolutional block performs temporal convolutions with stride= 2
and (2) the residual for the convolutional block is temporally subsampled by taking the mean of successive pairs of positions. The standard Conformer block follows the implementation from Phil Wang (lucidrains
), except that we use chunkwise attention and rely on PyTorch's scaled_dot_product_attention()
instead of the original einsum()
implementation.
Figure 4: Comparison of a standard and a subsampling Conformer block.
Training Data #
The Granite Speech 5.0 models are trained with a combination of natural and synthetic data. The table below lists the natural datasets used in training and specifies whether the dataset was used to train both models or only the non-commercial model.
| Name | # hours | Model | Source |
|---|---|---|---|
| MLS | 44600 | Both | |
https://huggingface.co/datasets/espnet/yodashttps://huggingface.co/datasets/mozilla-foundation/common_voice_17_0https://huggingface.co/datasets/openslr/librispeech_asrhttps://huggingface.co/datasets/facebook/voxpopulihttps://huggingface.co/datasets/edinburghcstr/amihttps://huggingface.co/datasets/esb/datasetshttps://huggingface.co/datasets/speechcolab/gigaspeechhttps://huggingface.co/datasets/kensho/spgispeechBoth models are also trained on three synthetic datasets:
- 2000 hours of multi-speaker data generated by concatenating single-speaker segments from MLS, YODAS, CommonVoice-17, VoxPopuli, and AMI;
- 500 hours of multi-speaker data generated by concatenating single-speaker segments from Earnings-22; and
- 240 hours of utterances containing numbers, currencies, website names, phone numbers, addresses, and items containing decimal points or dots which were generated using either
gpt-oss-120b
orgpt-oss-20b
and synthesized usingStyleTTS2
.
Usage #
Granite Speech 5.0 TurboCTC is supported natively in transformers. Until the next Transformers release, install from source:
pip install git+https://github.com/huggingface/transformers.git datasets
python
from datasets import Audio, load_dataset
from transformers import AutoModelForCTC, AutoProcessor
model_id = "ibm-granite/granite-speech-5.0-470m-turboctc"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCTC.from_pretrained(model_id, device_map="auto")
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
speech_samples = [el["array"] for el in ds["audio"][:5]]
inputs = processor(
speech_samples, sampling_rate=processor.feature_extractor.sampling_rate, device=model.device
)
inputs.to(model.device, dtype=model.dtype)
outputs = model.generate(**inputs)
print(processor.batch_decode(outputs, skip_special_tokens=True))