CrisperWhisper – Speech to Text Model That Transcribes What You Say Verbatim

The most accurate verbatim speech recognition you can run in production: controllable, multilingual, and timed to the word.

Release post · Paper · Full documentation · Models · Benchmark

Most speech-to-text systems never actually decide whether to write down what was said or what was meant. They inherit that choice from their training data and apply it inconsistently. CrisperWhisper 2.0 makes it an explicit, controllable choice. One recording, two transcripts:

Verbatim, exactly what was said, in one consistent format: [um] so we we need to, to reschedule the th- thursday meeting to [uh] march third at nine thirty [laughter]

Intended, the clean version the speaker meant, with numbers, dates, and emails formatted the way you'd write them: So we need to reschedule the Thursday meeting to March 3 at 9:30.

On top of that:

• Word-level timings. Around 30 ms mean boundary error on read speech and 41 ms on conversational speech, the most precise word timing of any system we benchmarked, on both.
• Verbatimize. Upgrade transcripts you already have: given audio plus a trusted clean transcript, the model reproduces your content word-for-word and inserts only the disfluencies and vocal events actually present in the audio (rare-word recall jumps from 6.8% to 96.1% vs. re-transcribing). This turns the world's abundant clean corpora into verbatim ones, ready for TTS data, clinical speech analysis, and dataset construction.
• Multilingual. Verbatim and intended modes work across most languages Whisper supports. CrisperWhisper 2.0 tops theNyra Verbatim Speech Benchmark leaderboard for disfluency F1 across ten languages, ahead of every closed-source alternative we tested.
• Seamless longform. Audio of any length, transcribed without the usual chunk-boundary artifacts: each window continues from the words already transcribed (conditional continuation ), so there are no duplicated or dropped words at the seams and no fragile timestamp-token bookkeeping.
• Production inference. A CTranslate2 runtime with speculative decoding and built-in mitigation of Whisper's looping-hallucination failure mode.

The Nyra Verbatim Speech Benchmark scores fillers, repetitions, cut-offs, and vocal sounds as separate, typed metrics. Its headline number is disfluency F1: how reliably a system writes down the disfluencies that were actually spoken, without inventing ones that weren't. Averaged over ten languages:

_{* CrisperWhisper 1.0 is English/German-only; its average covers those two languages. English and German use human-labeled evaluation sets; the other eight languages use synthetic verbatim sets. Per-language breakdowns and how the metric is computed are in the benchmark post.}

Mean absolute word-boundary error on read speech (TIMIT), lower is better:

_{Scored on exactly the words each system gets right. How the timings are extracted from supervised cross-attention, plus results on conversational speech, are in the aligner post.}

pip install "crisperwhisper[ct2]"

pip install "crisperwhisper[transformers]"
python
from crisperwhisper import CrisperWhisperModel

model = CrisperWhisperModel()          # nyralabs/CrisperWhisper2.0_large

result = model.transcribe("meeting.wav", language="en")
print(result.text)

clean = model.transcribe("meeting.wav", language="en", mode="intended")

result = model.transcribe("meeting.wav", language="en", word_timestamps=True)
for w in result.words:
    print(f"{w.start:6.2f}-{w.end:6.2f}  {w.word}")

result = model.verbatimize("clip.wav", "I think we should ship it Friday.")

Audio longer than 30 seconds is handled automatically (see longform below). The first load of a model downloads it from HuggingFace and, on the ct2 backend, converts it once into a local cache. That one-time conversion needs torch and transformers, which the lean [ct2] extra does not install -- add them once with:

pip install "crisperwhisper[ct2,convert]"   # or add [convert] later

A CPU-only torch is enough for the conversion, and the dependencies can be uninstalled once the converted model is cached. a directory that already contains a converted CT2 model (model.bin) never needs them.

The standard models are released under a non-commercial research license and are available for commercial licensing. The Pro models are available under commercial license only. For both, get in touch.

A small draft model proposes tokens and the main model verifies them. Same output, 1.3 to 1.4x faster:

model = CrisperWhisperModel("large", draft_model="turbo")
result = model.transcribe("meeting.wav", language="en",
                          speculative_decoding=True)

Everything below works out of the box and is covered in depth in DOCS.md:

Each mechanism has a deep-dive post:

• Measuring verbatimness: the Nyra Verbatim Speech Benchmark . Typed metrics for fillers, repetitions, cut-offs, and vocal sounds instead of one opaque WER number.
• How we unlocked multilingual style-controlled transcription at scale . Verbatim/intended control across languages.
• Turning emergent cross-attention into a precise aligner . Supervising Whisper's alignment heads for word boundaries around 30 ms.
• Longform transcription with conditional continuation . Resolving window seams with the words already transcribed instead of fragile timestamp tokens.
• Closing the verbatim data gap with Verbatimize . Upgrading clean corpora into verbatim ones at scale.
• Faster inference and mitigating hallucinations . The CTranslate2 stack, speculative decoding, and the anti-looping decoder.

DOCS.md covers the full API: backends and their trade-offs, every transcribe() option, dual-mode transcription, forced alignment, longform strategies, speculative-K tuning, hallucination-repair thresholds, quantization, model conversion, and the result object.

The inference code in this repository is MIT-licensed (see LICENSE): use it freely, commercially or otherwise. crisperwhisper/features.py is vendored from faster-whisper (MIT, SYSTRAN).

The model weights are not MIT. They are released under the Nyra Health Non-Commercial Research License: free for research and other non-commercial use; any commercial use requires a commercial license. The Pro models are available under commercial license only. For commercial licensing of either, contact Nyra.