{"slug": "crisperwhisper-speech-to-text-model-that-transcribes-what-you-say-verbatim", "title": "CrisperWhisper – Speech to Text Model That Transcribes What You Say Verbatim", "summary": "Nyra Labs released CrisperWhisper 2.0, a multilingual speech-to-text model that transcribes speech verbatim or as intended, with word-level timings averaging 30 ms boundary error on read speech and 41 ms on conversational speech. The model tops the Nyra Verbatim Speech Benchmark with a disfluency F1 of 93.5 for the Pro version, ahead of ElevenLabs Scribe v2 (79.2) and Microsoft MAI-Transcribe-1.5 (77.5).", "body_md": "**The most accurate verbatim speech recognition you can run in production:\ncontrollable, multilingual, and timed to the word.**\n\n[Release post](https://www.nyra-labs.com/crisperwhisper) ·\n[Paper](https://arxiv.org/abs/2607.18934) ·\n[Full documentation](/nyrahealth/CrisperWhisper/blob/main/DOCS.md) ·\n[Models](https://huggingface.co/nyralabs) ·\n[Benchmark](https://www.nyra-labs.com/research/nyra-verbatim-speech-benchmark)\n\nMost speech-to-text systems never actually decide whether to write down what\nwas *said* or what was *meant*. They inherit that choice from their training\ndata and apply it inconsistently. CrisperWhisper 2.0 makes it an explicit,\ncontrollable choice. One recording, two transcripts:\n\n**Verbatim**, exactly what was said, in one consistent format:\n`[um] so we we need to, to reschedule the th- thursday meeting to [uh] march third at nine thirty [laughter]`\n\n**Intended**, the clean version the speaker meant, with numbers, dates,\nand emails formatted the way you'd write them:\n`So we need to reschedule the Thursday meeting to March 3 at 9:30.`\n\nOn top of that:\n\n- **Word-level timings.** Around 30 ms mean boundary error on read speech\nand 41 ms on conversational speech, the most precise word timing of any\nsystem we benchmarked, on both.\n- **Verbatimize.** Upgrade transcripts you already have: given audio plus a\ntrusted clean transcript, the model reproduces your content word-for-word\nand inserts only the disfluencies and vocal events actually present in the\naudio (rare-word recall jumps from 6.8% to 96.1% vs. re-transcribing).\nThis turns the world's abundant clean corpora into verbatim ones, ready\nfor TTS data, clinical speech analysis, and dataset construction.\n- **Multilingual.** Verbatim and intended modes work across most languages\nWhisper supports. CrisperWhisper 2.0 tops the[Nyra Verbatim Speech Benchmark](https://www.nyra-labs.com/research/nyra-verbatim-speech-benchmark) leaderboard for disfluency F1 across ten languages, ahead of every\nclosed-source alternative we tested.\n- **Seamless longform.** Audio of any length, transcribed without the usual\nchunk-boundary artifacts: each window continues from the words already\ntranscribed (*conditional continuation* ), so there are no duplicated or\ndropped words at the seams and no fragile timestamp-token bookkeeping.\n- **Production inference.** A CTranslate2 runtime with speculative decoding\nand built-in mitigation of Whisper's looping-hallucination failure mode.\n\nThe [Nyra Verbatim Speech Benchmark](https://www.nyra-labs.com/research/nyra-verbatim-speech-benchmark)\nscores fillers, repetitions, cut-offs, and vocal sounds as separate, typed\nmetrics. Its headline number is **disfluency F1**: how reliably a system\nwrites down the disfluencies that were actually spoken, without inventing\nones that weren't. Averaged over ten languages:\n\n| # | System | Disfluency F1 | \n|---|---|---|\n| 1 | **CrisperWhisper 2.0 Pro** | **93.5** | \n| 2 | **CrisperWhisper 2.0** | **87.8** | \n| 3 | ElevenLabs Scribe v2 | 79.2 | \n| 4 | Microsoft MAI-Transcribe-1.5 | 77.5 | \n| 5 | CrisperWhisper 1.0* | 64.8 | \n| 6 | Inworld STT | 59.5 | \n| 7 | xAI Grok Speech-to-Text | 42.8 | \n| 8 | Deepgram Nova-3 | 37.8 | \n| 9 | Fish Audio ASR | 35.0 | \n| 10 | AssemblyAI Universal-3 Pro | 30.5 | \n\n<sub>* CrisperWhisper 1.0 is English/German-only; its average covers those\ntwo languages. English and German use human-labeled evaluation sets; the\nother eight languages use synthetic verbatim sets. Per-language breakdowns\nand how the metric is computed are in the\n[benchmark post](https://www.nyra-labs.com/research/nyra-verbatim-speech-benchmark).</sub>\n\nMean absolute word-boundary error on read speech (TIMIT), lower is better:\n\n| # | System | Boundary error | \n|---|---|---|\n| 1 | **CrisperWhisper 2.0** | **29.6 ms** | \n| 2 | xAI Grok Speech-to-Text | 37.1 ms | \n| 3 | CTC-seg | 49.3 ms | \n| 4 | ElevenLabs Scribe v2 | 51.3 ms | \n| 5 | NeMo-FA | 60.0 ms | \n| 6 | Deepgram Nova-3 | 63.3 ms | \n| 7 | WhisperX | 64.8 ms | \n| 8 | Cartesia Ink-Whisper | 69.4 ms | \n| 9 | Canary | 85.5 ms | \n\n<sub>Scored on exactly the words each system gets right. How the timings\nare extracted from supervised cross-attention, plus results on\nconversational speech, are in the\n[aligner post](https://www.nyra-labs.com/research/attention-to-aligner).</sub>\n\n```\n# NVIDIA GPU (Linux): fastest, includes speculative decoding.\n# An NVIDIA driver is all you need; CUDA libraries arrive via pip.\npip install \"crisperwhisper[ct2]\"\n\n# Pure PyTorch: runs anywhere torch does (macOS, Windows, CPU)\npip install \"crisperwhisper[transformers]\"\npython\nfrom crisperwhisper import CrisperWhisperModel\n\nmodel = CrisperWhisperModel()          # nyralabs/CrisperWhisper2.0_large\n# or pick a size: CrisperWhisperModel(\"turbo\")  # turbo / medium / small\n\n# Verbatim transcription (default): every filler, repetition, stutter,\n# false start, and vocal event\nresult = model.transcribe(\"meeting.wav\", language=\"en\")\nprint(result.text)\n\n# Intended: the clean, readable version\nclean = model.transcribe(\"meeting.wav\", language=\"en\", mode=\"intended\")\n\n# Word-level timestamps\nresult = model.transcribe(\"meeting.wav\", language=\"en\", word_timestamps=True)\nfor w in result.words:\n    print(f\"{w.start:6.2f}-{w.end:6.2f}  {w.word}\")\n\n# Verbatimize: upgrade an existing clean transcript with the\n# disfluencies that are actually in the audio\nresult = model.verbatimize(\"clip.wav\", \"I think we should ship it Friday.\")\n```\n\nAudio longer than 30 seconds is handled automatically (see\n[longform](#what-else-is-in-the-box) below). The first load of a model\ndownloads it from HuggingFace and, on the `ct2` backend, converts it once\ninto a local cache. That **one-time conversion** needs torch and\ntransformers, which the lean `[ct2]` extra does not install -- add them\nonce with:\n\n```\npip install \"crisperwhisper[ct2,convert]\"   # or add [convert] later\n```\n\nA CPU-only torch is enough for the conversion, and the dependencies can\nbe uninstalled once the converted model is cached. Loading a directory\nthat already contains a converted CT2 model (`model.bin`) never needs\nthem.\n\n| Shorthand | HuggingFace ID | Notes | \n|---|---|---|\n| `\"large\"` (default) | `nyralabs/CrisperWhisper2.0_large` | Best open quality | \n| `\"turbo\"` | `nyralabs/CrisperWhisper2.0_turbo` | Fastest, with some quality degradation; recommended as the speculative draft | \n| `\"medium\"` | `nyralabs/CrisperWhisper2.0_medium` | Near-large quality; best tradeoff between size and quality | \n| `\"small\"` | `nyralabs/CrisperWhisper2.0_small` | Smallest | \n| `\"large_pro\"` /`\"turbo_pro\"` /`\"medium_pro\"` /`\"small_pro\"` | `nyralabs/CrisperWhisper2.0_<size>_pro` | **Pro** : our best models, with improved performance, hotword boosting, trained on additional proprietary data | \n\nThe standard models are released under a\n[non-commercial research license](https://huggingface.co/nyralabs/CrisperWhisper2.0_large/blob/main/LICENSE.md)\nand are available for commercial licensing. The **Pro** models are available\nunder commercial license only. For both,\n[get in touch](https://www.nyra-labs.com/crisperwhisper).\n\nA small draft model proposes tokens and the main model verifies them. Same output, 1.3 to 1.4x faster:\n\n```\nmodel = CrisperWhisperModel(\"large\", draft_model=\"turbo\")\nresult = model.transcribe(\"meeting.wav\", language=\"en\",\n                          speculative_decoding=True)\n```\n\nEverything below works out of the box and is covered in depth in\n[DOCS.md](/nyrahealth/CrisperWhisper/blob/main/DOCS.md):\n\n| Option | What it does | \n|---|---|\n| `mode=\"verbatim\" / \"intended\"` | Choose what-was-said vs. what-was-meant per call | \n| `word_timestamps=True` | Per-word start/end times from supervised cross-attention alignment | \n| `hotwords=[...]` | Bias recognition toward names and rare terms ( **Pro models only** ) | \n| `model.transcribe_dual(...)` | Verbatim **and** intended in one pass (ct2) | \n| `model.verbatimize(audio, transcript)` | Insert real disfluencies into a trusted clean transcript | \n| `model.forced_align(audio, text)` | Timings for a transcript you already have | \n| Longform | Audio >30s transcribed seamlessly via conditional continuation, with no chunk-boundary duplicates, drops, or stitching | \n| Hallucination mitigation | On by default: detects and suppresses Whisper's looping-repetition failure mode during decoding | \n| `compute_type=\"float16\" / \"int8_float16\"` | Quantization | \n\nEach mechanism has a deep-dive post:\n\n- [Measuring verbatimness: the Nyra Verbatim Speech Benchmark](https://www.nyra-labs.com/research/nyra-verbatim-speech-benchmark) .\nTyped metrics for fillers, repetitions, cut-offs, and vocal sounds instead\nof one opaque WER number.\n- [How we unlocked multilingual style-controlled transcription at scale](https://www.nyra-labs.com/research/multilingual-style-control) .\nVerbatim/intended control across languages.\n- [Turning emergent cross-attention into a precise aligner](https://www.nyra-labs.com/research/attention-to-aligner) .\nSupervising Whisper's alignment heads for word boundaries around 30 ms.\n- [Longform transcription with conditional continuation](https://www.nyra-labs.com/research/longform-continuation) .\nResolving window seams with the words already transcribed instead of\nfragile timestamp tokens.\n- [Closing the verbatim data gap with Verbatimize](https://www.nyra-labs.com/research/verbatimize) .\nUpgrading clean corpora into verbatim ones at scale.\n- [Faster inference and mitigating hallucinations](https://www.nyra-labs.com/research/killing-hallucinations) .\nThe CTranslate2 stack, speculative decoding, and the anti-looping decoder.\n\n[DOCS.md](/nyrahealth/CrisperWhisper/blob/main/DOCS.md) covers the full API: backends and their trade-offs,\nevery `transcribe()` option, dual-mode transcription, forced alignment,\nlongform strategies, speculative-K tuning, hallucination-repair thresholds,\nquantization, model conversion, and the result object.\n\n**The inference code in this repository is MIT-licensed** (see\n[LICENSE](/nyrahealth/CrisperWhisper/blob/main/LICENSE)): use it freely, commercially or otherwise.\n`crisperwhisper/features.py` is vendored from\n[faster-whisper](https://github.com/SYSTRAN/faster-whisper) (MIT, SYSTRAN).\n\n**The model weights are not MIT.** They are released under the\n[Nyra Health Non-Commercial Research License](https://huggingface.co/nyralabs/CrisperWhisper2.0_large/blob/main/LICENSE.md):\nfree for research and other non-commercial use; any commercial use requires\na commercial license. The Pro models are available under commercial license\nonly. For commercial licensing of either,\n[contact Nyra](https://www.nyra-labs.com/crisperwhisper).", "url": "https://wpnews.pro/news/crisperwhisper-speech-to-text-model-that-transcribes-what-you-say-verbatim", "canonical_source": "https://github.com/nyrahealth/CrisperWhisper", "published_at": "2026-09-06 23:41:11+00:00", "updated_at": "2026-09-07 00:01:16.622753+00:00", "lang": "en", "topics": ["natural-language-processing", "artificial-intelligence", "machine-learning", "ai-research", "ai-products"], "entities": ["Nyra Labs", "CrisperWhisper 2.0", "ElevenLabs Scribe v2", "Microsoft MAI-Transcribe-1.5", "Nyra Verbatim Speech Benchmark", "CTranslate2"], "alternates": {"html": "https://wpnews.pro/news/crisperwhisper-speech-to-text-model-that-transcribes-what-you-say-verbatim", "markdown": "https://wpnews.pro/news/crisperwhisper-speech-to-text-model-that-transcribes-what-you-say-verbatim.md", "text": "https://wpnews.pro/news/crisperwhisper-speech-to-text-model-that-transcribes-what-you-say-verbatim.txt", "jsonld": "https://wpnews.pro/news/crisperwhisper-speech-to-text-model-that-transcribes-what-you-say-verbatim.jsonld"}}