One C++ binary, 54 ASR backends + 55 TTS engines + multilingual text translation, zero Python dependencies.
CrispASR started as a fork of whisper.cpp and extends that base into a unified speech engine called crispasr
, backed by full ggml C++ runtimes for major open-weights ASR and TTS architectures. One build, one binary, one consistent CLI β pick the backend at the command line or let CrispASR auto-detect it from your GGUF file. See Text-to-Speech for the TTS side.
$ crispasr -m ggml-base.en.bin -f samples/jfk.wav # OpenAI Whisper
$ crispasr -m parakeet-tdt-0.6b.gguf -f samples/jfk.wav # NVIDIA Parakeet
$ crispasr -m canary-1b-v2.gguf -f samples/jfk.wav # NVIDIA Canary
$ crispasr -m voxtral-mini-3b-2507.gguf -f samples/jfk.wav # Mistral Voxtral
$ crispasr --backend qwen3 -m auto -f samples/jfk.wav # -m auto downloads
$ crispasr --backend kokoro -m auto --tts "Hello world" --tts-output out.wav # TTS
No Python. No PyTorch. No separate per-model binary. No pip install
. Just one C++ binary and a GGUF file.
Browser: All backends compile to WebAssembly (4.3 MB) via build-wasm.sh
. Multithreaded, runs entirely client-side with COOP/COEP headers.
Demo: HuggingFace Space β
live transcription + TTS + language detection, auto-deployed from hf-space/
.
| Project | What it does |
|---|---|
| This repo β C++ speech engine. 54 ASR + 55 TTS backends, CLI + HTTP server + C-ABI + Python/Rust/Dart/Go/Ruby/Java bindings. | |
| Cross-platform Flutter transcription app built on CrispASR. Desktop + mobile, model browser with download queue, mic capture, SRT/VTT/JSON export, diarization, batch processing. Fully offline. | |
| Text-related engine via ggml β same philosophy as CrispASR but for embeddings, retrieval, OCR and OMR, Math and Music Notation. Numerous architectures (XLM-R, Qwen3-Embed, Gemma3, ModernBERT, ...), dense + sparse + ColBERT + reranking. PP-OCR, Tesseract, EasyOCR, InternVL2, etc. Python/Rust/Dart bindings. | |
| Python ASR GUI with 9 backends (faster-whisper, mlx-whisper, voxtral, insanely-fast-whisper, ...). The Python counterpart to CrispASR's C++ approach. |
β new to CrispASR? Two commands to your first working audio, no repo clone neededStart hereSupported backendsβASR+TTS+translation+post-processing+music & audio analysisFeature matrixInstall & buildβ quick install (full guide indocs/install.md);β thewhich prebuilt Linux tarball to download-hip
/-vulkan
builds require the matching GPU driver and donot fall back to CPU (the-cuda
tarballs do, since v0.8.30)Quick start β ASRTroubleshootingβ it printed the banner and stopped, reading the exit code,--no-gpu
bisect, which Windows zipβ 52 engines: Kokoro, Qwen3-TTS, VibeVoice, dots.tts, Orpheus, Chatterbox, IndexTTS, Irodori, VoxCPM2, CosyVoice3, CSM, Dia, Zonos, Bark, Piper, MeloTTS, and moreText-to-Speech (TTS)Streaming & live transcriptionServer mode (HTTP API)Concurrency, parallelism & scalingβ how one transcription uses multiple cores, concurrent server requests (--server-workers
), bulk offline transcription, replicas behind a load balancerCLI referenceβ flags, VAD, CTC alignment, output formats, auto-download, audio formatsEnvironment variablesβ theCRISPASR_<BACKEND>_<FEATURE>
convention, global knobs, and every per-backend variableLanguage bindingsβ Python / Rust / Dart / Go / Java / JavaScript / Ruby / mobileBenchmarking CrispASRβ how to measure transcribe time (not cold start): server/in-process reps, proof-of-work rules, phase-timing env varsArchitectureβ layered layout,src/core/
primitives, regression disciplineContributing β adding a new backendβ 5-file recipe, ground-truth diff workflowRegression matrixβtools/test-all-backends.py
capability tiersβ synthetic-audio marking (watermark + C2PA + spoken disclaimer), what counts as a voice clone, the speaker-biometrics boundary, why there is no emotion recognition, and what stays your duty as deployerEU AI ActQuantize modelsβcrispasr-quantize
for all backendsGPU backend selectionDebugging & profilingCredits
Everything below this section is a catalogue β 100+ backends, browse it when you need one. If you just want CrispASR working, this is the whole path. No repo clone, no Python, no model hunting.
Download one file from Releases and unzip it:
| Platform | Download | Notes |
|---|---|---|
| Windows | ||
crispasr-windows-x86_64-cpu.zip |
||
Needs AVX2 (2013+ Intel / 2015+ AMD). Older CPU β β¦-cpu-legacy.zip |
||
| Windows + NVIDIA | ||
crispasr-windows-x86_64-cuda.zip |
||
Self-contained; a CUDA Toolkit install is not required. CUDA-13-native build: β¦-cuda13.zip (Turing+) |
||
| macOS | ||
crispasr-macos.tar.gz |
||
| Metal GPU support built in | ||
| Linux | ||
crispasr-linux-x86_64.tar.gz |
||
β¦-cuda.tar.gz / β¦-vulkan.tar.gz for GPU |
Prefer to build it yourself? See Install & build. The -hip
and -vulkan
builds require the matching driver and do not fall back to CPU;
the Linux -cuda
tarballs do fall back.
Check it runs β this should print a version banner and exit:
crispasr --version # Windows: .\crispasr.exe --version
-m auto
downloads the model on first use (~135 MB here) and reuses it
afterwards β nothing to find or install. It lands in ~/.cache/crispasr/
(%USERPROFILE%\.cache\crispasr
on Windows).
crispasr --backend kokoro -m auto --tts "The quick brown fox jumps over the lazy dog." --tts-output hello.wav
Play hello.wav
. That is the TTS half working.
crispasr --backend parakeet -m auto -f hello.wav -l en
(~467 MB on first run. -l en
skips language auto-detection, which would
otherwise fetch a small extra model.) Both halves now work β swap in your own
.wav
and you are running.
| You want to⦠| Go to |
|---|---|
| Clone a voice from a recording | |
--i-have-rights |
Which backend should I pick?docs/cli.mddocs/streaming.mddocs/server.mdcrispasr --list-backends
Add -v
to any command for verbose progress, and --dry-run-resolve
to print which model files it would open (and whether they're on disk) without anything.
If a command printed its banner and then simply stopped β no error, no output file β that is a crash, not a refusal, and the exit code identifies it in one step. See ** docs/troubleshooting.md**.
CrispASR ships 54 ASR backends for transcription/translation and
55 TTS engines for synthesis. It also ships audio-to-audio S2S backends,
including Sidon restoration and the VoxCPM2 AudioVAE speech upscaler; see the feature matrix
for the complete capability list.
Pick at the CLI with --backend NAME
, or omit it to let the binary auto-detect from the GGUF metadata. Jump to the TTS table for the synthesis side.
| Backend | Model | Architecture | Languages | License |
|---|---|---|---|---|
| whisper | ||||
ggml-base.en.bin |
whisperdistil-whisper/distil-large-v3
parakeetnvidia/parakeet-tdt-0.6b-v3
parakeetnvidia/parakeet-tdt-0.6b-v2
parakeetnvidia/parakeet-tdt-1.1b
parakeetnvidia/parakeet-tdt_ctc-110m
parakeetnvidia/parakeet-tdt_ctc-1.1b
parakeetnvidia/parakeet-tdt_ctc-0.6b-ja
reazonspeechreazon-research/reazonspeech-nemo-v2
fastconformer-ctcnvidia/parakeet-ctc-0.6b
fastconformer-ctcnvidia/parakeet-ctc-1.1b
fastconformer-ctcgrider-transwithai/parakeet-ctc-1.1b-ja
canarynvidia/canary-1b-v2
-sl/-tl
)canary-qwennvidia/canary-qwen-2.5b
lfm2-audioLiquidAI/LFM2.5-Audio-1.5B
lfm2-audioLiquidAI/LFM2.5-Audio-1.5B-JP
mini-omni2gpt-omni/mini-omni2
cohereCohereLabs/cohere-transcribe-03-2026
cohereefwkjn/cohere-asr-ja-v0.1
granite,ibm-granite/granite-speech-{3.2-8b,3.3-2b,3.3-8b}
granite-4.0-1b-speech
more)granite-4.1ibm-granite/granite-speech-4.1-2b
more)granite-4.1-plusibm-granite/granite-speech-4.1-2b-plus
more)granite-4.1-naribm-granite/granite-speech-4.1-2b-nar
more)fastconformer-ctcnvidia/stt_en_fastconformer_ctc_large
voxtralmistralai/Voxtral-Mini-3B-2507
voxtral4bmistralai/Voxtral-Mini-4B-Realtime-2602
qwen3Qwen/Qwen3-ASR-0.6B
qwen3-1.7bQwen/Qwen3-ASR-1.7B
qwen3-ja-animejaykwok/Qwen3-ASR-1.7B-JA-Anime-Galgame-hf
mega-asrzhifeixie/Mega-ASR
higgs-sttbosonai/higgs-audio-v3-stt
more)wav2vec2jonatasgrosman/wav2vec2-large-xlsr-53-english
wav2vec2facebook/data2vec-audio-base-960h
wav2vec2facebook/hubert-large-ls960-ft
glm-asrzai-org/GLM-ASR-Nano-2512
kyutai-sttkyutai/stt-1b-en_fr
kyutai-sttkyutai/stt-2.6b-en
firered-asrFireRedTeam/FireRedASR2-AED
moonshineUsefulSensors/moonshine-{tiny,base}
moonshineβdefidoriel/moonshine-base-de
moonshineβtinyβdefidoriel/moonshine-tiny-de
moonshine-streamingUsefulSensors/moonshine-streaming-{tiny,small,medium}
gemma4-e2bgoogle/gemma-4-E2B-it
gemma4-e4bgoogle/gemma-4-E4B-it
--backend gemma4-e2b
omniasromniASR-CTC-1B-v2
more)**1600+**omniasrβ300momniASR-CTC-300M-v2
more)**1600+**omniasr-llmomniASR-LLM-300M-v2
more)**1600+**omniasr-llmomniASR-LLM-Unlimited-300M-v2
more)**1600+**vibevoicemicrosoft/VibeVoice-ASR
more)vibevoice-bitnetVibeVoice-ASR-BitNet
more)mimo-asrXiaomiMiMo/MiMo-V2.5-ASR
more)ark-asr* experimental/WIP*(basecstr/ark-asr-3b-GGUF
)AutoArk-AI/ARK-ASR-3B
more)moss-audioOpenMOSS-Team/MOSS-Audio-4B-Instruct
more)moss-transcribeOpenMOSS-Team/MOSS-Transcribe-preview-2B
more)moss-diarizeOpenMOSS-Team/MOSS-Transcribe-Diarize-0.9B
whisper*(tiron)** experimental*(baseTrelis/tiron
)Trelis/tiron
funasrFunAudioLLM/Fun-ASR-Nano-2512
fun-asr-mlt-nanoFunAudioLLM/Fun-ASR-MLT-Nano-2512
paraformerfunasr/paraformer-zh
foxnose*(speaker diarization)*Wespeaker/wespeaker-voxceleb-resnet34-LM
--diarize-method foxnose
: WeSpeaker ResNet34-LM 256-d embeddings + GMM/BIC speaker counting + spectral clustering + Viterbi temporal smoothing (more). 3.18 % DER on VoxConverse dev vs the upstream reference implementation's 3.07 %gigaam(baseai-sage/GigaAM-v3
)ai-sage/GigaAM-v3
e2e_rnnt
/ e2e_ctc
emit punctuation + casing + ITN from a SentencePiece vocab, rnnt
/ ctc
emit bare lowercase Cyrillic (more)sensevoiceFunAudioLLM/SenseVoiceSmall
-oj
JSON expose the tags as separate fields. Upstream's emotion classifier is not exposedβ seeEU AI Act| Backend | Model | Architecture | Input / output | License |
|---|---|---|---|---|
sidon |
KevinAHM/Sidon-GGUF |
sarulab-speech/sidon-v0.1
more)voxcpm2-vae, converted withopenbmb/VoxCPM2
--vae-only
more)
huggingface-cli download KevinAHM/Sidon-GGUF sidon-v0.1-f16.gguf --local-dir models
crispasr -m models/sidon-v0.1-f16.gguf -f input.wav --s2s --s2s-output restored.wav
python models/convert-voxcpm2-to-gguf.py --input openbmb/VoxCPM2 \
--output models/voxcpm2-vae-f32.gguf --vae-only
crispasr -m models/voxcpm2-vae-f32.gguf -f input.wav --s2s \
--s2s-output upscaled.wav
Synthesis backends, driven by the --tts
flag and a --tts-output PATH.wav
. See the dedicated Text-to-Speech section below for quick-start commands and engine selection guidance.
| Backend | Models | Architecture | Languages | License |
|---|---|---|---|---|
| miotts | ||||
MioTTS-0.6B |
vibevoice-tts,VibeVoice-Realtime-0.5B
VibeVoice-1.5B
kugelaudiokugelaudio-0-open
qwen3-tts,Qwen3-TTS-12Hz-0.6B-Base
,1.7B-Base
1.7B-VoiceDesign
more)qwen3-tts-customvoice1.7B-CustomVoice
--voice <name>
); optional style via --instruct
(e.g. "spoke very slowly") (more)moss-ttsOpenMOSS-Team/MOSS-TTS-v1.5
--voice ref.wav
; --backend moss-tts -m <backbone> --codec-model <codec>
moss-tts-localOpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5
--backend moss-tts-local -m <backbone> --codec-model <codec>
omnivoicek2-fsa/OmniVoice
more)melottsEN_V2myshell-ai/MeloTTS
more)pipercommunity voicesrhasspy/piper
--g2p-dict
)kokoro+ German backboneshexgrad/Kokoro-82M
SNAC 24 kHz
more)chatterbox+ Nano/turbo/fine-tune variantscstr/chatterbox-GGUF
more)chatterbox-finnish-nano
) fine-tunesindexttscstr/indextts-1.5-GGUF
voxcpm2-ttscstr/voxcpm2-GGUF
--voice <wav>
voxtral-ttsmistralai/Voxtral-4B-TTS-2603
cosyvoice3-ttscstr/cosyvoice3-0.5b-2512-GGUF
--voice <name>
, or any WAV via --voice ref.wav --ref-text "<exact transcript>"
. --backend cosyvoice3-tts-rl
selects upstream's RL-tuned talker (same companions)csmcstr/csm-1b-GGUF
more)lfm2-audio+cstr/lfm2-audio-1.5b-GGUF
jp
dianari-labs/Dia-1.6B
[S1]
/[S2]
tags (use >100-char prompts)zonos-tts+cstr/zonos-v0.1-transformer-GGUF
cstr/dac-44khz-GGUF
more)barkcstr/bark-small-GGUF
.npz
prompts (--voice <file.npz>
)speecht5cstr/speecht5-tts-GGUF
--voice <xvector.bin>
)fastpitchcstr/fastpitch-en-GGUF
more)bananamind-ttsBanaxi-Tech/BananaMind-TTS-V2.1-Preview
more)parler-ttscstr/parler-tts-mini-v1.1-GGUF
--instruct
)outettscstr/outetts-0.3-1b-GGUF
--voice <speaker.json>
)pocket-ttscstr/pocket-tts-GGUF
more)tada+cstr/tada-tts-1b-GGUF
HumeAI/tada-3b-ml
tada-ref.gguf
, custom voices via --voice <tada-ref.gguf>
built with models/convert-tada-ref-to-gguf.py
(more)TTS feature matrix
| Backend | Voice cloning | Sampling | kHz | Auto-download | Flash attn |
|---|---|---|---|---|---|
| vibevoice-tts | yes | temp | 24 | yes | yes |
| qwen3-tts | yes* | temp | 24 | yes | yes |
| omnivoice | yes | temp | 24 | β | β |
| kokoro | β | β | 24 | yes | β |
| orpheus | β | temp | 24 | yes | yes |
| chatterbox | yes | temp | 24 | yes | yes |
| outetts | yes (JSON) | temp | 24 | yes | yes |
| indextts | yes | temp | 24 | yes | yes |
| voxcpm2-tts | yes | β | 48 | yes | β |
| cosyvoice3-tts | yes | temp | 24 | yes | yes |
| f5-tts | yes | β | 24 | yes | β |
| irodori-tts | yes (WAV) | VoiceDesign: --instruct |
|||
| 48 | yes | β | |||
| csm | β | temp | 24 | yes | β |
| dia | β | temp | 44 | yes | β |
| bark | yes (.npz) | temp | 24 | yes | β |
| speecht5 | yes (xvec) | β | 16 | yes | β |
| parler-tts | β | temp | 44 | yes | β |
| fastpitch | β | β | 22 | β | β |
| piper | β | β | 22 | β | β |
| pocket-tts | yes | temp | 24 | yes | β |
| tada | yes | temp | 24 | yes | β |
| dots-tts | yes (--voice ref.wav ) |
||||
| 16-step CFG Euler | 48 | yes | β | ||
| confucius4-tts | yes (--voice ref.wav ) |
||||
| 25-step CFG Euler | 22.05 | yes | β |
- CustomVoice variant only; Base uses baked speakers via
--voice <name>
.
Output language. -tl <lang>
(or -l
) selects the language to speak;
cosyvoice3-tts
, qwen3-tts
and moss-tts
act on it natively. For
cross-lingual cloning β an English reference clip speaking German, the
subtitle-dubbing case β also pass -sl <lang>
for the language the reference is
spoken in, so cosyvoice3 drops the reference transcript instead of carrying its
accent. Over HTTP: "language"
"source_lang"
on POST /v1/audio/speech
. See docs/tts.md.
Text-to-text translation, distinct from the audio-side --translate
flag (which routes audio β English text on whisper / canary / etc.).
Driven by --text "..." -sl <src> -tl <tgt>
.
| Backend | Models | Architecture | Languages | License |
|---|---|---|---|---|
| m2m100 | ||||
facebook/m2m100_418M |
more)m2m100-wmt21+facebook/wmt21-dense-24-wide-en-x
facebook/wmt21-dense-24-wide-x-en
more)en-x
/ x-en
checkpoints)madladgoogle/madlad400-3b-mt
more)
./build/bin/crispasr --backend m2m100 -m auto \
--text "Hello world, how are you today?" \
-sl en -tl de
./build/bin/crispasr --backend m2m100-wmt21 -m auto \
--text "The president said he would not attend." \
-sl en -tl de # uses wmt21-dense-24-wide-en-x
./build/bin/crispasr --backend m2m100-wmt21 \
-m models/wmt21-dense-24-wide-x-en-q4_k.gguf \
--text "Le prΓ©sident a dit qu'il ne serait pas prΓ©sent." \
-sl fr -tl en # uses wmt21-dense-24-wide-x-en
./build/bin/crispasr --backend madlad -m auto \
--text "Hello world." \
-sl en -tl ta
For 2-stage pipelines (e.g., ASR β m2m100), use the dedicated
--tr-sl
/ --tr-tl
flags; they fall back to -sl
/ -tl
when
unset, so single-stage standalone usage is just -sl/-tl
.
Work with all backends.
| Model | Task | Architecture | Languages | License | HuggingFace |
|---|---|---|---|---|---|
| FireRedPunc | |||||
| Punctuation restoration | BERT-base (12L, d=768), 5 classes | Chinese + English | Apache-2.0 | ||
cstr/fireredpunc-GGUF |
fullstop-punccstr/fullstop-punc-multilang-GGUF
punctuate-allcstr/punctuate-all-GGUF
PCS--punc-model pcs
truecaserβlstm--truecase-model lstm
truecaserβcrf--truecase-model crf
truecaserβde--truecase-model auto
CLD3cstr/cld3-GGUF
GlotLID-V3cstr/glotlid-GGUF
LID-176cstr/fasttext-lid176-GGUF
Shared codec modules used by TTS backends. Also available standalone for encode/decode.
| Model | Architecture | Sample Rate | Token Rate | License | HuggingFace |
|---|---|---|---|---|---|
| MioCodec v2 | |||||
| WavLM encoder β FSQ(12800) β Transformer decoder + AdaLN-Zero + SnakeBeta upsampler + iSTFT | 44.1 kHz | 25 Hz (341 bps) | MIT | ||
cstr/miocodec-v2-44k-GGUF |
SNAC 24 kHzcstr/snac-24khz-GGUF
All runtimes share ggml-based inference. The speech-LLM backends (qwen3, voxtral, voxtral4b, granite, glm-asr, kyutai-stt) inject audio encoder frames directly into an autoregressive language model's input embeddings, instead of using a dedicated CTC/transducer/seq2seq decoder. The fastconformer-ctc backend hosts the NeMo FastConformer-CTC standalone ASR family β stt_en_fastconformer_ctc_{large,xlarge,xxlarge}
and the architecturally-identical parakeet-ctc-{0.6b,1.1b}
(different training data + tokenizer, same encoder + head shape) β with greedy CTC decoding. Same C++ runtime as the canary-ctc aligner.
Beyond speech, CrispASR runs several music/audio analysis tasks β each a small GGUF with the architecture auto-detected, no Python. See docs/cli.md for the per-task flags and output formats.
Source separation(--separate
) β split a mix into stems (<input>_<stem>.wav
) viamel-band-roformer(vocal/instrumental, MIT) or** htdemucs**(4-stem).--stems vocals,drums
selects a subset;--sep-output-dir
sets the output location.Piano transcription(--backend piano-transcription
) β piano audio β MIDI note events (88 keys @ 100 fps, ByteDance/Kong CRNN; F16 GGUF β 77 MB).Guitar tablature(--tab
) β per-frame fret-per-string grid viaTabCNN(Wiggins & Kim, ISMIR 2019; CC BY 4.0 weights). The backend emits per-string emission scores, not a decided tablature β run your own constrained Viterbi viacrispasr_session_tab_emissions()
for playable output.Beat / downbeat tracking(--beats
) β beat grid viaBeat This!(CPJKU, ISMIR 2024; MIT for codeandweights, no patent-encumbered DBN).Chord recognition(--chords
) β chord timeline (.lab
) viaBTC(ISMIR 2019). Weights are CC-BY-NC-SA, gated behind--accept-license cc-by-nc-sa-4.0
.Pitch / F0 estimation(--pitch
) β monophonic pitch track viaCREPE(MIT).
Run crispasr --list-backends
to see it live. Each backend declares capabilities at runtime; if you ask for a feature the selected backend does not support, CrispASR prints a warning and silently ignores the flag.
Sortable / filterable view: docs/feature-matrix.html β click any column header to sort, type to filter rows, click cap pills to require a capability. Generated from
crispasr --list-backends-json
(single source of truth β drift impossible). Regenerate via python tools/gen-feature-matrix.py
. A Markdown twin lives at .
docs/feature-matrix.md
The static table below is a curated subset focusing on the ASR backends and the cross-cutting features that matter for ASR pipelines. The full 109-backend Γ 27-cap surface is in the generated views.
| Feature | whisper | parakeet | canary | cohere | granite | graniteβ4.1 | voxtral | voxtral4b | qwen3 | fcβctc | wav2vec2 | glmβasr | kyutaiβstt | firered | moonshine | moonβstream | omniasr | omniasrβllm | vibevoice | gemma4βe2b | mimoβasr | funasr | paraformer | sensevoice |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Native timestamps | β | β | β | β | β | |||||||||||||||||||
| CTC timestamps | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | ||||
| Word-level timing | β | β | β | β | -am |
|||||||||||||||||||
| ββ | -am |
|||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
| β | -am |
|||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
-am |
||||||||||||||||||||||||
| Per-token confidence | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | ||||
| Language auto-detect | β | β | LID | LID | LID | LID | LID | LID | β | LID | LID | β | LID | LID | LID | LID | LID | LID | LID | β | LID | LID | LID | β |
| Speech translation | β | β | β | β | β | β | ||||||||||||||||||
| Speaker diarization | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β |
| Grammar (GBNF) | β | |||||||||||||||||||||||
| Temperature sampling | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | |||||
| Beam search | β | β | β | β | β | β | β | β | β | β | β | |||||||||||||
| Flash attention | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | ||
| Punctuation toggle | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | |||||||||
| Punc restoration | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp |
| Source / target language | β | β | β | β | β | |||||||||||||||||||
Audio Q&A (--ask ) |
- | * | β | * | * | * | * | |||||||||||||||||
| Streaming | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β |
Auto-download (
-m auto) | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | KV quant (CRISPASR_KV_QUANT, plus per-half_K/_V) | β | β | β | β | β | β | β | β | β | β | |||||||||||||| mmap weights (CRISPASR_GGUF_MMAP) | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | β | | | TTS | β |
The matrix above covers 24 ASR backends. Additional ASR backends not shown: nemotron
(39-lang streaming ASR with cache-aware FastConformer + RNN-T), lfm2-audio
(ASR + TTS + S2S in one model), moss-audio
(audio understanding + ASR), moss-transcribe
(Qwen3-Omni encoder + Qwen3-1.7B ASR), mini-omni2
(ASR + TTS + S2S), kugelaudio
(7B audio understanding). See docs/feature-matrix.md for the full 109-backend matrix.
TTS-only backends(
kokoro
, qwen3-tts
- variants,
vibevoice-tts
, orpheus
- DE variants,
chatterbox
/ chatterbox-turbo
/ chatterbox-nano
/ kartoffelbox-turbo
/ lahgtna-chatterbox
, dia
, bark
, outetts
, zonos
, csm
, f5-tts
, irodori-tts
, parler-tts
, speecht5
, piper
, fastpitch
, pocket-tts
, melotts
, cosyvoice3
, voxcpm2
, tada-tts
) all carry the TTS, AUTO_DOWNLOAD, TEMPERATURE, and FLASH_ATTN caps; per-backend cloning + voice-pack support is documented in the Text-to-Speech modelstable above and
. The vibevoice and lfm2-audio columns mark dual-mode (ASR + TTS) backends.
docs/tts.md
Key: β = native/built-in, -am
= via CTC forced aligner (-am canary-ctc-aligner.gguf
or -am qwen3-forced-aligner.gguf
), LID = via external language identification pre-step (-l auto
), pp = via --punc-model
post-processor (FireRedPunc or fullstop-punc), * = experimental or partial support, β = PLUS variant only (native [T:N]
word timestamps with -owts
; base uses -am
). granite-4.1 covers both the regular and -plus
variants; granite-4.1-nar is a non-autoregressive variant with encoder+projector only (no LLM decode features). The KV quant row marks backends that honor CRISPASR_KV_QUANT={f16,q8_0,q4_0}
β CTC-style backends without a KV cache (parakeet, fc-ctc, wav2vec2, kyutai-stt, firered, moonshine variants, omniasr-CTC) don't apply. The same backends also honor the per-half CRISPASR_KV_QUANT_K
/ CRISPASR_KV_QUANT_V
overrides (llama.cpp --cache-type-k
/ --cache-type-v
parity) for asymmetric K-vs-V precision; common recipe K=q8_0 V=q4_0
saves ~40 % more KV memory than symmetric Q8_0. The mmap weights row marks backends consuming core_gguf::load_weights()
and therefore honoring CRISPASR_GGUF_MMAP=1
; whisper itself uses upstream's and is unaffected. See docs/cli.md Memory footprint for usage + recommended combos.
Speaker diarization as a post-processing step via --diarize
:
energy
/xcorr
β stereo-only, no extra depsfoxnose
βbest accuracy, no external deps: WeSpeaker ResNet34-LM embeddings + GMM/BIC speaker counting + spectral clustering + Viterbi smoothing. Estimates the speaker count rather than needing it up front;--diarize-embedder auto
fetches the GGUF (24 MB, CC-BY-4.0). 7.3 % DER on VoxConverse dev wherepyannote
- TitaNet scores 7.8 %, and 3.18 % vs the upstream reference's 3.07 % when scored on turns (more)
pyannote
β native GGUF (no Python, no sherpa-onnx); add--diarize-embedder auto
(TitaNet) or--diarize-embedder indextts
(ECAPA-TDNN) for globally stable speaker IDs across long filessherpa
/ecapa
β externalsherpa-onnxsubprocess; runs once globally on full audio for consistent speaker IDs (#110)vad-turns
β mono-friendly gap-based proxy
The server endpoint supports response_format=diarized_json
for structured speaker-labelled output with normalised speaker letters (A, B, C β¦) β see docs/server.md.
Full reference + tuning knobs (cluster threshold, max speakers, pluggable embedder adapters): see docs/cli.md#diarization.
Language identification for backends without native LID: --lid-backend whisper
(default, 75 MB ggml-tiny.bin), --lid-backend silero
(native GGUF, 16 MB, 95 languages), or --lid-backend firered
(FireRedLID, 1.7 GB, 120 languages β Conformer encoder + Transformer decoder).
Voice activity detection: --vad
uses the default Silero VAD (~885 KB, auto-downloaded). Each VAD segment is transcribed independently, producing separate SRT/VTT entries with correct timestamps. Use --vad --split-on-punct
for best subtitle output. Four VAD backends: Silero (default), FireRedVAD (-vm firered
, recommended), MarbleNet (-vm marblenet
, 439 KB, 6 languages), Whisper-VAD-EncDec (-vm whisper-vad
, experimental).
Punctuation restoration (--punc-model
): CTC-based backends output lowercase without punctuation. Named shortcuts: auto
/firered
(Chinese+English), fullstop
(EN/DE/FR/IT, XLM-R-large), punctuate-all
(12 languages, XLM-R-base), pcs
(47 languages, punc + truecasing + sentence boundary detection in one model). Or pass a GGUF path directly. Also available via Python/Rust/Dart wrappers (crispasr.PuncModel
).
Truecasing (--truecase-model
): Restore German noun/name capitalization in lowercase ASR output. Three options in ascending quality: auto
(statistical, 1.7 MB), crf
(CRF with context, 8.5 MB), lstm
(BiLSTM char-level, 3.2 MB, recommended β 97.9% F1, handles adjective/noun distinction and formal "Ihnen"). All auto-download from cstr/truecaser-de. Or use
--punc-model pcs
for neural punc + truecasing in one pass (47 languages).## Which backends produce punctuation natively?
| Backend | Punctuation | Capitalization | Notes |
|---|---|---|---|
| whisper | β | β | Full punctuation and casing |
| parakeet | β | β | |
| canary | β | β | |
| cohere | β | β | Toggleable via --no-punctuation |
| granite | β | β | LLM output |
| voxtral | β | β | LLM output |
| voxtral4b | β | β | LLM output |
| qwen3 | β | β | LLM output |
| funasr | β | β | LLM output (Qwen3-0.6B decoder). Chinese chars carry full-width period; mlt-nano variant adds Latin-script casing + punctuation. |
| sensevoice | β | β | CTC output with native ITN β toggle off via --no-punctuation , which controls Arabic-digit vs spelled-out numerals + comma/period emission. |
| paraformer | no | ||
| no | |||
NAR character-level output β add --punc-model |
|||
| gigaam | β (e2e_* ) |
||
β (e2e_* ) |
|||
The e2e_* revisions carry punctuation + casing + inverse text normalization in the SentencePiece vocabulary. The charwise ctc / rnnt revisions emit lowercase Cyrillic with no punctuation β but auto-restoration is still suppressed for them, because the auto-enabled FireRedPunc is a Chinese/English model and injects full-width CJK punctuation into Russian. Use an e2e_* revision for punctuated output, or pass an explicit --punc-model . |
|||
| glm-asr | β | β | LLM output |
| kyutai-stt | β | β | LLM output |
| moonshine | β | β | Encoder-decoder output |
| fastconformer-ctc | |||
| no | |||
| no | |||
CTC β add --punc-model |
|||
| wav2vec2 | |||
| no | |||
| no | |||
CTC β add --punc-model |
|||
| firered-asr | |||
| no | |||
| no | |||
CTC β add --punc-model |
|||
| omniasr (CTC) | |||
| no | |||
| no | |||
CTC β add --punc-model |
|||
| omniasr (LLM) | |||
| β | β | Autoregressive decoder |
Other freely-licensed alternatives that could be added: felflare/bert-restore-punctuation (MIT, English, includes truecasing), xashru/punctuation-restoration (Apache-2.0, 40+ languages, BiLSTM-CRF).
Progressive subtitle output (--flush-after
): By default, non-whisper backends buffer all segments and print output at the end. For real-time subtitle consumption (PotPlayer, custom media players), use --flush-after 1
to print each SRT entry to stdout immediately after its VAD segment is transcribed:
crispasr --backend parakeet -m parakeet.gguf --vad --flush-after 1 -osrt -f long_audio.wav
JSON output with language detection: When using -l auto -oj
, the JSON output includes detected language info:
{
"crispasr": {
"backend": "cohere",
"language": "en",
"language_detected": "en",
"language_confidence": 0.977,
"language_source": "ecapa"
},
"transcription": [...]
}
| Need | Pick |
|---|---|
| Battle-tested, all features exposed | whisper |
| Lowest English WER | cohere |
| Fastest (16x realtime on CPU) | |
| moonshine (tiny), fc-ctc (10x) | |
| Multilingual + word timestamps + fast | parakeet (2.9x RT) |
| Multilingual with explicit language control | |
| canary | |
| Speech translation (Xβen or enβX) | |
| canary, voxtral, qwen3 | |
| 30 languages + Chinese dialects | |
| qwen3 | |
| 1600+ languages | |
| omniasr (CTC or LLM) | |
| Realtime streaming ASR (native incremental encoder, ~2Γ RT feed; sub-second-token target deferred to phase 2) | |
| voxtral4b | |
| Highest-quality offline speech-LLM | voxtral |
| Apache-licensed speech-LLM | granite, voxtral, qwen3, omniasr-llm |
| Lightweight CTC-only (fast, no decoder) | |
| wav2vec2, fc-ctc, data2vec, omniasr | |
| Russian | |
gigaam (e2e_rnnt β 8.4 % avg WER, punctuation + ITN), whisper, qwen3 |
|
| Mandarin + Chinese dialects | |
| firered-asr, qwen3, glm-asr, funasr, paraformer, sensevoice | |
| Multilingual (31 langs) speech-LLM | |
| fun-asr-mlt-nano, qwen3, omniasr-llm, gemma4-e2b | |
| Multilingual (50+ langs) + LID + audio-event in one pass | |
| sensevoice (encoder-only CTC, non-AR, 15Γ faster than Whisper-Large) |
Audio-LLM backends (qwen3
, voxtral
, granite
, glm-asr
, etc.) run full transformer decoder stacks (28+ layers, 2048-dim) and are dramatically slower on CPU than encoder-only backends. On older dual-core hardware they can drop below 0.01Γ realtime. If you're on CPU-only hardware:
- Prefer
moonshine(16Γ RT),** fc-ctc**(10Γ RT),** parakeet**(2.9Γ RT), or** whisper**for usable speeds. - Use
--flush-after 1
to see results as each VAD slice completes instead of waiting for the entire file. - Use
-pp
/--print-progress
for per-slice progress indicators on all backends (unified backends show slice-level progress; whisper shows encoder-level progress). - Quantize models to Q4_K or Q5_K to reduce memory and compute.
Cohere, canary, granite, voxtral and voxtral4b need an explicit
language code up front. If you don't know the language, pass
-l auto
and crispasr runs an optional LID pre-step before the main transcribe() call:
crispasr --backend cohere -m $TC/cohere-transcribe-q5_0.gguf \
-f unknown.wav -l auto
These LID providers are available:
--lid-backend whisper
(default) β uses a small multilingual ggml-*.bin model via the crispasr C API. Auto-downloads ~75 MB on first use. 99 languages. -
--lid-backend silero
β native GGUF port of Silero's 95-language classifier. 16 MB F32. Runs as a ggml graph (multi-threaded SIMD on CPU, GPU offload on Metal/CUDA; on Vulkan the graph is routed to CPU pending an upstream kernel fix). Analyzes the first 30 s of audio (CRISPASR_SILERO_LID_MAX_S
overrides);CRISPASR_SILERO_LID_LEGACY=1
restores the old scalar path. -
--lid-backend ecapa
βrecommended: ECAPA-TDNN (Apache-2.0). Purpose-built for language ID. Very high accuracy on TTS benchmark. Two variants via--lid-model
:β VoxLingua107, 43 MB F16, 107 languages, ISO codes (en, de, ...).cstr/ecapa-lid-107-GGUF
Default.β CommonLanguage, 40 MB F16, 45 languages, full names (English, German, ...).cstr/ecapa-lid-commonlanguage-GGUF
--lid-backend firered
β FireRedLID (Conformer encoder + Transformer decoder). Q4_K (544 MB), 120 languages including Chinese dialects. Slower but covers more languages. -
--lid-backend probe
β no second model at all: ask theASR model itself. Transcribes a 20 s clip once per language the model declares and keeps the best-scoring candidate (length Γ text-LID agreement Γ distinct-token ratioΒ², the last term catching the repetitive output a wrong-language prompt produces). Currently implemented bycohere. The reason to prefer it is correctness, not just the saved download: an external detector knows 99 languages while Cohere Transcribe accepts 14 β and its Arabic finetune onlyen
/ar
β so external LID regularly returns a language the model was never trained on, and Cohere answers a wrong languagefluentlyrather than failing. The probe cannot. Cost is one encode + one short decode per candidate, so it runs automatically only for a model with β€ 4 languages (CRISPASR_COHERE_PROBE_MAX_LANGS
);CRISPASR_COHERE_PROBE_TEXTLID=0
drops the text-LID agreement term.The ceiling is about cost, not accuracy. Measured on the real models: the two-language Arabic finetune picksar
for an Arabic clip (p=0.675) anden
forsamples/jfk.wav
(p=0.647); the 14-language base model, probed across all 14, also gets both right (en
p=0.169,ar
p=0.254) β it is simply slower than an external detector. The encoder output is language-independent, so the probe encodesonce and decodes per candidate (encode is ~87 % of a pass); a 14-candidate probe measured12 s β 4-5 s against one-encode-per-candidate, byte-identical output.CRISPASR_COHERE_PROBE_REUSE_ENC=0
restores the naive path.The one soft spot worth knowing: asking the model for a language it was
nottrained on can yield a clean translation rather than garbage, which a text LID then confirms β "fluent French out" is not evidence of French in. That is what makes a mismatched whitelist dangerous, and it is measurable: force a 14-language list onto the two-language Arabic finetune and itsfr
probe returns real French and wins. The real base model'sfr
probe instead code-switches ("Et so, my fellow Americansβ¦", agreement 0.00) and loses, as it should.
These VAD providers are available:
Silero VAD(default) β ~885 KB, auto-downloaded via--vad
. Industry-standard, well-tested.FireRedVADβ DFSMN-based, 2.4 MB, F1=97.57%. Pass--vad -vm firered
to auto-download. Recommended.MarbleNetβ NVIDIA 1D separable CNN, 439 KB, 6 languages (EN/DE/FR/ES/RU/ZH). Pass--vad -vm marblenet
to auto-download. Smallest model. ()cstr/marblenet-vad-GGUF
Whisper-VAD-EncDec*(experimental)*β Whisper-base encoder + TransformerDecoder head, 22 MB Q4_K. Trained on Japanese ASMR; may not generalise well to all domains. Pass--vad -vm whisper-vad
. Slower than others (~1s vs ~50ms). ()cstr/whisper-vad-encdec-asmr-GGUF
Pass --lid-backend off
to skip LID entirely.
Audio LID (above) tags what was spoken; text LID tags what was
written. Text LID runs on a transcript or any UTF-8 string and is
useful for routing post-ASR pipelines (translation, punctuation, sub
selection) without re-running an audio model. Three GGUF families,
one binary β the dispatcher picks by general.architecture
:
| Backend | Labels | Size (F16) | License | HF repo |
|---|---|---|---|---|
| CLD3 (Google compact language detector v3) | ||||
| 109 ISO 639-1 | 440 KB | |||
| Apache-2.0 | ||||
cstr/cld3-GGUF |
GlotLID-V3(cis-lmu fastText)cstr/glotlid-GGUF
LID-176(Facebook fastText)cstr/fasttext-lid176-GGUF
ΒΉ LID-176 is CC-BY-NC-4.0 β non-commercial use only. CLD3 + GlotLID-V3 are Apache-2.0 with no such constraint. Pick CLD3 for the smallest, fastest path; GlotLID for maximum coverage (low-resource languages); LID-176 only if you need its specific 176-label space and accept its non-commercial terms.
Standalone CLI β auto-routes by GGUF arch, with auto-download:
crispasr-lid -m auto --text "Bonjour le monde" # β cstr/cld3-GGUF (default, ~440 KB)
crispasr-lid -m auto:glotlid --text "Bonjour le monde" -k 5
crispasr-lid -m auto:lid-fasttext176 --text "Hallo Welt"
crispasr-lid -m cld3-f16.gguf --text "δ½ ε₯½δΈη"
echo "ΠΡΠΈΠ²Π΅Ρ ΠΌΠΈΡ" | crispasr-lid -m auto --quiet
Post-ASR pipeline β --lid-on-transcript
runs the same dispatcher
on the assembled transcript (also accepts auto[:variant]
):
crispasr -m ggml-tiny.bin -f speech.wav --lid-on-transcript auto
The dispatcher (src/text_lid_dispatch.{h,cpp}
) is a thin C ABI faΓ§ade β one integer compare per call; per-stage diff harness is green at cosβ₯0.999 across 8 multilingual smoke samples.
Don't want to build? Prebuilt binaries for Windows, macOS and Linux are on the releases page β see Start here for which file to take. The rest of this section is for building from source.
git clone --recursive https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
The ggml/
submodule is required. If you cloned without --recursive
, run
git submodule update --init --recursive
first β otherwise CMake stops with a message telling you to do exactly that.
Produces build/bin/crispasr
(main CLI), build/bin/crispasr-quantize
,
and build/bin/crispasr-diff
. No Python, PyTorch, or pip required at runtime β just a C++17 compiler and CMake 3.14+.
For GPU acceleration, add the matching ggml flag at configure time:
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON # NVIDIA
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON # Apple Silicon
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON # cross-vendor
**See ** for the full guide: all GPU backends (CUDA / Metal / Vulkan / MUSA / SYCL), Windows convenience scripts, ffmpeg ingestion, optional BLAS, glibc notes, and the
docs/install.md
scripts/dev-build.sh
wrapper.If a build runs but the binary exits with no output, see docs/troubleshooting.md.
Deeper ASR examples below. If this is your first run, use Start here instead. For TTS, the runnable guide is docs/tts.md (Text-to-Speech below is the model catalogue).
./models/download-ggml-model.sh base.en
./build/bin/crispasr -m models/ggml-base.en.bin -f samples/jfk.wav
curl -L -o parakeet.gguf \
https://huggingface.co/cstr/parakeet-tdt-0.6b-v3-GGUF/resolve/main/parakeet-tdt-0.6b-v3-q4_k.gguf
./build/bin/crispasr -m parakeet.gguf -f samples/jfk.wav
./build/bin/crispasr -m parakeet.gguf -f samples/jfk.wav -ml 1
./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -sl de -tl de
./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -sl de -tl en
./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -l de --translate
./build/bin/crispasr --backend voxtral -m auto -f samples/jfk.wav
./build/bin/crispasr --backend voxtral -m auto -f samples/jfk.wav -l en
./build/bin/crispasr --backend qwen3 -m auto -f audio.zh.wav
./build/bin/crispasr --backend qwen3 -m qwen3-1.7b --auto-download -f audio.wav
./build/bin/crispasr --backend qwen3 -m qwen3-ja-anime --auto-download -f anime.wav
Long audio: the default is safe 30 s chunking. --chunk-seconds 0
decodes
the whole file in ONE pass (matches the reference model verbatim on multi-minute
clips, #218) β but the encoder's full attention is O(NΒ²) in audio length, so
keep single-pass clips under ~10 minutes on 16 GB machines. For long-form use
prefer the plain -q4_k
/-q8_0
GGUFs over the -imatrix
variants (see the model card).
./build/bin/crispasr --backend glm-asr -m auto -f audio.wav
./build/bin/crispasr --backend glm-asr -m auto --chunk-seconds 0 -f long.wav
Note: in single-pass mode the model (like the reference) skips leading
non-speech audio; the default 30 s-chunked mode transcribes more of such
clips. Custom --ask
/ non-English --language
instructions need a GGUF with baked BPE merges (re-published 2026-07; older GGUFs fall back to the default transcription prompt with a warning).
huggingface-cli download cstr/mimo-asr-GGUF mimo-asr-q4_k.gguf \
--local-dir ~/.cache/crispasr
huggingface-cli download cstr/mimo-tokenizer-GGUF mimo-tokenizer-q4_k.gguf \
--local-dir ~/.cache/crispasr
./build/bin/crispasr \
--backend mimo-asr \
-m ~/.cache/crispasr/mimo-asr-q4_k.gguf \
--codec-model ~/.cache/crispasr/mimo-tokenizer-q4_k.gguf \
-f samples/jfk.wav
The 4.5 GB Q4_K is the recommended quant; F16 (14.9 GB) needs ~16 GB
RAM during inference. JFK matches the upstream Python
MimoAudio.asr_sft
reference verbatim; performance on M1+Metal is ~0.3Γ realtime (Q4_K dequant per step is the bottleneck β F16 + KV-reuse follow-ups are queued under PLAN #51a/b/c).
curl -L -o wav2vec2-en-q4k.gguf \
https://huggingface.co/cstr/wav2vec2-large-xlsr-53-english-GGUF/resolve/main/wav2vec2-xlsr-en-q4_k.gguf
./build/bin/crispasr -m wav2vec2-en-q4k.gguf -f samples/jfk.wav
curl -L -o wav2vec2-de-q4k.gguf \
https://huggingface.co/cstr/wav2vec2-large-xlsr-53-german-GGUF/resolve/main/wav2vec2-xlsr-de-q4_k.gguf
./build/bin/crispasr -m wav2vec2-de-q4k.gguf -f audio.de.wav
python models/convert-wav2vec2-to-gguf.py \
--model-dir jonatasgrosman/wav2vec2-large-xlsr-53-german \
--output wav2vec2-de.gguf --dtype f32
./build/bin/crispasr-quantize wav2vec2-de.gguf wav2vec2-de-q4k.gguf q4_k
CrispASR has three feature areas that warrant their own docs pages:
βStreaming & live transcription--stream
,--mic
,--live
, sliding-window chunking, per-token confidence.β Kokoro (multilingual, smallest), Qwen3-TTS (highest fidelity, voice cloning), VibeVoice (lowest-latency streaming), Orpheus (3 B Llama + SNAC), Chatterbox (flow-matching + HiFT vocoder, German via Kartoffelbox), IndexTTS, VoxCPM2, and CosyVoice3 (9 langs + 18 zh dialects; baked-voice bank + arbitrary-WAV cloning). Voice packs, language routing, and qwen3-tts environment switches. All TTS output is watermarked; post-embed verification warns if confidence is low. UseText-to-Speech (TTS)--detect-watermark file.wav
to check any WAV for AI watermarks.β persistent model, OpenAI-compatibleServer mode (HTTP API)/v1/audio/transcriptions
(ASR) and/v1/audio/speech
+/v1/voices
(TTS, automatic on any loaded CAP_TTS backend), per-request voice + speed + instructions, CORS, long-form sentence chunking, API keys, Docker Compose, prebuilt CUDA images.β one transcription already uses multiple cores; the server accepts requests concurrently but serializes inference on one model by default;Concurrency, parallelism & scaling--server-workers N
runs N model instances so pure-ASR requests run concurrently; and for bulk/throughput workloads, process-level fan-out (xargs -P
/ GNUparallel
) or N replicas behind a load balancer. Also covers what isnotsupported (batched multi-stream inference, PagedAttention) and why.
Quickest taste of each:
crispasr --mic -m model.gguf
crispasr --backend vibevoice-tts -m auto --tts "Hello world" --tts-output hello.wav
crispasr --backend cosyvoice3-tts -m auto --tts "Hello world" --tts-output cosy.wav
COSYVOICE3_FLOW_STEPS=5 crispasr --backend cosyvoice3-tts -m auto \
--tts "Hello world" --tts-output cosy-fast.wav
crispasr --server -m model.gguf --port 8080
curl -F "file=@audio.wav" http://localhost:8080/v1/audio/transcriptions
crispasr --server --backend qwen3-tts-customvoice -m auto --voice-dir ./voices --port 8080
curl http://localhost:8080/v1/audio/speech \
-H 'Content-Type: application/json' \
-d '{"input":"Hello world","voice":"vivian"}' -o out.wav
CosyVoice3 uses batched classifier-free guidance and request-sized KV
caching by default. Baked voices load only the LLM, flow, HiFT, and voice
bank; the larger S3 tokenizer and CAMPPlus companions load lazily when a
.wav
cloning voice is first requested.
Common flags:
crispasr -m auto --backend parakeet -f audio.wav --vad -osrt --split-on-punct
| Flag | Meaning |
|---|---|
-m FNAME / --backend NAME |
|
Model path (or auto ) and forced backend |
|
-f FNAME |
|
| Input audio (repeatable; positional accepted) | |
--vad |
|
| Silero VAD chunking β strongly recommended for multi-minute audio | |
-osrt / -ovtt / -otxt / -oj / -ojf |
|
Output formats (also -ocsv , -olrc ) |
|
-am FNAME |
|
| CTC aligner GGUF for word-level timestamps on LLM backends | |
--align-only |
|
Standalone forced alignment: text/.srt + audio β timestamped SRT/JSON (no ASR needed); .srt input keeps its cues and gets re-timed (`--align-granularity auto |
word |
-tp F / -bs N |
|
| Sampling temperature / beam search width | |
-n N / --frequency-penalty F |
|
| Generated-token cap / opt-in repeated-token penalty for supported autoregressive ASR backends | |
-l auto / --detect-language |
|
| LID pre-step for backends without native lang detect | |
--hotwords "A,B,C" |
|
| Contextual biasing β boost named terms during CTC/TDT decode or LLM prompt | |
-ck N |
|
| Fallback chunk size when VAD is off (default 30 s) | |
--list-backends |
|
| Print the capability matrix and exit |
**See ** for the full reference: every flag, VAD details, CTC alignment workflow, output JSON layout, the auto-download registry, and supported audio formats.
docs/cli.md
See for Python / Rust / Dart / Go / Java / JavaScript / Ruby / mobile.
docs/bindings.md
CrispASR is structured as a stable C-ABI in src/
(every algorithm:
VAD, diarize, LID, alignment, cache, registry) consumed by all
language wrappers, with thin presentation layers in examples/cli/
.
Per-model runtimes live in src/{whisper,parakeet,canary,...}.cpp
,
sharing primitives from src/core/
(mel, ffn, attention, GGUF , FastConformer / Conformer / Granite-LLM blocks, etc.).
β full layered layout, file-by-file tour ofdocs/architecture.md
src/
andexamples/cli/
, per-backend internals table, regression discipline.β adding a new backend in five files, clang-format-18 setup, thedocs/contributing.md
crispasr-diff
PyTorch-ground-truth workflow, and the TTS audio-cosine-vs-reference regression target.βdocs/regression-matrix.md
tools/test-all-backends.py
capability tiers, cache modes (keep
/ephemeral
),--skip-missing
for CI.
Shared libraries (cross-repo with CrispEmbed):
crisp_audio/
β Whisper-shape audio encoder (Conv-stem + Transformer)crisp_punc/
β punctuation restoration (FireRedPunc + PCS)crisp_lid/
β text-based language identification (fastText + CLD3)crisp_truecase/
β truecasing (statistical + CRF + BiLSTM)
Both are self-contained static libraries with CMakeLists.txt. CrispEmbed
links them via add_subdirectory(../CrispASR/crisp_*/)
; CrispASR uses them directly. If the shared dir is absent, both repos fall back to local copies of the source files.
For benchmarks see PERFORMANCE.md; for the session-by-session port log and the bug-class lessons, see
LEARNINGS.md
build/bin/crispasr-quantize
is a single, model-agnostic GGUF re-quantization tool that works across all supported model families (Whisper, Parakeet, Canary, Cohere, Voxtral, Qwen3, Granite, Wav2Vec2, MiMo-ASR, GLM-ASR, Moonshine, VibeVoice, Kokoro, Qwen3-TTS, β¦):
./build/bin/crispasr-quantize input.gguf output.gguf q4_k
**See ** for the full guide: supported quant types, K-quant alignment fallback, recommended quant per backend, and worked examples for each architecture.
docs/quantize.md
All backends use ggml_backend_init_best()
which automatically picks the highest-priority compiled backend: CUDA > Metal > Vulkan > CPU. To force a specific backend:
crispasr --gpu-backend vulkan -m model.gguf -f audio.wav
crispasr --gpu-backend vulkan -dev 1 -m model.gguf -f audio.wav
crispasr -ng -m model.gguf -f audio.wav
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 crispasr -m model.gguf -f audio.wav
Build flags: -DGGML_CUDA=ON
, -DGGML_METAL=ON
, -DGGML_VULKAN=ON
.
Notes:
--gpu-backend vulkan
selects the Vulkan backend, but it does not choose which physical GPU to use. Use-dev N
to select the Vulkan device index.- On some Windows laptops, Vulkan device
0
is the Intel iGPU and the NVIDIA GPU is1
. If Vulkan looks unexpectedly slow, rerun with-dev 1
. - The Windows convenience script
build-vulkan.bat
creates a separate Vulkan-capable binary atbuild-vulkan\bin\crispasr.exe
.
For most backends, -v
/ --verbose
surfaces per-stage timings and
device picks. For headless / library use (where the CLI flag isn't
plumbed through), set CRISPASR_VERBOSE=1
instead.
crispasr -v --backend gemma4-e2b -m model.gguf -f audio.wav
HF_TOKEN=hf_xxx crispasr -m auto --backend gemma4-e2b -f audio.wav
The server has its own auth env: CRISPASR_API_KEYS
(see Server mode).
Per-backend debug / bench / dump-dir env vars (developer)
These are useful when porting a new backend or chasing a regression.
The *_BENCH=1
toggles emit per-stage timings even without -v
; the
*_DEBUG=1
toggles emit per-step diagnostic prints; the *_DUMP_DIR=
paths write per-stage F32 tensors for diff-testing against a PyTorch reference (see Debug a new backend against PyTorch ground truth).
| Env var | Purpose |
|---|---|
CRISPASR_VERBOSE=1 |
|
Forces verbose mode for any backend (parallel to the -v flag). |
|
CRISPASR_DUMP_DIR=path/ |
|
Generic per-stage F32 tensor dump for the crispasr-diff harness. |
|
GEMMA4_E2B_BENCH=1 |
|
| Per-stage timings for the Gemma-4-E2B backend. | |
COHERE_BENCH=1 / COHERE_DEBUG=1 |
|
| Cohere transcribe per-stage timings / per-step diagnostics. | |
COHERE_PROF=1 |
|
| Cohere graph-level profiling (per-op timings). | |
COHERE_THREADS=N |
|
| Override thread count for the Cohere backend. | |
| `COHERE_DEVICE=cpu | cuda |
| Force the Cohere backend onto a specific device. | |
COHERE_DUMP_ATTN=path/ |
|
| Dump attention activations for Cohere (used by the diff harness). | |
FIRERED_BENCH=1 |
|
| Per-stage timings for the FireRedASR backend. | |
FIREREDPUNC_DEBUG=1 |
|
| Per-step diagnostics for the FireRed punctuation post-step. | |
MOONSHINE_STREAMING_BENCH=1 |
|
| Per-stage timings for moonshine-streaming. | |
OMNIASR_BENCH=1 / OMNIASR_DEBUG=1 / OMNIASR_DUMP_DIR= |
|
| OmniASR per-stage timings, diagnostics, and stage dumps. | |
PARAKEET_DEBUG=1 |
|
| Parakeet TDT per-step diagnostics (joint network, blank-id sanity). | |
QWEN3_TTS_BENCH=1 / QWEN3_TTS_DEBUG=1 / QWEN3_TTS_DUMP_DIR= |
|
| Qwen3-TTS per-stage timings, diagnostics, and stage dumps. | |
VIBEVOICE_BENCH=1 / VIBEVOICE_DEBUG=1 / VIBEVOICE_DUMP_DIR= |
|
| VibeVoice ASR per-stage timings, diagnostics, and stage dumps. | |
VIBEVOICE_REF_FEATURES=path |
|
| Replace the live encoder with a saved feature tensor (regression harness). | |
VIBEVOICE_TTS_DUMP=path/ |
|
| VibeVoice TTS per-stage dumps (token IDs, base/TTS hidden, neg condition, frame-0 noise/v_cfg/latent/acoustic_embed) for the diff harness. | |
VIBEVOICE_TTS_DUMP_PERFRAME=1 |
|
Per-frame VibeVoice TTS dumps written as perframe_<stage>_f<NNN>.bin . Pair with VIBEVOICE_TTS_DUMP=path/ and VIBEVOICE_TTS_NOISE=path for stage-by-stage AR diff against tools/run_official_vibevoice.py . |
|
VIBEVOICE_TTS_TRACE=1 |
|
Extra one-line traces (negative-condition prefill rms, scaling/bias factors loaded). Same effect as library verbosity β₯ 2; there is no CLI flag for it (-v caps verbosity at 1). |
|
VIBEVOICE_VOICE_AUDIO=path.wav |
|
Reference voice WAV for 1.5B-base TTS without a .gguf voice cache. |
|
VIBEVOICE_TTS_NOISE=path |
|
Override the per-frame Gaussian init noise. Flat little-endian float32 [N_frames, vae_dim] β typically the noise.bin written by tools/run_official_vibevoice.py . |
|
| `VIBEVOICE_VAE_BACKEND=cpu | metal |
| Pin the VAE decoder onto a specific backend. | |
WAV2VEC2_BENCH=1 / WAV2VEC2_VERBOSE=1 / WAV2VEC2_DUMP_DIR= |
|
| wav2vec2 per-stage timings, verbose graph traces, and stage dumps. | |
CRISPASR_VOXTRAL4B_STREAM_TIMING=1 |
|
| Per-stage timings for the voxtral4b streaming path (encoder drain / prefill / first-text-token / decode-step p50/p95). | |
CRISPASR_VOXTRAL4B_STREAM_CHUNK_MS=N |
|
| Override the internal encoder chunk size (default 240 ms). Must be a multiple of 80 ms. Larger = faster feed (kernel-launch amortisation), longer live-caption latency floor. | |
CRISPASR_VOXTRAL4B_STREAM_BATCH_ENCODER=1 |
|
| Regression-debug: ignore the streaming encoder's audio_embeds and re-run the whole batch encoder at flush. | |
CRISPASR_VOXTRAL4B_STREAM_DEBUG=1 / CRISPASR_VOXTRAL4B_STREAM_DIFF=1 |
|
| Per-step decode prints / side-by-side encoder cosine vs the batch encoder. | |
CRISPASR_VOXTRAL4B_STREAM_LIVE=1 |
|
Live-captions decode-during-feed (PLAN #7 phase 3). get_text() polled during feed returns progressive transcript. Default OFF (PTT semantics). Wrappers: Python Session.stream_open(live=True) , Rust stream_open_ex(.., live: true) . |
|
CRISPASR_VOXTRAL4B_STREAM_DECODER_THREAD=1 |
|
Decoder worker thread (PLAN #7 phase 4, implies live mode). Lets feed() return between encoder chunks without waiting for the decode loop β useful for mic-driven workloads. On M1 the Metal queue serializes encoder and decoder so total wall-clock is unchanged; faster GPUs with kernel-level parallelism see real overlap. |
|
CRISPASR_VOXTRAL4B_FUSED_QKV=0 |
|
| Opt out of the runtime fused-QKV LLM path (default-on, ~7-8 % decode speedup on M1 Q4_K, ~500 MB extra memory). | |
CRISPASR_QWEN3_ASR_FUSED_QKV=0 |
|
| Opt out of the runtime fused-QKV LLM path for qwen3-asr (default-on; works on F16/F32/Q4_K/Q8_0/...). | |
CRISPASR_VOXTRAL_FUSED_QKV=1 |
|
| Opt in to the runtime fused-QKV LLM path for voxtral 3B. Off by default (no measurable speedup on JFK-shape decodes; useful for long-form workloads where decode dominates). | |
QWEN3_TTS_FUSED_QKV=1 |
|
| Opt in to the runtime fused-QKV talker path. | |
GRANITE_DISABLE_ENCODER_GRAPH=1 |
|
| Force the granite-speech / -plus / -nar encoder back to the per-layer CPU loop (slower but kept around for debugging). The single ggml-graph encoder with per-layer Shaw RPE is the default and is bit-near-identical to the CPU loop while being ~2Γ faster end-to-end across all three variants. | |
CRISPASR_NO_REL_POS=1 |
|
| Ablate the relative-position bias in the Gemma-4 audio encoder (development only). | |
ECAPA_REF_FBANK=path |
|
| Reference filterbank tensor for the ECAPA-TDNN LID model (regression harness). | |
CRISPASR_SHERPA_LID_BIN=path |
|
| Override the auto-detected sherpa-onnx LID binary. | |
CRISPASR_ARG_DEVICE=N |
|
Default GPU device index when -dev isn't passed. |
|
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 |
|
| Let CUDA swap to RAM when VRAM is exhausted. | |
GGML_VK_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES |
|
| Standard ggml/CUDA device-visibility filters. |
HF_TOKEN
and HUGGING_FACE_HUB_TOKEN
are both honoured for gated-model downloads (in that order).
β the original ggml inference engine and Whisper runtime this fork is built onwhisper.cppβ the tensor library everything runs onggmlNVIDIA NeMoβ parakeet-tdt-{0.6b-v2,0.6b-v3,1.1b}, parakeet-tdt_ctc-{110m,1.1b,0.6b-ja}, parakeet-ctc-{0.6b,1.1b}, canary-1b-v2, canary-ctc aligner, and the FastConformer-CTC family (stt_en_fastconformer_ctc_{large,xlarge,xxlarge} plus CTC branches of the stt_*_fastconformer_hybrid_large[_pc] fleet: en-pc, de, es, fr, it, nl, pl, ru, ua, hr, be, ar, fa, ka, hy, uz, kk-ru β all usable both as ASR backends and as compact ~82 MB-am
forced aligners)Cohereβ cohere-transcribe-03-2026** Qwen team (Alibaba)β Qwen3-ASR-0.6B, Qwen3-ASR-1.7B, Qwen3-ForcedAligner-0.6B Mistral AI**β Voxtral Mini 3B and 4B Realtime** IBM Granite team**β Granite Speech 3.2-8b, 3.3-2b, 3.3-8b, 4.0-1b** Meta / wav2vec2**β wav2vec2 CTC models (XLSR-53 English, German, multilingual via any Wav2Vec2ForCTC checkpoint)β optional diarization via subprocess (ONNX models)sherpa-onnxβ VAD (native GGUF) and language identification (native GGUF, 95 languages)Sileroβ speaker diarization segmentation (native GGUF port)pyannote+miniaudio+stb_vorbisβ embedded/linked audio decoders (WAV/MP3/FLAC/AIFF/OGG/Opus, no ffmpeg; AAC/M4A/ALAC via Apple AudioToolbox)libopus/opusfile(MIT) β in-tree clean-room codec suite: MP3/AAC-LC/Ogg-Opus output encoders (TTSglint.mp3
/.aac
/.opus
) and cross-platform ADTS AAC-LC + Ogg Opus input decoders (no libopus needed)(Anthropic) β significant portions of the crispasr integration layer, all model converters, and the FastConformer/attention/mel/FFN/BPE core helpers were co-authored with ClaudeClaude Code
Same as upstream whisper.cpp: MIT.
Per-model weights are covered by their respective HuggingFace model licenses (see Supported backends). The crispasr
binary itself links model runtimes that are mostly permissively licensed (MIT / Apache-2.0 / CC-BY-4.0 for weights).