{"slug": "rumblex-word-timestamps-and-subtitles-from-rumble-2-flash-s-ctc-head", "title": "Rumblex – word timestamps and subtitles from Rumble-2-Flash's CTC head", "summary": "Independent developer Edward Glockner released rumblex, a pip-installable transcription tool built on Reson8's 0.5B-parameter Rumble-2-Flash speech recognition model that derives word-level timestamps from the model's auxiliary CTC head by force-aligning decoder text to audio, reusing the encoder pass so alignment costs one linear layer plus a Viterbi search rather than a second model. rumblex splits long recordings with Silero VAD, batches transcription, adds optional per-word speaker labels via pyannote, and writes JSON, SRT, VTT (with optional per-word karaoke tags), TSV and TXT; on a LibriSpeech run with a vLLM server it decoded at 601× real time versus 467× without vLLM, at the same WER and timing error. The tool requires Python 3.11 or 3.12 and ffmpeg, runs on a 6 GB GTX 1060 in float32 at the default batch size of 8, and is not affiliated with Reson8.", "body_md": "Transcription with word-level timestamps, speaker labels and subtitles, built on\n[Reson8/Rumble-2-Flash](https://huggingface.co/Reson8/Rumble-2-Flash).\n\nRumble-2-Flash ships an auxiliary CTC head next to its decoder. rumblex uses it for word timestamps: the decoder writes the text, and the CTC head force-aligns that text to the audio. It reuses the encoder pass the decoder already ran, so alignment costs one linear layer plus a Viterbi search, not a second model.\n\nRumble-2-Flash is a 0.5B-parameter speech recognition model fine-tuned from Qwen3-ASR-0.6B. Its\n[model card](https://huggingface.co/Reson8/Rumble-2-Flash) lists 45 languages; the benchmarks here cover\nEnglish and Dutch.\n\nrumblex is an independent project and is not affiliated with Reson8.\n\n- Reads anything ffmpeg can (wav, mp3, m4a, mp4, mkv, ...). Long recordings are split into speech chunks with Silero VAD.\n- Batched transcription. A chunk's output doesn't depend on what it is batched with. The model's\n`language` ,`prompt` (context / hotwords),`digits` ,`filler` and`verbatim` options are exposed.\n- Word timestamps from the CTC head. If alignment fails on a chunk, its words get proportional times instead, so a long job doesn't stop.\n- Optional speaker labels via pyannote, per word. Segments are split where the speaker changes.\n- Writes JSON, SRT, VTT (optionally with per-word karaoke tags), TSV and TXT.\n\nWord times and speaker labels from one run, played back in real time over the audio's waveform.\n\n```\npip install rumblex\npip install \"rumblex[diarize]\"   # with speaker labels (pyannote)\n```\n\nYou also need Python 3.11 or 3.12 and ffmpeg on `PATH`. pip takes PyTorch from PyPI, which is a CPU-only build\non Windows. For a GPU on Windows, or for an older GPU, install torch and torchaudio for your CUDA version\nfrom [pytorch.org](https://pytorch.org/get-started/locally/) first. GPUs from Ampere on run in bf16,\nolder ones (e.g. Pascal) in float32. A GPU is strongly recommended; the model runs on a 6 GB GTX 1060 in\nfloat32 at the default batch size of 8. CPU works but is slow.\n\nFrom source:\n\n```\ngit clone https://github.com/EdwardGlockner/rumblex && cd rumblex\nuv sync                     # PyTorch's CUDA 12.6 builds, which still support Pascal\nuv sync --extra diarize\n```\n\nThe lock file pins PyTorch's CUDA 12.6 wheels on Linux and Windows and PyPI's build on macOS. For RTX\n50-series GPUs (which need CUDA 12.8+), install torch yourself and use `pip install -e .` instead.\n\n```\nrumblex meeting.mp4                                   # all formats into the current folder\nrumblex talk.wav -f srt vtt --max_line_width 42       # subtitles with cues split at 42 chars\nrumblex interview.m4a --language Dutch --diarize -f srt txt\nrumblex call.wav --prompt \"Kubernetes, metoprolol\"   # context / hotwords for the model\npython\nimport rumblex\n\nmodel = rumblex.load_model()  # Reson8/Rumble-2-Flash, GPU if available\nresult = rumblex.transcribe(\"meeting.mp4\", model)  # segments with word timestamps\nfor seg in result.segments:\n    print(seg.start, seg.end, seg.text)\nrumblex.write_result(result, \"meeting.mp4\", \"out/\", [\"srt\", \"json\"])\n```\n\nMore in [examples/quickstart.py](https://github.com/EdwardGlockner/rumblex/blob/main/examples/quickstart.py) and\n[examples/batch_subtitles.py](https://github.com/EdwardGlockner/rumblex/blob/main/examples/batch_subtitles.py), which writes subtitles for a folder of files.\n\nOn Linux with a recent NVIDIA GPU, decoding can run on a [vLLM](https://github.com/vllm-project/vllm)\nserver. `pip install \"rumblex[vllm]\"` adds the client; vLLM itself goes wherever the server runs:\n\n```\nvllm serve Reson8/Rumble-2-Flash --max-model-len 4096 --gpu-memory-utilization 0.5\nrumblex meeting.mp4 --vllm_url http://localhost:8000/v1\n```\n\nWord timestamps still come from a local pass of the audio encoder, so leave GPU memory for it. On our\nLibriSpeech run this was 601× real time against 467× without vLLM, with the same WER and timing error\n([docs/bench_2026-11-20_rtx4090_release](https://github.com/EdwardGlockner/rumblex/blob/main/docs/bench_2026-11-20_rtx4090_release/README.md)).\n\nThe pyannote models are gated. Accept the conditions on\n[pyannote/speaker-diarization-community-1](https://huggingface.co/pyannote/speaker-diarization-community-1),\nthen run `hf auth login`. pyannote.audio 4 sends usage telemetry by default. rumblex turns it off\nunless you set `PYANNOTE_METRICS_ENABLED=true` yourself.\n\nBoth systems ran on the same audio on one rented RTX 4090, in one session, with the code released as\n0.1.0: rumblex in bf16 at batch 32 (the default is 8), and WhisperX 3.8.6 with `large-v3` in float16 at\nbatch 16 with its default wav2vec2 aligners (large-v3-turbo was not measured). The language was given to\nboth. For the English sets rumblex ran with `--no_digits`, since their references spell numbers out.\nWER uses Whisper's normalizers. Speed is one long file per dataset (20–60 min) made from that dataset's\naudio, timed from decoded audio to aligned words with the models already loaded. Raw results, settings\nand caveats:\n[docs/bench_2026-11-20_rtx4090_release](https://github.com/EdwardGlockner/rumblex/blob/main/docs/bench_2026-11-20_rtx4090_release/README.md).\n\n| dataset | rumblex WER | WhisperX WER | rumblex speed | WhisperX speed | \n|---|---|---|---|---|\n| AMI meetings (500 utt.) | 10.9% | 17.2% | 329× real time | 104× | \n| Earnings-22 calls (500 utt.) | 9.3% | 10.6% | 430× | 102× | \n| LibriSpeech test-clean (200 utt.) | 2.2% | 1.6% | 467× (601× with vLLM) | 92× | \n| FLEURS Dutch (364 utt.) | 9.2% | 7.0% | 380× | 79× | \n\nrumblex was 3–5× faster on every set. Over 40% of WhisperX's time goes to its alignment step (7–17 s per file); rumblex's alignment reuses the encoder pass and took 0.3–0.9 s. rumblex had the lower WER on meetings and earnings calls, WhisperX on clean read speech and on Dutch. Most of rumblex's lead on AMI and Earnings-22 is fewer deleted words (177 vs 353 and 231 vs 427). rumblex passes clips of up to 20 s to the model whole, while WhisperX runs its VAD on every clip, so on long recordings, where both use a VAD, the gap may be smaller. These are samples of a few hundred utterances per set, and speeds also depend on the host CPU.\n\nThe [model card](https://huggingface.co/Reson8/Rumble-2-Flash) reports 6.05% on FLEURS Dutch under the\nOpen ASR Leaderboard's scoring, which we did not reproduce. An earlier diagnostic run (vLLM vs\ntransformers, bf16 vs float32, with and without VAD) gave 9.2–9.6% every time, so none of those pipeline\nchoices explains the gap. Part of it is that FLEURS mixes digits and spelled-out numbers in its\nreferences ([docs/bench_2026-11-20_rtx4090_diagnose](https://github.com/EdwardGlockner/rumblex/blob/main/docs/bench_2026-11-20_rtx4090_diagnose/README.md)).\n\nMeasured on the 200 LibriSpeech utterances in the same run, against Montreal Forced Aligner reference\nalignments. The settings were chosen on `dev_clean` ([docs/alignment_eval.md](https://github.com/EdwardGlockner/rumblex/blob/main/docs/alignment_eval.md)).\n\n| mean absolute error | rumblex | WhisperX | \n|---|---|---|\n| word start | 58 ms | 75 ms | \n| word end | 57 ms | 34 ms | \n\nThe Rumble encoder produces 13 frames per second (80 ms apart within each second), while WhisperX's aligner works at 20 ms, and that shows in the word ends. rumblex moves a word's end back to where the audio goes quiet, which brought its end error down from 75 to 57 ms in development. The reference is another forced aligner, not human labels.\n\n1. ffmpeg decodes the audio to 16 kHz mono.\n2. Silero VAD finds the speech, running over many stretches of the file at once (on the GPU if there is one). Speech is packed into chunks of up to 20 s. Pauses under 0.5 s don't split speech unless a chunk would run past 20 s, and pauses over 2 s never end up inside a chunk. Audio of 20 s or less skips VAD and goes in whole.\n3. Rumble-2-Flash transcribes the chunks in batches. A hook keeps the encoder's `ln_post` states\n(13 frames/s).\n4. The CTC head and a Viterbi search align the decoder's own tokens to those states. Tokens are grouped into words at the leading-space marker. A word ends where the next one starts, or earlier if the audio goes quiet before that.\n5. Optionally, pyannote labels speakers and each word takes the speaker it overlaps most.\n6. The writers produce JSON, SRT, VTT, TSV and TXT.\n\nModel details measured for this project: [docs/model_inspection.md](https://github.com/EdwardGlockner/rumblex/blob/main/docs/model_inspection.md).\n\n- Word timing resolution is limited by the encoder's frame rate (13 per second). Word ends in particular are less precise than WhisperX's 20 ms aligner (57 vs 34 ms mean error on LibriSpeech).\n- Each chunk is decoded on its own, so a sentence that runs across a chunk boundary in long audio can be broken there, sometimes with a capital letter mid-sentence.\n- Words are split at spaces, so languages written without spaces (Chinese, Japanese) get one \"word\" per chunk.\n- On clean read English and on FLEURS Dutch, WhisperX large-v3 had the lower WER in our runs. Only English and Dutch have been measured.\n- Speaker labels need the `[diarize]` extra and access to pyannote's gated models.\n- The vLLM backend needs Linux and a GPU vLLM supports.\n\n```\nuv sync --extra dev\nuv run pytest\nuv run ruff check . && uv run ruff format --check .\n```\n\nBenchmarks and research scripts: [scripts/README.md](https://github.com/EdwardGlockner/rumblex/blob/main/scripts/README.md). Every measured number in this\nREADME links to its raw results; the full list is in [docs/README.md](https://github.com/EdwardGlockner/rumblex/blob/main/docs/README.md).\n\nApache-2.0. Rumble-2-Flash is Apache-2.0 as well. The pyannote models have their own terms.", "url": "https://wpnews.pro/news/rumblex-word-timestamps-and-subtitles-from-rumble-2-flash-s-ctc-head", "canonical_source": "https://github.com/EdwardGlockner/rumblex", "published_at": "2026-11-21 14:00:41+00:00", "updated_at": "2026-11-21 14:13:37.767550+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "natural-language-processing", "ai-infrastructure", "developer-tools"], "entities": ["rumblex", "Edward Glockner", "Reson8", "Rumble-2-Flash", "Qwen3-ASR-0.6B", "Silero VAD", "pyannote", "vLLM"], "also_reported_by": [], "takeaway": "rumblex gets word-level timestamps from Rumble-2-Flash's auxiliary CTC head by force-aligning the decoder's text against the encoder pass already computed, costing one linear layer plus a Viterbi search instead of a second model.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/rumblex-word-timestamps-and-subtitles-from-rumble-2-flash-s-ctc-head", "markdown": "https://wpnews.pro/news/rumblex-word-timestamps-and-subtitles-from-rumble-2-flash-s-ctc-head.md", "text": "https://wpnews.pro/news/rumblex-word-timestamps-and-subtitles-from-rumble-2-flash-s-ctc-head.txt", "jsonld": "https://wpnews.pro/news/rumblex-word-timestamps-and-subtitles-from-rumble-2-flash-s-ctc-head.jsonld"}}