{"slug": "shipping-local-speech-to-text-inside-a-tauri-app", "title": "Shipping local speech-to-text inside a Tauri app", "summary": "Set, a fully offline Markdown notes app built on the Tauri framework, ships local dictation using OpenAI's Whisper small model in its 5-bit quantized form via whisper.cpp, a 190 MB download that adds 2.8 MB to the app's binary. On 1,000 LibriSpeech clips (almost 20,000 words), the q5_1 model produced a 4.2% word error rate on clean speech and 10.8% on harder recordings, and on a 2020 M1 MacBook Pro it transcribed a one-minute take in 3.5 seconds with the GPU and 4.9 seconds on CPU alone, peaking at 346 MB of memory. The developer chose whisper.cpp because platform speech APIs differ across macOS, Windows and Linux, and linked it into the Rust app through whisper-rs 0.16 bindings with Metal enabled.", "body_md": "# Shipping local speech-to-text inside a Tauri app\n\nSet is a fully offline notes app that keeps every page as a Markdown file on your own disk. It also has local dictation as an opt-in feature. The speech recognition runs on device, using OpenAI's Whisper model through [whisper.cpp](https://github.com/ggml-org/whisper.cpp).\n\nSet ships the 5-bit quantized version of Whisper small, a 190 MB download. On 1,000 [LibriSpeech](https://www.openslr.org/12) clips (almost 20,000 words), it got 3.9% of words wrong on clean speech and 10.2% on harder recordings. On a 2020 M1 MacBook Pro, once warmed up, it transcribes a minute of speech in 3.5 seconds with the GPU, or 4.9 seconds on the CPU alone, using under 350 MB of memory with the GPU. The code adds 2.8 MB to the app's binary.\n\nThis is a rundown of why I built it, how it fits inside a Tauri app, and the benchmarks behind the model.\n\n## Why I built dictation\n\nI think out loud. When I'm working through a problem or an idea, I talk it through, usually, and the structure I end up writing often starts from this random stream of thinking. I could also be reading an article and talking through the interesting points in it.\n\nSo I wanted speaking to be a way to start a page, or an idea inside one, and then shape the writing from there. This has to be on device and work offline, both for privacy and latency reasons. It has to be accurate enough, but it’s a draft, not a transcript, it doesn't need to be perfect.\n\n## Why whisper.cpp\n\nPlatforms do offer their own speech recognition, but the APIs differ across macOS, Windows and Linux (Linux has no standard one), and Set runs on all three.\n\nwhisper.cpp is a C/C++ port of Whisper with no runtime dependencies, and it uses Apple's Metal GPU API on Macs. Set is a Rust app ([Tauri](https://tauri.app)), so it links whisper.cpp through the `whisper-rs` bindings, which compile whisper.cpp from source as part of the Cargo build. It’s the same model and code running on each platform.\n\n## Choosing the model\n\nWhisper comes in sizes from tiny (39M parameters) to large (1.55B). whisper.cpp publishes them as single `ggml` files, including quantized variants. I benchmarked (warm) six on the machine I develop on:\n\n- **Machine:** MacBook Pro (13-inch, M1, 2020), 16 GB, macOS 14.2.1\n- **Library:**`whisper-rs` 0.16 with Metal on\n- **Settings:** exactly as Set runs them: greedy decoding, language auto-detected, no timestamps, 7 threads\n- **Accuracy:** word error rate (WER) on 100 random utterances each from[LibriSpeech](https://www.openslr.org/12)`test-clean` (13.4 minutes, 2,200 words) and`test-other` (10.7 minutes, 1,706 words; harder speakers and recording conditions)\n- **Latency:** time to transcribe a 30, 60 and 120 second recording, the length of a real dictation take, median of 3\n- **Memory:** peak memory footprint as macOS reports it (`/usr/bin/time -l` )\n\n| Model | Download | WER clean | WER other | 30 s take | 60 s take | 120 s take | Peak memory | \n|---|---|---|---|---|---|---|---|\n| tiny | 78 MB | 7.6% | 20.8% | 0.8 s | 1.8 s | 3.5 s | 165 MB | \n| base | 148 MB | 5.8% | 16.1% | 1.1 s | 1.3 s | 3.9 s | 237 MB | \n| small | 488 MB | 4.0% | 11.0% | 2.1 s | 3.5 s | 5.4 s | 627 MB | \n| **small, q5_1** | **190 MB** | **4.2%** | **10.8%** | **2.1 s** | **3.5 s** | **4.4 s** | **346 MB** | \n| medium | 1.5 GB | 3.4% | 8.7% | 5.4 s | 6.9 s | 10.8 s | 1,795 MB | \n| large-v3-turbo | 1.6 GB | 2.5% | 7.6% | 5.4 s | 8.7 s | 14.0 s | 1,793 MB | \n\nWER uses a simple normalizer (lowercase, no punctuation, numbers spelled out), which is stricter than the one in the Whisper paper, so absolute numbers run slightly higher than published ones.\n\n**Why small:**\n\n- **tiny and base miss too much:** On clean speech they're fine, but on harder audio they get one word in five or six wrong.\n- **small is the best trade:** It cuts base's errors on hard audio by a third (16.1% → 11.0%), and a one-minute take comes back in 3.5 seconds.\n- **medium and turbo cost more than they provide:** They take two to three times as long and nearly three times the memory, for 0.6–1.5 points of WER on clean speech and 2.3–3.4 on hard audio.\n- **Multilingual, not**`.en`**:** Whisper also comes in English-only variants, but Set detects the language by default and lets you pick one of 15 languages, so it needs the multilingual model.\n\nThese numbers are from an M1 with Metal. On Windows and Linux, Set's builds run whisper.cpp on the CPU only (more below).\n\n### The quantized model\n\n`small-q5_1` is the same model with its weights quantized to 5 bits. On the first 200 clips it looked as accurate as the full model, so I ran both on 1,000 clips (almost 20,000 words) and compared them clip by clip:\n\n|  | small | small, q5_1 | \n|---|---|---|\n| clean, 10,468 words | 3.93% | 3.90% | \n| other, 9,194 words | 10.23% | 10.18% | \n\nThere's no measurable difference. On 468 of the 500 clean clips they make exactly the same number of errors, and the rest split evenly: 16 better, 16 worse. It's as fast or faster, a 190 MB download instead of 488 MB, and uses about 280 MB less memory, with Metal and on the CPU.\n\n### Language detection\n\nOn 6 of the 1,000 clips, each model decided that English speech was another language and transcribed it as Polish, Portuguese or Welsh, mostly on short clips of 5 to 8 words. Choosing English in settings fixed 6 of the 7 clips involved.\n\n## Audio\n\nAudio comes in through `cpal` from the default input device at whatever format and rate the device prefers, which in practice is 44.1 or 48 kHz, often stereo, as floats or integers depending on the platform and device. Whisper wants 16 kHz mono floats. So Set:\n\n- **Accepts every common sample format** (f32, i16, u16, i32, i8, u8) and converts to f32.\n- **Downmixes to mono** by averaging the channels.\n- **Buffers the whole take in memory.** Five minutes of mono f32 at 48 kHz is about 58 MB.\n- **Reports the input level** every 66 ms, for the level meter while you talk.\n- **Caps a take at five minutes** , then stops recording and transcribes what it has.\n\nWhen you stop, it resamples to 16 kHz by averaging each window of input samples rather than picking every third one, so high frequencies don't fold back into the speech band as noise (aliasing). A proper low-pass filter would be better, but averaging is enough for speech going into Whisper.\n\n## The engine\n\nTranscription only happens once (for now) when you stop talking:\n\n``` js\nlet mut params = FullParams::new(SamplingStrategy::Greedy { best_of: 1 });\nparams.set_n_threads(threads()); // cores - 1, at most 8\nparams.set_language(Some(language.unwrap_or(\"auto\")));\nparams.set_no_timestamps(true);\nparams.set_suppress_blank(true);\nparams.set_suppress_nst(true); // no \"[Music]\" or \"(coughs)\"\n```\n\n- **Greedy decoding** , one candidate at a time, the fastest way to decode.\n- **The model is loaded for each take and dropped afterwards.** Loading it takes about 90 ms once the file is in the OS cache. The first dictation after launching the app is much slower, because the model is read from disk and Metal compiles its GPU kernels: with the full model, Set's log shows 8 seconds of speech taking 13.6 seconds, and 11 seconds taking 1.8 seconds two minutes later.\n- **The text is inserted where you're writing.** It replaces the selection, or goes in at the cursor.\n- **whisper.cpp and ggml log constantly to stderr** , which a bundled app sends to the system log. Set installs log callbacks that drop everything below a warning and route the rest into its own log. It never logs what you said.\n\n## Costs\n\n**Build:** `whisper-rs` compiles whisper.cpp from C/C++ source. From scratch, that takes about 24 seconds on this 8-core M1, and longer on smaller CI runners.\n\n**Binary size:** The arm64 `set` binary is 32.7 MB with dictation and 29.9 MB without, so whisper.cpp, the audio library and the HTTP client add 2.8 MB (2.4 MB after stripping symbols). The model is the real cost, and that's downloaded only by people who turn dictation on.\n\n**Model download:** The model isn't bundled with the app. It would turn a 74 MB app into a 264 MB one for everyone, including people who never dictate. Instead, the first time you turn dictation on, Set downloads it from the whisper.cpp model repository on Hugging Face into the app's data folder.\n\n**GPU support:** Metal on macOS only. whisper.cpp also supports CUDA, Vulkan and HIP, but those need the vendor's toolkit at build time, which I can't assume on every build machine. So on Windows and Linux, dictation runs on the CPU for now.\n\nMeasured on the same M1 with the GPU turned off:\n\n| small, M1 | WER clean | WER other | 30 s take | 60 s take | 120 s take | Peak memory | \n|---|---|---|---|---|---|---|\n| Metal | 4.0% | 11.0% | 2.1 s | 3.5 s | 5.4 s | 627 MB | \n| CPU only | 4.0% | 11.0% | 3.1 s | 5.0 s | 8.3 s | 968 MB | \n| **CPU only, q5_1** | **4.2%** | **10.8%** | **2.8 s** | **4.9 s** | **7.3 s** | **655 MB** | \n\nWith the full model, 199 of the 200 transcripts are identical to the Metal run (194 with q5_1). It's about 50% slower and still usable: a minute of speech in five seconds. That's an M1's CPU, though, which is fast for a laptop. I haven't measured on the older x86 laptops that many Windows and Linux users will have.\n\n## Next\n\n- \n**Benchmark other languages.** The benchmarks are only run against English speech.\n- \n**Streaming.** Right now you see nothing until you stop talking. whisper.cpp can transcribe in overlapping windows as audio arrives, which would show text as you speak.\n- \n**GPU backends beyond Metal** , once they can be built reliably on each platform's release runner.", "url": "https://wpnews.pro/news/shipping-local-speech-to-text-inside-a-tauri-app", "canonical_source": "https://writewithset.com/blog/shipping-local-speech-to-text-inside-a-tauri-app", "published_at": "2026-09-23 23:56:57+00:00", "updated_at": "2026-09-24 00:30:57.224277+00:00", "lang": "en", "topics": ["ai-tools", "natural-language-processing", "developer-tools", "ai-products"], "entities": ["Set", "Tauri", "OpenAI", "Whisper", "whisper.cpp", "whisper-rs", "LibriSpeech", "Apple Metal"], "alternates": {"html": "https://wpnews.pro/news/shipping-local-speech-to-text-inside-a-tauri-app", "markdown": "https://wpnews.pro/news/shipping-local-speech-to-text-inside-a-tauri-app.md", "text": "https://wpnews.pro/news/shipping-local-speech-to-text-inside-a-tauri-app.txt", "jsonld": "https://wpnews.pro/news/shipping-local-speech-to-text-inside-a-tauri-app.jsonld"}}