{"slug": "a-on-device-stt-wisper-flow-alternative-that-uses-mistral-voxtral-model-and-afm", "title": "A on device STT Wisper Flow alternative that uses Mistral Voxtral model and AFM", "summary": "An open-source macOS dictation app called Whistral offers a fully local alternative to Wispr Flow, running Mistral's Voxtral Mini 4B Realtime speech model on-device via MLX and using Apple's on-device Foundation Model for smart correction. The app requires an Apple Silicon Mac (M1 or later) running macOS 27.2 or newer and about 3.9 GB of free disk space for the model, and it supports 13 languages. On one Apple M5 machine with 16 GB of unified memory running the default 6-bit model, the first partial text appeared about 0.7 seconds after speech started and dictation finalized about 0.6 seconds after the user stopped speaking, per the project's local benchmark.", "body_md": "An open-source, fully local alternative to Wispr Flow for macOS. Hold a\nshortcut, speak, release — your words are typed into whatever app you're\nusing. Powered by Mistral's [Voxtral Mini 4B Realtime](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602)\nspeech model running **on-device** via [MLX](https://github.com/ml-explore/mlx).\nNo audio, text, or telemetry ever leaves your Mac.\n\n- **Push-to-talk or toggle dictation** — a global shortcut (default ⌥Space)\nworks system-wide, in any app.\n- **Multilingual Smart Correction & Cross-Sentence Resolution** — uses Apple's\non-device Foundation Model to eliminate fillers (*uh* ,*um* ,*like* /*äh* ,*ähm* /*eh* ,*este* /*يعني* ), accurately resolve cross-sentence self-corrections\n(e.g.*\"meet at 10, sorry I meant 11am\"* →*\"meet at 11am\"* ,*\"Können wir uns morgen treffen? Nein warte, übermorgen\"* →*\"Können wir uns übermorgen treffen?\"* ),\nwhile preserving real intent (apologies, additions, and intentional phrasing).\n- **Speed vs. Quality presets** — choose between Speed (Fast) and Quality\n(Accurate) for both speech recognition lookahead and LLM post-processing.\n- **In-app update checking** — automatically detects new releases via GitHub\nReleases in the background, surfacing a one-click download button in the menu\nbar and Settings.\n- **Completely local & private** — speech recognition (Voxtral on MLX) and smart correction\n(Apple Foundation Model) run 100% on your Mac; audio is never uploaded.\n- **Apple HIG-native borderless HUD** — a lightweight, non-intrusive floating\noverlay near your cursor shows live audio levels and streaming text using native\nmacOS materials, typography, and standard system controls.\n- **13 languages** — Voxtral's multilingual ASR and the smart corrector seamlessly\nprocess English, German, Spanish, French, Italian, Portuguese, Arabic, Dutch,\nKorean, Japanese, Russian, Chinese, and Turkish.\n- **Smart insert** — transcribed text lands in the focused text field via a\npaste that briefly borrows, then restores, your clipboard; a synthetic\nkeystroke mode is available for apps that dislike synthetic pastes.\n- **Native & unobtrusive** — a menu-bar app that follows Apple's Human\nInterface Guidelines: non-activating panels, passive status feedback,\nproper permissions flow with purpose strings.\n\n- Apple Silicon Mac (M1 or later), macOS 27.2 or newer\n- ~3.9 GB free disk space for the model (downloaded once, on your first run)\n\nMeasured on one Apple M5 machine with 16 GB of unified memory, running the\ndefault 6-bit model; numbers come from a local benchmark, so treat them as a\nsingle data point rather than a promise. The first partial text appears about\n0.7 s after speech starts (the first session of a run takes up to about 0.9 s\nto warm up), and dictation finalizes about 0.6 s after you stop speaking. The\nmodel loads in about 1 s once cached — a first-ever launch downloads the\n~3.9 GB model first. To rerun the benchmark yourself, see\n[Tests/perf-report.md](https://github.com/ThriveEngineer/Whistral/blob/main/Tests/perf-report.md) and `python3 Tests/bench_latency.py`.\n\n1. \nClone this repository.\n2. \nOpen `Whistral.xcodeproj` in Xcode and press**Run** , or build from the\ncommand line:\n\n```\nxcodebuild -project Whistral.xcodeproj -scheme Whistral \\\n  -configuration Debug -destination \"platform=macOS,arch=arm64\" build\n```\n\n3. \nOn first launch, Whistral walks you through onboarding: microphone permission, Accessibility permission (for the global shortcut and text insertion), engine setup, and the model download.\n\n- The Swift app captures microphone audio with `AVAudioEngine` , converts it\nto 16 kHz mono, and streams it to a small**Python sidecar** (`Whistral/Resources/Engine/whistral_engine.py` ) over a length-prefixed\nstdin/stdout protocol.\n- The sidecar wraps [voxmlx](https://github.com/awni/voxmlx) , an MLX\nimplementation of the streaming Voxtral pipeline, and emits NDJSON events\n(`partial` ,`segment` ,`final` ) back to Swift.\n- If **Smart Correction** is enabled, the final Voxtral transcript is piped to\nApple's on-device Foundation Model (`FoundationModels` framework). The model\nremoves spoken disfluencies, filler words, and resolves self-corrections\n(e.g.,*\"10am, sorry I meant 11am\"* →*\"11am\"* ). The model session is prewarmed\nin the background while you speak for minimal turn-around latency.\n- The HUD is a non-activating `NSPanel` , so your keystrokes and focus are\nnever stolen. Text insertion is a synthetic ⌘V that saves and restores\nyour clipboard.\n- The full sidecar protocol (framing, commands, events) is documented in\n[docs/engine-protocol.md](https://github.com/ThriveEngineer/Whistral/blob/main/docs/engine-protocol.md) .\n\n- Audio is recorded **only** while the shortcut is held (or dictation is\ntoggled on) and stays in memory.\n- The model weights and the embedded Python engine are stored locally under\n`~/Library/Application Support/Whistral/` .\n- No network calls except the one-time model download from\n[Hugging Face](https://huggingface.co/mlx-community/Voxtral-Mini-4B-Realtime-6bit) and the background version check to GitHub Releases.\n\n| Setting | Description | \n|---|---|\n| Dictation shortcut | Re-recordable global hotkey (modifier + key required) | \n| Push-to-talk / toggle | Hold the key to talk, or press to start/stop | \n| Insert at cursor | Auto-insert the transcription into the focused app (on by default) | \n| Insertion method | Paste (fast, restores your clipboard) or synthetic keystrokes | \n| Smart Correction | Enable/disable on-device Apple Foundation Model post-processing | \n| Correction Mode | Speed (Fast, prewarmed session) or Quality (deeper cross-sentence context) | \n| Correction Style | Clean (fillers & corrections only), Natural (default: fixes grammar), Polished | \n| Recognition Preset | Speed (240 ms), Medium (480 ms default), Quality (720 ms), or Custom | \n| Transcription delay | Fine-tune streaming lookahead delay (80 ms–2400 ms) | \n| Updates | Automatic background update checking and manual Check for Updates | \n| Sound feedback | Subtle system sounds when dictation starts and finishes | \n| Cancel with Esc | Hardwired: pressing Esc cancels the current dictation | \n| Model | Voxtral Mini 4B Realtime — 6-bit (default) or BF16 (original) | \n| Launch at login | Register with `SMAppService` | \n\n- `Tests/test_engine_e2e.py` drives the sidecar protocol end-to-end:\nload → start → feed audio → stop, printing every engine event.\n- The UI is built to follow Apple's Human Interface Guidelines — menu bar\nbehavior, privacy and permission flow, onboarding, and accessibility —\nper [HIG](https://developer.apple.com/design/human-interface-guidelines/) .\n- The app icon (also used as the logo mark above) is authored as an\nIcon Composer `.icon` document with light, dark, and tinted variants — in\nlight and dark appearance it is a flat seven-bar waveform (amber → red) on\na dark background, and the tinted variant renders the bars as monochrome\nglyphs; the rounded**system squircle** silhouette, corner rounding, and\ndrop shadow you see are applied by macOS itself — the source layers are\nfull-bleed squares.\n\nApache License 2.0 — see [LICENSE](https://github.com/ThriveEngineer/Whistral/blob/main/LICENSE). The model weights carry their\nown license from [mistralai](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602)\n(Apache 2.0); the quantized MLX community port follows the same license.", "url": "https://wpnews.pro/news/a-on-device-stt-wisper-flow-alternative-that-uses-mistral-voxtral-model-and-afm", "canonical_source": "https://github.com/ThriveEngineer/Whistral", "published_at": "2026-11-20 19:34:03+00:00", "updated_at": "2026-11-20 19:45:24.785573+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "large-language-models", "natural-language-processing", "ai-products"], "entities": ["Whistral", "Wispr Flow", "Mistral", "Voxtral Mini 4B Realtime", "MLX", "Apple", "Apple Foundation Model", "voxmlx"], "also_reported_by": [], "takeaway": "Whistral streams 16 kHz mono audio from a Swift AVAudioEngine app to a Python sidecar wrapping voxmlx, which emits NDJSON partial, segment and final events, and on an M5 with 16 GB of memory the first partial text appeared about 0.7 s after speech started with finalization about 0.6 s after speech s", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/a-on-device-stt-wisper-flow-alternative-that-uses-mistral-voxtral-model-and-afm", "markdown": "https://wpnews.pro/news/a-on-device-stt-wisper-flow-alternative-that-uses-mistral-voxtral-model-and-afm.md", "text": "https://wpnews.pro/news/a-on-device-stt-wisper-flow-alternative-that-uses-mistral-voxtral-model-and-afm.txt", "jsonld": "https://wpnews.pro/news/a-on-device-stt-wisper-flow-alternative-that-uses-mistral-voxtral-model-and-afm.jsonld"}}