cd /news/artificial-intelligence/speech-to-markdown-with-local-models Β· home β€Ί topics β€Ί artificial-intelligence β€Ί article
[ARTICLE Β· art-54018] src=github.com β†— pub= topic=artificial-intelligence verified=true sentiment=↑ positive

Speech to Markdown with Local Models

Voice-to-Md.dev released Speech-to-Markdown, a 100% local app for macOS and iOS that converts voice into structured markdown documents using whisper.cpp and a local LLM, with no cloud dependency. The app offers global dictation and an agent mode with format, edit, and append capabilities.

read5 min views1 publishedJul 10, 2026
Speech to Markdown with Local Models
Image: source

Talk. Get clean markdown. 100% local. πŸ”— voice-to-md.dev

Speech-to-Markdown turns your voice into structured documents β€” and it ships as two apps:

πŸ–₯️ Desktop β€” macOS | πŸ“± Mobile β€” iOS | | |---|---|---| | Platform | macOS 13+ (menu-bar app) | iOS 26+, iPhone 15 Pro or newer (Apple Intelligence required; Apple-Silicon iPads too) | | Speech-to-text | |

whisper-cpp

, ffmpeg

, a local LLM servernoneβ€” zero setup** 100% offline**β€” nothing leaves your phoneJump to: πŸ–₯️ Desktop app Β· πŸ“± Mobile app

A menu-bar app: voice β†’ whisper.cpp β†’ your local LLM β†’ clean markdown. No cloud. No API keys. Nothing leaves your Mac. Everything in this part β€” including the dependencies and LLM-server setup below β€” applies to the Mac app only; the mobile app needs none of it.

↑ This entire product spec was dictated by voice β€” a local LLM structured it in real time.

⌨️ Global Dictation β€” Speak, and the transcript is typed straight into whatever field has focus. Terminal, browser, Slack β€” anything. A Spotlight-style pill shows what's happening:⌘βŒ₯]

anywhere. - πŸ“ Agent Mode β€” live markdown editor. Speak freely; a local LLM streams your words into a clean, structured markdown document in real time. Raw transcript stays one click away. Start it from the menu bar:

Everything in Agent Mode is driven from one floating, draggable capsule that hovers over the editor:

Left to right:

β€” don't wait for the auto-flush: transcribe whatever you just said and send it to the LLM✈️ Send (βŒ˜β†©

)right now. Enabled while recording and idle (no LLM call in flight).πŸŽ™οΈ Micβ€” start, , and resume the session. Red pulse = listening.** Status**β€”Recording

/Processing

/d

at a glance, with errors surfaced inline.Mode switcher andformat dropdownβ€” how your voice is applied (see below) and what comes out:** MD**(default),** TXT**, or** HTML**. Each format ships its own expectations + example to the LLM, works in every mode, and the session file extension follows (.md

/.txt

/.html

β€” switching mid-session renames the file). Your choice is remembered across launches.πŸ‘οΈ Previewβ€” opens the session document in the default app for its type (browser for HTML, your editor for markdown/plain text).πŸ—‘οΈ Trashβ€” clears the session (audio, transcript, and document) after a confirmation.

The modes:

Mode Icon What it does
Format (default)
πŸ“„ The whole document + your new words go to the LLM, which returns the complete restructured document. Best for free-form dictation where the model keeps improving the overall structure.
Edit
✏️ Your voice is an instruction, not content: "change the subtitle to Weekly Notes", "turn that list into a table". Select text in the editor first and the model treats it as the focus of the edit.
Append
βž• Speed mode for long documents: only the last 3 sentences + your new words are sent, and the model returns just the new content, which is appended. The LLM never re-reads the whole file, so latency stays flat as the document grows.

The flow underneath: audio is transcribed by whisper.cpp in ~4 s chunks and buffered; once ~30 words accumulate (or you for 5 s, or hit Send), the buffer is flushed to the LLM using the prompt for the current mode and output format β€” and tokens stream straight into the editor.

One line β€” installs dependencies, builds from source, and drops the app in /Applications:

curl -fsSL https://raw.githubusercontent.com/xajik/voice-to-md/main/install.sh | bash

Or from a checkout:

git clone https://github.com/xajik/voice-to-md.git && cd voice-to-md
make install       # deps + build + copy to /Applications

First launch: grant Microphone + Accessibility access, then download a Whisper model from Settings… in the menu bar (Base is a great start).

Signing: builds are automatically signed with your "Apple Development" identity when one is in the keychain, so re-installs keep their Microphone/Accessibility grants. No identity β†’ ad-hoc fallback (macOS will re-ask for permissions after updates). Override with make install SIGN_IDENTITY="…"

.

Agent Mode talks to any OpenAI-compatible server. Point Speech-to-Markdown at it in Settings… (default: http://127.0.0.1:8000/v1

, model auto-picked). The Whisper STT model is picked there too:

Pick your server:

Serve any MLX model on port 8000 β€” Speech-to-Markdown's default, zero config needed:

brew install omlx
omlx serve Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-4bit
brew install ollama
ollama pull qwen3.5        # or: ollama pull gemma3
ollama serve

Base URL: http://127.0.0.1:11434/v1

Download from lmstudio.ai, grab a model, start the local server. Base URL: http://127.0.0.1:1234/v1

brew install llama.cpp
llama-server -m your-model.gguf --port 8080

Base URL: http://127.0.0.1:8080/v1

Model Why
Qwen3.5 27B (4-bit)
Best formatting quality; the default pick
Gemma 26B (8-bit)
Fast, great instruction following

Anything that follows instructions well works β€” Speech-to-Markdown streams tokens as they arrive, so even bigger models feel instant.

Agent Mode on your phone β€” everything runs offline, on the device. No LLM server, no whisper install, no settings, no network. None of the desktop dependencies above apply here.

Requirements:iOS 26 or lateron anApple Intelligence device β€” iPhone 15 Pro or newer(or an Apple-Silicon iPad). Apple Intelligence must be enabled in Settings; the app shows an actionable banner if it isn't.

STT: Apple's SpeechAnalyzer streams your speech to text live, fully on-device (volatile text shows in gray as you talk; finalized words feed the document).LLM: Apple** Foundation Models**β€” the on-device Apple Intelligence model β€” formats the transcript. No server, no API keys, works in airplane mode.- Same three modes ( Format / Edit / Append), same** MD / TXT / HTML**output formats, sessions restorable from history, documents visible in the Files app.

Building from source: make build-ios

/ make test-ios

(see Development). Running on a device needs a development team: xcodebuild -scheme SpeechToMarkdownIOS -allowProvisioningUpdates CODE_SIGN_STYLE=Automatic DEVELOPMENT_TEAM=<TEAMID> …

(and Developer Mode enabled on the phone).

make check      # verify dependencies
make build      # release build + sign
make test       # unit tests
make generate   # regen .xcodeproj after editing project.yml
make install    # build + install to /Applications
make uninstall  # remove from /Applications

make build-ios  # build for the iOS simulator
make test-ios   # run the iOS test suite

Both apps share one core (Shared/

): transcript buffering, prompts, formats, session files. Desktop pipeline: AVAudioEngine β†’ whisper.cpp β†’ local LLM server. Mobile pipeline: AVAudioEngine β†’ SpeechAnalyzer β†’ Foundation Models. Either way, everything stays on your hardware.

── more in #artificial-intelligence 4 stories Β· sorted by recency
── more on @voice-to-md.dev 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/speech-to-markdown-w…] indexed:0 read:5min 2026-07-10 Β· β€”