# Speech to Markdown with Local Models

> Source: <https://github.com/xajik/voice-to-md>
> Published: 2026-07-10 10:25:36+00:00

**Talk. Get clean markdown. 100% local. 🔗 voice-to-md.dev**

Speech-to-Markdown turns your voice into structured documents — and it ships as **two apps**:

🖥️ Desktop — macOS |
📱 Mobile — iOS |
|
|---|---|---|
| Platform | macOS 13+ (menu-bar app) | iOS 26+, iPhone 15 Pro or newer (Apple Intelligence required; Apple-Silicon iPads too) |
| Speech-to-text |
|

`whisper-cpp`

, `ffmpeg`

, a local LLM server**none**— zero setup** 100% offline**— nothing leaves your phoneJump to: [🖥️ Desktop app](#%EF%B8%8F-desktop-app-macos) · [📱 Mobile app](#-mobile-app-ios)

A menu-bar app: voice → whisper.cpp → your local LLM → clean markdown. No cloud. No API keys. Nothing leaves your Mac. Everything in this part — including the dependencies and LLM-server setup below — applies to the **Mac app only**; the mobile app needs none of it.

*↑ This entire product spec was dictated by voice — a local LLM structured it in real time.*

-
**⌨️ Global Dictation —** Speak, and the transcript is typed straight into whatever field has focus. Terminal, browser, Slack — anything. A Spotlight-style pill shows what's happening:`⌘⌥]`

anywhere. -
**📝 Agent Mode — live markdown editor.** Speak freely; a local LLM streams your words into a clean, structured markdown document in real time. Raw transcript stays one click away. Start it from the menu bar:

Everything in Agent Mode is driven from one floating, draggable capsule that hovers over the editor:

Left to right:

— don't wait for the auto-flush: transcribe whatever you just said and send it to the LLM✈️ Send (`⌘↩`

)*right now*. Enabled while recording and idle (no LLM call in flight).**🎙️ Mic**— start, pause, and resume the session. Red pulse = listening.** Status**—`Recording`

/`Processing`

/`Paused`

at a glance, with errors surfaced inline.**Mode switcher** and**format dropdown**— how your voice is applied (see below) and what comes out:** MD**(default),** TXT**, or** HTML**. Each format ships its own expectations + example to the LLM, works in every mode, and the session file extension follows (`.md`

/`.txt`

/`.html`

— switching mid-session renames the file). Your choice is remembered across launches.**👁️ Preview**— opens the session document in the default app for its type (browser for HTML, your editor for markdown/plain text).**🗑️ Trash**— clears the session (audio, transcript, and document) after a confirmation.

The modes:

| Mode | Icon | What it does |
|---|---|---|
Format (default) |
📄 | The whole document + your new words go to the LLM, which returns the complete restructured document. Best for free-form dictation where the model keeps improving the overall structure. |
Edit |
✏️ | Your voice is an instruction, not content: "change the subtitle to Weekly Notes", "turn that list into a table". Select text in the editor first and the model treats it as the focus of the edit. |
Append |
➕ | Speed mode for long documents: only the last 3 sentences + your new words are sent, and the model returns just the new content, which is appended. The LLM never re-reads the whole file, so latency stays flat as the document grows. |

The flow underneath: audio is transcribed by whisper.cpp in ~4 s chunks and buffered; once ~30 words accumulate (or you pause for 5 s, or hit **Send**), the buffer is flushed to the LLM using the prompt for the current mode and output format — and tokens stream straight into the editor.

One line — installs dependencies, builds from source, and drops the app in /Applications:

```
curl -fsSL https://raw.githubusercontent.com/xajik/voice-to-md/main/install.sh | bash
```

Or from a checkout:

```
git clone https://github.com/xajik/voice-to-md.git && cd voice-to-md
make install       # deps + build + copy to /Applications
# …or for development: make setup && make run
```

First launch: grant **Microphone** + **Accessibility** access, then download a Whisper model from **Settings…** in the menu bar (Base is a great start).

**Signing:** builds are automatically signed with your "Apple Development" identity when one is in the keychain, so re-installs keep their Microphone/Accessibility grants. No identity → ad-hoc fallback (macOS will re-ask for permissions after updates). Override with `make install SIGN_IDENTITY="…"`

.

Agent Mode talks to any **OpenAI-compatible** server. Point Speech-to-Markdown at it in **Settings…** (default: `http://127.0.0.1:8000/v1`

, model auto-picked). The Whisper STT model is picked there too:

Pick your server:

Serve any MLX model on port 8000 — Speech-to-Markdown's default, zero config needed:

```
brew install omlx
omlx serve Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-4bit
brew install ollama
ollama pull qwen3.5        # or: ollama pull gemma3
ollama serve
```

Base URL: `http://127.0.0.1:11434/v1`

Download from [lmstudio.ai](https://lmstudio.ai), grab a model, start the local server.
Base URL: `http://127.0.0.1:1234/v1`

```
brew install llama.cpp
llama-server -m your-model.gguf --port 8080
```

Base URL: `http://127.0.0.1:8080/v1`

| Model | Why |
|---|---|
Qwen3.5 27B (4-bit) |
Best formatting quality; the default pick |
Gemma 26B (8-bit) |
Fast, great instruction following |

Anything that follows instructions well works — Speech-to-Markdown streams tokens as they arrive, so even bigger models *feel* instant.

Agent Mode on your phone — **everything runs offline, on the device**. No LLM server, no whisper install, no settings, no network. None of the desktop dependencies above apply here.

Requirements:iOS 26 or lateron anApple Intelligence device — iPhone 15 Pro or newer(or an Apple-Silicon iPad). Apple Intelligence must be enabled in Settings; the app shows an actionable banner if it isn't.

**STT**: Apple's SpeechAnalyzer streams your speech to text live, fully on-device (volatile text shows in gray as you talk; finalized words feed the document).**LLM**: Apple** Foundation Models**— the on-device Apple Intelligence model — formats the transcript. No server, no API keys, works in airplane mode.- Same three modes (
**Format / Edit / Append**), same** MD / TXT / HTML**output formats, sessions restorable from history, documents visible in the Files app.

Building from source: `make build-ios`

/ `make test-ios`

(see [Development](#%EF%B8%8F-development)). Running on a device needs a development team: `xcodebuild -scheme SpeechToMarkdownIOS -allowProvisioningUpdates CODE_SIGN_STYLE=Automatic DEVELOPMENT_TEAM=<TEAMID> …`

(and Developer Mode enabled on the phone).

```
# macOS app
make check      # verify dependencies
make build      # release build + sign
make test       # unit tests
make generate   # regen .xcodeproj after editing project.yml
make install    # build + install to /Applications
make uninstall  # remove from /Applications

# iOS app
make build-ios  # build for the iOS simulator
make test-ios   # run the iOS test suite
```

Both apps share one core (`Shared/`

): transcript buffering, prompts, formats, session files. Desktop pipeline: AVAudioEngine → whisper.cpp → local LLM server. Mobile pipeline: AVAudioEngine → SpeechAnalyzer → Foundation Models. Either way, everything stays on your hardware.