{"slug": "frontier-level-dictation-on-your-iphone-keyboard-self-hosted", "title": "Frontier-level dictation on your iPhone keyboard, self-hosted", "summary": "A developer set up self-hosted, frontier-level dictation on an iPhone keyboard by running Alibaba's Qwen3-ASR 1.7B model through the open-source Diction gateway and Agent CLI on a home RTX 3090, after the free tier of closed-source Wispr Flow hit its weekly word limit. According to the Hugging Face Open ASR Leaderboard as of September 19, 2026, Qwen3-ASR 1.7B posts a 4.3% word error rate, behind Zoom Scribe v2 Pro at 3.6% and ElevenLabs Scribe v2 at 4.0% but ahead of NVIDIA Parakeet TDT 0.6B v3 at 4.9% and OpenAI Whisper large-v3 at 5.8%. The setup routes audio over a private network to an OpenAI-compatible transcription endpoint, with no LLM cleanup step, so Diction's Writing Style and Tones features do not work.", "body_md": "On a recent month-long trip visiting family in Europe, I did a lot of work from my phone, using [my mobile coding workflow](/post/agentic-mobile-workflow/).\nIt still boggles my mind that real, productive work from a phone is possible now.\nMost of that work is talking to coding agents, and the built-in iOS dictation is garbage for that.\nI had an iOS Shortcut that sent recordings to [Agent CLI](https://github.com/basnijholt/agent-cli), [my local AI toolbox](/post/auto-install-extras/), running at home instead.\nThe problem was that the recording screen takes over the whole display, so I could not see the thing I was commenting on.\nThe transcript then landed in my clipboard, and I had to paste it myself.\n\nSo I threw my principles overboard and installed [Wispr Flow](https://wisprflow.ai), a voice keyboard for iOS.\nI loved it instantly.\niOS does not let keyboards use the microphone, so the keyboard’s button hands off to the Wispr Flow app, which keeps recording in the background.\nI was less happy about that, since it meant a closed-source app always had the microphone open and sent my voice to their servers.\nI was on the free plan, so I was probably the product.\nI also hit the free weekly word limit very quickly, and started looking for an alternative.\n\nThat is how I found [Diction](https://apps.apple.com/app/id6759807364), which has since become my favorite app when I’m away from my laptop.\nIt works the same way, background recording included.\nThe difference is that it has a self-hosted mode with an open-source gateway, so my voice goes to a machine I own instead of their servers.\n\nI run the [gateway](https://github.com/DictionLabs/Diction) on my home machine in the U.S., reachable only over my private network.\nThe gateway streams audio to Agent CLI, which runs Alibaba’s [`Qwen/Qwen3-ASR-1.7B-hf`](https://huggingface.co/Qwen/Qwen3-ASR-1.7B-hf) on an RTX 3090 behind an OpenAI-compatible transcription endpoint.\nThere is no LLM cleanup step, so Diction’s Writing Style and Tones features do not work with this setup.\nI don’t miss them: Qwen’s raw transcription is already good enough for me.\n\nFor the longest time I used [Faster-Whisper](https://github.com/SYSTRAN/faster-whisper) with Whisper large-v3.\nWhisper is still the default choice for most people, but many newer models beat it.\nI tried some of them, including NVIDIA’s [Parakeet](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3).\nParakeet is fast, but it cannot take custom instructions.\nThose instructions are how Diction’s **My Words** feature biases a transcription toward specialized names.\n\nAt the start of September, I looked at the [Hugging Face Open ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard), which tests speech-to-text models on the same recordings and ranks them.\nThe score is the word error rate (WER): the share of words a model gets wrong, counting words it swaps, drops, or makes up.\nLower is better, and a WER of 4% means about one wrong word in every 25.\nThe test recordings range from audiobooks and podcasts to meetings and earnings calls, so the average says more than any single clean benchmark.\n\nThese are the averages on the public English test sets as of September 19, 2026:\n\n| Model | WER | Can I run it at home? | \n|---|---|---|\n| Zoom Scribe v2 Pro | 3.6% | No, paid API (#1 overall) | \n| ElevenLabs Scribe v2 | 4.0% | No, paid API | \n| **Qwen3-ASR 1.7B** | **4.3%** | **Yes, Apache-2.0** | \n| AssemblyAI Universal-3.5 Pro | 4.3% | No, paid API | \n| NVIDIA Parakeet TDT 0.6B v3 | 4.9% | Yes | \n| OpenAI Whisper large-v3 | 5.8% | Yes | \n\nI picked the best model on the board whose weights you can download: Qwen3-ASR 1.7B.\nIt lands within a point of the best paid API, which is what I mean by frontier-level.\nSince I maintain Agent CLI, I told an agent to add a Qwen backend, and shortly afterward [it had landed](https://github.com/basnijholt/agent-cli/pull/636) and was running on my server.\n\nAgent CLI’s transcription server loads a model on the first request and unloads it after an idle timeout, so the model only takes VRAM while I use it.\nThe same process serves an OpenAI-compatible API, which the Diction gateway talks to, and the [Wyoming protocol](https://www.home-assistant.io/integrations/wyoming/) that Home Assistant uses for local voice.\n\nYou need an NVIDIA GPU with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html), Docker Compose 2.30 or newer, and a private route from your iPhone to the server.\nI use Headscale, but Tailscale or plain WireGuard works just as well.\nDo not expose the gateway to the public internet.\n\nSave the following as `Dockerfile.agent-cli` in an empty directory.\nIt starts with the CUDA image from Agent CLI and adds the Transformers dependencies needed for Qwen:\n\n```\nFROM ghcr.io/basnijholt/agent-cli-whisper:latest-cuda\n\nUSER root\n\nARG AGENT_CLI_VERSION=0.103.0\n\nRUN uv pip install \\\n --python /app/.venv/bin/python \\\n --upgrade \\\n \"agent-cli[whisper-transformers]==${AGENT_CLI_VERSION}\"\n\nRUN apt-get update \\\n && apt-get install --yes --no-install-recommends gcc libc6-dev \\\n && rm -rf /var/lib/apt/lists/*\n\nUSER whisper\n```\n\nSave this as `compose.yaml` next to the Dockerfile.\nIt points Diction’s gateway at Agent CLI’s transcription port.\n`WHISPER_TTL` raises the idle timeout from the default five minutes to a day, so dictation rarely waits for a reload:\n\n```\nservices:\n asr:\n image: agent-cli-whisper-qwen:0.103.0\n build:\n context: .\n dockerfile: Dockerfile.agent-cli\n args:\n AGENT_CLI_VERSION: \"0.103.0\"\n environment:\n WHISPER_MODEL: Qwen/Qwen3-ASR-1.7B-hf\n WHISPER_EXTRA_ARGS: --backend transformers\n WHISPER_TTL: \"86400\"\n WHISPER_LOG_LEVEL: info\n WHISPER_DEVICE: cuda\n volumes:\n - model-cache:/home/whisper/.cache\n gpus: all\n restart: unless-stopped\n\n gateway:\n image: dictionlabs/gateway:v13.0\n depends_on:\n - asr\n environment:\n CUSTOM_BACKEND_URL: http://asr:10301\n CUSTOM_BACKEND_MODEL: Qwen/Qwen3-ASR-1.7B-hf\n CUSTOM_BACKEND_CANONICAL_ID: Qwen/Qwen3-ASR-1.7B-hf\n CUSTOM_BACKEND_NEEDS_WAV: \"true\"\n ports:\n - \"8080:8080\"\n restart: unless-stopped\n\nvolumes:\n model-cache:\n```\n\nBuild and start both services:\n\n```\ndocker compose up -d --build\n```\n\nThe first transcription downloads Qwen into the `model-cache` volume, so that request can take a few minutes.\n`docker compose logs -f` shows the progress.\n\nOn the iPhone, go to **Settings → General → Keyboard → Keyboards → Add New Keyboard → Diction**.\nTap Diction in the keyboard list, enable **Allow Full Access**, and grant microphone access when prompted.\nThen open Diction, go to **Preferences → Mode → Self-Hosted**, enter `http://<private-server-ip>:8080`, and tap **Test connection**.\n\n`PipeFunc` and `MindRoom` come back with the correct spelling and capitalization.\nIt shipped in gateway v13.0, which is why the Compose file needs at least that version.\nOn my machine, Qwen3-ASR 1.7B uses about 4.7 GiB of GPU memory and roughly 3 GiB of system memory while loaded.\n\nA day later, I met a friend in a bar in the Netherlands and wanted to let him try it.\n\nHe was already on my tailnet for some of my other self-hosted services, so his phone only needed an ACL change to reach the gateway. I opened the terminal app on my phone, connected to an agent, and dictated that change. In less than a minute, Diction was connected and working on his phone. His speech crossed the Atlantic, was transcribed by Qwen on my home GPU, and appeared back in the app with barely any delay.\n\nThat moment sold the whole setup to me. I was in a bar without a laptop, and I used dictation to grant access to the dictation service.", "url": "https://wpnews.pro/news/frontier-level-dictation-on-your-iphone-keyboard-self-hosted", "canonical_source": "/post/diction-agent-cli-qwen/", "published_at": "2026-09-22 00:00:00+00:00", "updated_at": "2026-09-23 03:23:47.049051+00:00", "lang": "en", "topics": ["ai-tools", "ai-products", "natural-language-processing", "ai-infrastructure", "developer-tools"], "entities": ["Diction", "Agent CLI", "Alibaba", "Qwen3-ASR 1.7B", "Wispr Flow", "NVIDIA Parakeet TDT 0.6B v3", "OpenAI Whisper large-v3", "Hugging Face Open ASR Leaderboard"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/frontier-level-dictation-on-your-iphone-keyboard-self-hosted", "markdown": "https://wpnews.pro/news/frontier-level-dictation-on-your-iphone-keyboard-self-hosted.md", "text": "https://wpnews.pro/news/frontier-level-dictation-on-your-iphone-keyboard-self-hosted.txt", "jsonld": "https://wpnews.pro/news/frontier-level-dictation-on-your-iphone-keyboard-self-hosted.jsonld"}}