I work at a German company. Meetings are in German, sometimes English — and even though I speak both, there are moments in fast meetings where I zone out for two seconds and think: wait, what did they just say?
I couldn't use any of the existing captioning tools, because they all pipe your meeting audio to a cloud server. Sending confidential work calls to a third party was a non-starter.
Then I realized something: macOS 26 quietly shipped every building block you need for a real-time translator. On-device speech recognition (SpeechAnalyzer), on-device machine translation (Translation framework), and an on-device LLM (FoundationModels / Apple Intelligence). No servers, no API keys, no per-minute fees.
So I built Wakaru — a menu bar app that turns
This post covers how it works and the four gotchas that cost me the most time. If you're planning to build anything on macOS 26's new speech or translation APIs, this might save you a few days.
My first prototype was Electron. It worked — but transcription + translation took about 2 seconds per sentence. For subtitles, 2 seconds might not sound like a lot, but it is: by the time the caption showed up, the conversation had moved on.
When I saw that macOS 26 had the entire pipeline available natively and on-device, I rebuilt it in Swift. The difference was dramatic — captions now appear while the sentence is still being spoken.
(The Electron version wasn't wasted, though. After a lot of tuning it got reasonably fast, and I'm planning to release it for Windows soon.)
ScreenCaptureKit ──▶ SpeechAnalyzer ──▶ Translation framework ──▶ Subtitle overlay
(system audio) (speech-to-text) or FoundationModels (NSPanel)
(translation)
The key decision: capture system audio (what the Mac is playing) instead of the microphone. That's what makes Wakaru app-agnostic — it doesn't integrate with Zoom or Teams; it doesn't need to know they exist.
The subtitles are drawn on a borderless, transparent, click-through NSPanel
, so you can click straight through the captions to whatever is underneath.
The naive version of this app is a window where original text and translations pile up like a chat log. That's easy to build — and unusable as subtitles.
What I ended up with:
text.count / 7 + 2
seconds (clamped to 4–12s). Short interjections vanish quickly; long sentences stay until you can actually finish them.
let lifetime = min(max(4, Double(text.count) / 7 + 2), 12)
SpeechAnalyzer prefers 16 kHz audio. My first version captured at 48 kHz and resampled with AVAudioConverter
— and the first caption took seconds to appear. The converter buffers audio internally before it emits anything.
The fix: ScreenCaptureKit lets you pick the sample rate at capture time.
let cfg = SCStreamConfiguration()
cfg.capturesAudio = true
cfg.sampleRate = 16_000 // match SpeechAnalyzer's preferred format
cfg.channelCount = 1
Capture at 16 kHz mono from the start and the only conversion left is Float32 → Int16, sample by sample. The latency disappeared.
Even after fixing the sample rate, captions were still sluggish. It turned out to be three separate problems:
.fastResults
in reportingOptions
to get them as they happen. For live captions this is non-negotiable.prepareToAnalyze
when the user hits start, and recognition is instant from the first word.finalizeAndFinishThroughEndOfInput()
drains the entire audio backlog before returning — a stop/restart (e.g. switching languages) took seconds. For subtitles you don't care about queued audio, so use cancelAndFinishNow()
.This was the biggest trap of all.
SpeechAnalyzer's partial results don't just grow at the end. Text you already displayed gets rewritten retroactively — filler words ("uh, uh") get collapsed, words get swapped, punctuation appears late.
Wakaru cuts completed sentences out of the growing transcript and translates each one. That means it has to remember where the already-translated part ends. If you store that boundary as a character offset, it silently drifts every time the recognizer rewrites history. The symptoms: the same sentence gets translated twice, or fragments go missing.
The fix: stop trusting positions, and use content as a bookmark. Wakaru remembers the last few words of the most recently committed sentence, and on every update, searches for that anchor in the rewritten text to re-derive the boundary.
One wrinkle remains: if the speaker literally says the same thing twice ("Thank you. Thank you."), the anchor appears in two places. So the rule is "pick the occurrence closest to the previous boundary estimate" — the character offset survives, demoted from source-of-truth to tie-breaker.
This one surprised me the most. TranslationSession
can only be obtained inside SwiftUI's .translationTask
view modifier. There is no "just give me a session" API. Wakaru is a menu bar app — there was nowhere natural to put it.
The workaround: the subtitle overlay is a SwiftUI view anyway, so I attached an invisible .translationTask
to it. It receives the session and hands it to a hub object that the rest of the app calls into.
.translationTask(hub.configuration) { session in
await hub.serve(session) // publish the session to the pipeline
}
Bonus trap: if you stop and restart with the same language pair, the new configuration compares equal to the old one and the task never restarts — you have to call invalidate()
explicitly.
On Apple Intelligence Macs, Wakaru has a high-accuracy mode that translates with the on-device LLM instead of the NMT model, passing the previous 3 sentences as context — pronouns, idioms, and short replies come out much more natural.
But a ~3B on-device model in a real-time loop needs guardrails:
The rule that matters: the LLM is never allowed to stall the captions. Every failure mode falls back per-sentence to the standard engine.
Wakaru runs on macOS 26+ (Apple silicon). It's free for 14 days (and stays free for 1 hour/day after that), with a one-time purchase for unlimited time — no subscription:
Happy to answer any questions about the implementation in the comments! 👇