{"slug": "a-voice-model-that-never-stops-talking-isn-t-duplex", "title": "A voice model that never stops talking isn't duplex", "summary": "A developer built a full-duplex speech-to-speech voice assistant on a Mac Mini M4 (16 GB RAM) using LiquidAI's LFM3-Omni-2B in GGUF served through llama-server 0.9.0's realtime WebSocket API, and found that llama.cpp performs no server-side voice activity detection and emits no speech_started event, leaving interruption handling entirely to the client. The custom client, voice_loop.py, runs Silero VAD over a PortAudio stream and sends response.cancel on barge-in, dropping 2.80 s of queued speech and returning a cancelled status with 240 ms time to first audio and 1.9x real-time factor. The developer also reported that the Mac Mini M4 has no physical microphone, so the loop ran against fed WAV audio, and that three llama.cpp bugs remain unfixed on the build.", "body_md": "On 2026-11-06 I stood up a full-duplex speech-to-speech assistant on a Mac Mini M4 with 16 GB of RAM, four tests deep on stock llama-server builds, and my own client interrupted the model mid-answer at 240 ms time to first audio. The claim this backs: nothing upstream listens for you talking over an assistant. llama.cpp's realtime WebSocket API has no server-side VAD and emits no `speech_started` event, so interruption handling is entirely your client's job, and a demo that skips that job is a press-to-talk loop with a better name attached to it. Latency, quant math, even whether the machine owns a microphone, all get checked after that one design decision.\n\nModel is LiquidAI/LFM3-Omni-2B in GGUF, text plus vision plus audio in, audio out through LiquidCodec at 24 kHz, input audio read at 16 kHz. Server side was llama-server 0.9.0 from Homebrew (I bumped the engine from 0.5.0 that same day) behind `-t 8 -ngl 99 --reasoning off -c 8192`, endpoint `ws://127.0.0.1:9931/v1/realtime`. Upstream support isn't hypothetical here, PR #30636 landed the WebSocket base, PR #31269 adds realtime conversation plus transcription and states explicitly it works with LFM3-Omni-2B-GGUF, whose official minimal client ships as examples/realtime_client.py on the openai SDK.\n\nTest B first, the stock client against my server: first output 126 ms after end of input, a 5.44 s answer streamed clean. Then offline speech, test A, where `llama-tts` on Q8_0 spent prompt eval 0.27 s plus generation 1.54 s to produce 2.88 s of audio, RTF 1.59x, second sample 1.61x. The model manufactures speech faster than playback before any duplex plumbing exists at all, which is why the interesting part of this lane was never throughput.\n\n`voice_loop.py`, mine, in ~/Developer/lfm3-voice. Client-side VAD runs silero_vad.onnx (from the Liquid4All cookbook) over a PortAudio RawInputStream, and when speech is detected mid-answer the client sends `response.cancel`. Measured turn: I interrupted at stream position 5.06 s while the assistant was still talking, cancel fired, 2.80 s of queued speech got dropped, status returned cancelled with TTFA 240 ms and RTF 1.9x. Follow-up turns completed at TTFA 240-244 ms, RTF 2.17x and 2.21x, and the speakers rendered 15.82 s of audio through RawOutputStream with two-channel interleave to device 2, the Mac mini Speakers.\n\nReading `server-realtime.cpp` shows the server agreeing with that path: cancel() keeps the session alive and the partial stays in history, so an interrupted turn costs nothing structurally. The captured log for every barge-in ended as `stop: cancel task` then a clean `slot release`. One recurring warning, `embeddings required but some input tokens were not marked as outputs -> overriding`, prints on every LFM3-Omni run and is benign audio/mm noise.\n\nOne PortAudio quirk bit the capture path directly: blocksize-0 raw streams come up stopped on both input and output until you call `.start()` explicitly, symptom being a silent capture and zero segments from a VAD that never got fed.\n\nsounddevice on this box returns BlackHole virtual inputs (16ch and 2ch) plus Mac mini Speakers as output, and nothing else. Recording ambient returned RMS 0.0, recording while `afplay` blasted a file through the speakers returned RMS 0.0 as well. A Mac Mini M4 has no physical mic, which is why my loop ran against fed wav audio until USB hardware showed up. My capture pipeline passed with a fed wav through the same path (`--vad-debug` probed 6 s clean, zero segments from the silent feed) so every stage works except the one needing air. A voice assistant on a desktop-class mini stays a test harness until someone plugs in USB hardware, and one call to `sd.query_devices` tells you that before you design the loop. The client already carries `--mic-device N` for the day the hardware arrives.\n\nRead against llama.cpp issues on master v0.9.0 dated 2026-11-04, three bugs stand unfixed on my build. #31689: `/v1/realtime` ignores requested output language and voice, non-English comes out speaking with an English accent, while `llama-tts --tts-lang` honors it. #31696: `session.update` asking for 24 kHz input is accepted silently and the audio is still read at 16 kHz, so the fix is sending the mic at 16 kHz. #31436: forcing audio-only modality degrades generated speech to gibberish, interleaved text plus audio instructions work fine.\n\nThe shape they share is the API accepting configuration it doesn't implement and returning success, with the damage visible only in the audio. All three pass a code review and only show up against real audio.\n\nFree disk decided the quant choice. BF16 was off the table on space, so Q8_0 became the primary weight set. The pair is 1.846 GB model plus 507 MB mmproj, 2.35 GB total, and free space on that volume went from 8.5 GiB down to 6.4 GiB during the download. BF16 at 4.4 GB would have left under 2 GiB on that volume. Server RSS sits around 2.4 GB resident for the Q8_0 pair.\n\nSame build then serves chat through a `mac-mini` provider with declared context length 65536 (engine `-c 65536`, KV roughly 48 KB per token derived from 24 layers times 4 kv heads times 128 head_dim in f16), trained window is 128K so no rope scaling was needed. Proof: `hermes chat -Q --oneshot --provider mac-mini -m lfm3-omni-2b` returned \"pong\" with rc=0 in 68 s wall, while a raw warm round-trip against `/v1/chat/completions` did 10 tokens in 0.65 s near 15 tok/s. A 2B on a mini answers prompts fast and agent turns slow, so that lane belongs behind voice output plus small tool schemas. Persistence came from launchd unit `com.alfred.lfm3-omni-server.plist` with KeepAlive=true across logout, crash and reboot, macOS 26 rejecting the newer LaunchServices dialect for third-party services and taking the classic Label/ProgramArguments schema instead.\n\nLatency numbers here are one machine, one quant, four test runs, sample size of a handful of turns. Directionally the duplex loop stays under half a second to first audio all night, but nobody should read TTFA 240 ms as a cross-hardware benchmark.\n\nThe transferable part is architectural. Server-side speech-to-speech APIs in llama.cpp ship cancel semantics without listening, so the client owns turn-taking, and any claim about full-duplex behavior needs a capture path that exists on the box running it. Two checks before you adopt someone's repo: grep their client for the cancel call, then query devices for a microphone. Both take minutes and both decide whether the thing can work at all, long before a benchmark table matters.\n\nWhat's your interrupt path? If you can't name the exact call that stops the response mid-answer, tell me what happens when you talk over your assistant.", "url": "https://wpnews.pro/news/a-voice-model-that-never-stops-talking-isn-t-duplex", "canonical_source": "https://dev.to/prodbymarcu/a-voice-model-that-never-stops-talking-isnt-duplex-do6", "published_at": "2026-11-09 23:08:49+00:00", "updated_at": "2026-11-09 23:23:39.823699+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "developer-tools"], "entities": ["llama.cpp", "LiquidAI", "LFM3-Omni-2B", "llama-server", "Silero VAD", "PortAudio", "Mac Mini M4", "Homebrew"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-voice-model-that-never-stops-talking-isn-t-duplex", "markdown": "https://wpnews.pro/news/a-voice-model-that-never-stops-talking-isn-t-duplex.md", "text": "https://wpnews.pro/news/a-voice-model-that-never-stops-talking-isn-t-duplex.txt", "jsonld": "https://wpnews.pro/news/a-voice-model-that-never-stops-talking-isn-t-duplex.jsonld"}}