On 6 November 2026 the Coval voice-AI benchmarks listed Qwen3 ASR 1.7B (Alibaba, Apache-2.0, open weights) at a mean 35 ms time-to-final-segment. That put it first of 28 measured STT systems, with a 2.9 % word error rate. The same board showed Soniox STT RT v5 at roughly 46–47 ms. For anyone building a live call or interview overlay the model-inference number has now dropped into the noise floor of a well-tuned audio path.
Qwen3-ASR-1.7B is a roughly 1.7 B active-parameter multilingual model released under Apache-2.0. It does language identification plus transcription across dozens of languages and Chinese dialects, and the same checkpoint supports both streaming and offline inference. The Coval numbers come from a dedicated Baseten deployment; local runs will be slower, but the headline is the same: a competent open-weight model can now finish a final segment in a few tens of milliseconds once the audio is in memory.
That is useful. It is also not the whole story on Windows.
A live translation or interview-assist loop looks like this:
Step 4 used to dominate. At 35–50 ms for a final segment it no longer does. The remaining latency is almost entirely in the Windows audio engine and the process isolation around an overlay.
Typical measured numbers on a recent Windows 11 machine (onboard Realtek or a common USB headset):
Add a conservative VAD (Silero or a tiny local model) and you are already at 80–150 ms before the first sample reaches the ASR runtime. The model’s 35 ms is then a small fraction of the end-to-end delay the user actually feels.
For low-latency capture the useful path is still WASAPI exclusive or a carefully tuned shared-mode session with a short period.
// Sketch: exclusive-mode capture, event-driven
IMMDeviceEnumerator *enumerator;
IMMDevice *device;
IAudioClient *client;
IAudioCaptureClient *capture;
WAVEFORMATEX *fmt;
// ... CoCreate, GetDefaultAudioEndpoint(eCapture, eConsole) ...
client->Initialize(AUDCLNT_SHAREMODE_EXCLUSIVE,
AUDCLNT_STREAMFLAGS_EVENTCALLBACK,
0, 0, fmt, nullptr);
client->SetEventHandle(hEvent);
client->GetService(IID_PPV_ARGS(&capture));
client->Start();
In exclusive mode you own the device. Buffer size is negotiated with the driver; many USB headsets will accept 10 ms or lower. Shared mode is safer for coexistence with other apps but forces you into the engine’s mix period (often 10 ms base, sometimes longer under load). Loopback for system audio (the other side of a call) is only available in shared mode and adds the mix delay.
If you are building an overlay that must stay out of the way of the call application itself, you almost always end up in a separate process. That means another copy of the audio engine client, extra scheduling, and the usual problems with DPI awareness, UAC, and focus stealing. The capture thread still has to stay at a high priority or you start dropping buffers the moment the UI thread paints the overlay.
The Coval 35 ms figure is for a dedicated GPU endpoint. Running the same open weights locally (llama.cpp, Ollama, Foundry Local, or a custom ONNX/DirectML path) trades latency for privacy and offline use. On a machine with a decent NPU or a mid-range GPU a quantized 1.7 B model can still stay under 100–150 ms once the audio is buffered, which is acceptable for many interview and call scenarios once the capture path is cleaned up.
The practical stack many people end up with looks like:
Each hop adds measurable delay. The model is no longer the place to look first.
The open-weight model removes one class of excuses. The remaining work is the unglamorous audio plumbing that has been the same for years.
If you are shipping a Windows tool that does live language help on calls or interviews, the capture and overlay path is the product. The ASR model is a replaceable component. Once the end-to-end delay is under roughly 200–250 ms most users stop noticing the lag; above that the conversation feels broken even if the transcription is perfect.
Qwen3 ASR 1.7B at 35 ms on a hosted endpoint (or low hundreds of milliseconds local) simply moves the bottleneck back to the place it has always lived on Windows: the audio engine, the buffer sizes you negotiate, and the process isolation you need for a non-intrusive overlay.
The numbers will keep improving. The capture budget will not shrink by itself.
I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.