Foundry Local + Qwen3.8-27B Turns GitHub Copilot Local — Windows Audio Still Owns the Real-Time Budget

Microsoft's Foundry Local release lets developers run Qwen3.8-27B, a dense 27B vision-language model with 262K context and tool-calling, as a local OpenAI-compatible endpoint that GitHub Copilot can use via a bring-your-own-key provider. On an RTX 5090, Microsoft measured 1.5–1.9x faster generation than llama.cpp across prompts from 4K to 260K tokens using ONNX Runtime, CUDA graphs, INT4 weights and speculative decoding, with Terminal Bench 2.1 at 73.0 and SWE-bench Pro at 61.7. The writeup notes that for real-time audio on Windows, the OS audio graph (80–120 ms capture buffering plus 40–60 ms loopback) remains the dominant latency term, not the model.

Microsoft’s Foundry Local blog on 5 November 2026 put a concrete local agent path in front of every developer with a 24 GB+ NVIDIA GPU. Load the 4-bit Qwen3.8-27B ~18 GB , point GitHub Copilot’s Bring-Your-Own-Key provider at the local OpenAI-compatible endpoint, and the model picker shows a private, offline, zero-per-token coding assistant. The release notes and CLI steps are short. The practical surface is larger. Qwen3.8-27B is a dense 27 B vision-language model with a 262 K native context, thinking mode, and tool-calling. Microsoft optimized the full pipeline on ONNX Runtime + CUDA: DFlash2 speculative decoding, CUDA graph fusions tuned for SM121, INT4 weights, KV-cache quantization, prefix caching, and continuous batching. On an RTX 5090 they measured 1.5–1.9× generation speed versus llama.cpp across prompt lengths from 4 K to 260 K tokens, with the gap widening past 131 K — the regime of repository-scale agent work. Terminal Bench 2.1 sits at 73.0, SWE-bench Pro at 61.7. For everyday coding loops this is now a real option rather than a demo. The CLI path is three commands after the WinGet install: winget install Microsoft.FoundryLocal foundry model load qwen3.8-27b foundry server status foundry server status prints the base URL usually http://127.0.0.1:<port . In the Copilot app you add a Foundry Local provider, paste the URL, pick Qwen3.8-27B from the catalog, and switch the model. CUDA and cuDNN must be on PATH; Foundry Local does not bundle them. Once the server is up the OpenAI-compatible /v1/chat/completions and tool-calling endpoints are live. Prefix caching means repeated repository context does not re-prefill from zero. Continuous batching lets the same process handle multiple agent turns without serializing everything behind a single lock. On Windows the binary already carries the usual application manifest. That removes one class of “why won’t the CUDA DLLs load under a scheduled task or overlay process” failures. The remaining work is the usual: pin the DLL search path, keep the service account’s PATH correct, and make sure the GPU is not already saturated by a game or another inference server. None of the above changes the dominant term once the workload leaves pure text generation and enters real-time audio. A local 27 B model that can emit a coding patch in a few hundred milliseconds is still useless for a live interview or call if the microphone path has already burned 80–120 ms of buffering and the loopback path has added another 40–60 ms before the first sample reaches your process. Typical measurements on a recent Windows 11 machine with an onboard Realtek and a USB headset still show: The model is no longer the long pole; the OS audio graph is. Speculative decoding and INT4 help the GPU side. They do nothing for the WASAPI shared-mode period, the Bluetooth SCO/A2DP latency, or the fact that a global hotkey overlay must still pull samples through the same engine. A practical local speech pipeline on Windows therefore looks like this: The capture and injection steps dominate. Everything else can be swapped. That is why tools that own the Windows audio path and the overlay — rather than just wrapping another OpenAI-compatible endpoint — still matter once you leave pure coding agents. Local coding agents just crossed a practical threshold. Local real-time speech tools have not. The capture stack, the overlay, and the end-to-end latency budget are still the parts that require hands-on Windows work rather than another model download. I'm building Talkaside https://talkaside.com/ , a Windows app for job interviews and calls in another language. Drafted with AI assistance from this project's code and logs.