# Advancing the Quality-Efficiency Frontier for Coding AI on Windows PCs

> Source: <https://devblogs.microsoft.com/foundry-on-windows/coding-ai-on-windows-pcs-quality-efficiency/>
> Published: 2026-11-04 21:13:22+00:00

Powered by NVIDIA RTX Spark, we are bringing the promise of unmetered local intelligence to increasingly demanding coding workloads—giving developers greater freedom to experiment, delegate, and iterate on their own hardware, without per-token cloud charges.

Highly capable mixture-of-experts (MoE) models are making this opportunity tangible. These models tackle complex coding tasks spanning reasoning, implementation, testing, and debugging, while advances in efficiency are bringing more of that capability onto local devices. With 128 GB of unified memory, RTX Spark PCs enables us to ask a crucial question: **how much useful coding intelligence can we deliver within a developer’s machine?**

Low-bit compute is a key enabler for this opportunity. Reducing precision lowers model memory requirements and can reduce inference bandwidth demands, creating room for larger models and longer contexts. But smaller is not automatically better: practical gains depend on preserving quality while coordinating model precision, inference runtimes, kernels, and memory management. The goal is more useful intelligence within the available resources, not fewer bits at the expense of accuracy or responsiveness.

**[MAI-Code-1.1-Flash](https://microsoft.ai/news/mai-code-1-1-flash-br-better-faster-at-a-quarter-of-the-cost/)** reduces its weight footprint from 276 GB at 16-bit precision to a 53 GB mixed-precision checkpoint, delivering a median of **51.12 tokens per second with a 120K-token input context on RTX Spark** in our measured configuration. This shows how model and systems advances can turn smaller footprints into more capable local AI.

After the user starts the task, Copilot audits the codebase for unused code, imports, dependencies, and obsolete feature flags, investigates findings across the repository, and produces an actionable cleanup plan, with the low-bit model running locally on RTX Spark PCs.

## **Balancing Quality, Footprint, and Responsiveness**

Running a large coding model locally requires balancing quality, footprint, and responsiveness within one device budget. Quantization reduces the bytes occupied by each weight, but the target model, KV cache, and runtime must still fit within the available allocation. Because every token streams the active weights through memory, the design must also account for memory-bandwidth pressure.

These constraints create three coupled problems. At the model level, mixed precision and careful calibration are needed to preserve quality under low-bit quantization. At the inference level, low-bit formats need efficient kernels to improve responsiveness. At the system level, KV-cache memory grows with context length and competes with model weights for the available device footprint.

The practical objective is therefore not simply to minimize precision. Model quality, weight and KV-cache footprint, and decoding responsiveness must be optimized together within the available device resources. MAI-Code-1.1-Flash provides one concrete example of this approach.

## **How Low-Bit Models and Systems Engineering Enable Local AI: The MAI-Code-1.1-Flash Example**

[MAI-Code-1.1-Flash](https://microsoft.ai/news/mai-code-1-1-flash-br-better-faster-at-a-quarter-of-the-cost/) is a 137.9-billion-parameter mixture-of-experts model. To enable practical on-device coding, we addressed three coupled challenges: fitting the model weights, accelerating decoding, and supporting long coding contexts within the device’s memory budget. Quantization reduced the weight footprint from roughly 276 GB at 16-bit precision to **53.0 GiB (3.31 bpw)**, with a more compact **46.1 GiB (2.87 bpw)** variant also developed. The released 53.0 GiB checkpoint is available through Microsoft Foundry and as the local model in GitHub Copilot; it is the baseline for this section’s system measurements unless otherwise noted.

We tackled the first challenge—fitting the model weights—through mixed-precision quantization. MoE experts contain approximately 96.6% of MAI-Code-1.1-Flash’s parameters and therefore dominate its weight footprint, but sensitivity to quantization varies across tensor roles and layers. To reach these smaller footprints while preserving coding quality, we explored precision allocations guided by two complementary signals: **tensor role** and **calibration-derived sensitivity.**

### **Optimizing Mixed Precision for Coding Quality and Model Footprint**

Tensor-role-based allocation assigns different precisions to expert projections and other model components. In **Q2-XXS**, expert gate and up projections account for 64.4% of parameters and use about 2 bits per weight, while expert down projections account for 32.2% and use about 4 bits per weight. The remaining 3.4%—attention, dense feed-forward, embedding, output, routing, and normalization weights—retain higher precision where needed. The resulting weight artifact occupies **46.1 GiB** at an average of **2.87 bpw**; this excludes KV-cache and other runtime memory.

Calibration-derived layer sensitivity provides another way to generate candidate allocations. Quantization error depends on both the weight perturbation and the activation flowing through it: Δy = (ŵ − w)x. Because each expert sees only the tokens routed to it, we collect activation statistics from representative coding trajectories while preserving the observed routing.

These sensitivity signals helped us generate and screen candidate tensors more amenable to quantization but did not directly determine the final recipe for precision allocation. Layer sensitivity varied substantially, and more aggressive layer-level precision reduction did not consistently improve the quality-size tradeoff. We therefore selected the final quantized models through end-to-end evaluation, as shown in Figure 3.

End-to-end evaluation revealed several variants with similar coding quality at different model sizes, making selection a tradeoff between quality and footprint. We therefore compared candidates across a broad set of coding and non-coding benchmarks rather than relying on a single metric. Figure 3 illustrates this with SWE-Bench Verified and Terminal-Bench 2.1: variants that appear nearly equivalent on one benchmark can separate on another.

This led to two useful on-device operating points. **Q2-XS** (53.0 GiB, 3.31 bpw) retains a larger weight footprint, while **Q2-XXS** (46.1 GiB, 2.87 bpw) pushes compression further. The two designs occupy different points on the quality-size frontier rather than representing a single universally optimal precision assignment.

The broader lesson is that precision allocation is an empirical model-design problem. Calibration and sensitivity analysis can narrow the search space, but the final operating point must be validated on the workloads the model is expected to execute.

**Accelerating Decoding Speed for Long Coding Tasks**

Quantization reduces the model footprint; speculative decoding improves throughput by letting a smaller draft propose token blocks for the target model to verify in one pass.

For MAI-Code-1.1-Flash, we trained a draft model from scratch using the open [DFlash2](https://inco.ai/blog/dflash2/) architecture, built on [DFlash](https://arxiv.org/abs/2602.06036). It uses target-model internal information and block diffusion to predict token blocks, with grouped dynamic convolutions to share information across neighboring positions and a lightweight selector to choose a coherent candidate sequence.

Across held-out BigCodeBench, HumanEval, and LiveCodeBench benchmarks, candidate selection raised **mean accepted tokens (MAT) from 4.26 to 4.43** with the same draft. Speedup also depends on drafting and verification costs: latency per token is (*t*<sub>draft</sub> + *t*<sub>verify</sub>) / *τ*, where *τ* is MAT per verification cycle.

For long coding tasks, the draft’s growing KV cache competes with model weights for device memory. At **262K context**, the unbounded draft cache reaches nearly 70% of the target model’s KV-cache memory. We bound this growth with **8K sliding-window attention in the first four layers** of the five-layer DFlash2 draft, keeping full-context attention in the final layer.

Without retraining, this change reduced reserved draft-KV memory at 262K context from **5,120 MiB to 1,156 MiB (77.4%)**, while draft acceptance, estimated MAT, and decode throughput differed by no more than **2%** from the full-attention draft. On a fixed 30-task SWE-Bench set under the same serving configuration, pass/fail outcomes agreed on **29 of 30 tasks**, suggesting similar behavior on this test set.

### **Measured Performance on RTX Spark**

On RTX Spark (**128 GB unified memory; 64 GB GPU allocation**), we compared the same Q2-XS target alone and with our DFlash2 draft across six input lengths from **1K to 120K tokens**. Each request generated **8,192 tokens**, with three repetitions per case; results report medians.

Under this measured configuration, the draft increased median decode throughput by **1.52–1.98×** across tested input lengths. At 120K input, median throughput rose from **25.81 to 51.12 tokens/s**, while median end-to-end latency fell **27%**, from **527.25 to 382.41 seconds**. These figures describe this specific Q2-XS and DFlash2 setup and should not be generalized to different serving conditions.

With layer-wise sliding-window attention, peak GPU memory in the draft-enabled configuration rose from **59.42 to just 63.50 GB** as input grew from 1K to 120K tokens.

## **Bringing Useful Coding Intelligence onto the Developer’s Machine**

We opened with a question: how much useful coding intelligence can we deliver within a developer’s machine? Low-bit inference creates room for more useful coding work close to a developer’s data and tools, while cloud-scale models remain available when tasks require greater capability. MAI-Code-1.1-Flash makes this complementary local-and-cloud approach tangible, giving developers more freedom to experiment, delegate, and iterate without per-token cloud charges.

Lower precision does more than shrink the model. It releases memory and bandwidth that can be reinvested in more capable models, longer contexts and larger KV caches, speculative decoding, and greater concurrency. The optimization target therefore shifts from bits per weight in isolation to **token efficiency**: the useful intelligence a system delivers within a fixed device compute and memory budget. MAI-Code-1.1-Flash shows how quantization, speculative decoding, and bounded draft KV-cache growth compose into a practical on-device system.

The next frontier is a new generation of local coding agents that respond faster, sustain longer coding trajectories, and take on increasingly capable work within the same device budget. Advances in low-bit model design, speculative decoding, and KV-cache management can help make that experience practical. The ambition is not simply smaller models, but sustained, capable agentic coding on a developer’s own machine.

The era of low-bit AI compute is therefore not about minimizing the number of bits. It is about using precision as a system-design dimension to deliver more useful intelligence from the compute already available.

## **Closing**

We pushed the boundaries of what’s possible on today’s Windows PCs in a rapidly evolving, complex technical landscape. By advancing low-bit quantization, speculative decoding and KV-cache management, we have achieved remarkable gains in three critical areas with MAI-Code-1.1-Flash: memory footprint, decoding throughput and long-context efficiency, while preserving coding quality. These results underscore Microsoft’s commitment to developing models that are not only powerful in capability but also highly efficient. By making MAI-Code-1.1-Flash available as local model in GitHub Copilot on RTX Spark, Microsoft is bringing frontier-class coding intelligence onto the developer’s own Windows PC, giving developers more freedom to experiment, delegate and iterate without per-token cloud charges.

It has taken a village to innovate across multiple layers of the stack and bring MAI-Code-1.1-Flash on-device, and we are grateful to all our team members, including (listed in alphabetical order), Amol Ambardekar, Eric Sommerlade, Justin Wagle, Kumar Gaurav, Pashmina Cameron, Rohith Kukkala, Sunando Sengupta, Vivek Pradeep. The team also extends thanks to partner teams who supported the work, including Austin Hodges, Chris Dern, Daniel Pagan, Dirk Groeneveld, Greg Woo, Keith Mahoney, Kunal Vaishnavi, Lei Xu, Logan Iyer, Murali Ambati, Ramya Krishna Akula, Rutvi Trivedi, Tianlei Wu, Tim Kurtzman, Tucker Burns, Urmish Thakker, Vicente Rivera, Wei Fang, Weizhu Chen, Yanan Cai, Young Jin Kim, Yuting Sun.