OpenVINO 2026.4: EAGLE-3, MTP, and NPU Profiling Are Here

Intel released OpenVINO 2026.4, adding EAGLE-3 tree drafting that lifts speculative decoding throughput on Qwen3-8B from 1.65x to over 1.85x at batch size 1, Multi-Token Prediction for Gemma 4, Qwen3.5 and Qwen3.6 on CPU and GPU, and unified VTune profiling of CPU, GPU and NPU in a single session. The release also makes Windows ML support official (gold), adds a Node.js ASRPipeline for Whisper and Qwen3-ASR, and ships DFlash acceleration for Qwen on GPU plus Xe3 iGPU optimizations for Gemma 4 long-context inference on Core Ultra Series 3. OpenVINO Model Server 2026.4 adopts tree and chain drafting and adds MTP in preview.

Intel’s OpenVINO 2026.4 arrived this week with three upgrades that close real developer gaps: EAGLE-3 tree drafting for higher-throughput speculative decoding on VLM pipelines, Multi-Token Prediction for Gemma 4 and the Qwen3 family, and — finally — a unified VTune profiling path that covers CPU, GPU, and NPU in a single session. If you’re running local inference on Intel hardware, this is a solid upgrade with immediate payoff. Speculative Decoding Gets Serious Two speculative decoding improvements land together, and they’re complementary. The bigger story is EAGLE-3 Tree Drafting . The old chain approach drafted one token at a time in a linear sequence. Tree drafting Top-K instead branches multiple candidate tokens at each step — the verifier then checks the entire tree in one pass and accepts the longest correct branch. At batch size 1, that shift moves throughput from 1.65x chain to over 1.85x on Qwen3-8B. That’s the batch size that defines almost every AI PC and local assistant scenario, so the win is real. OpenVINO Model Server 2026.4 picks up both tree and chain drafting, though tree is the recommended default for low-batch inference going forward. The second improvement is Multi-Token Prediction MTP , now available for Gemma 4, Qwen3.5, and Qwen3.6 on both CPU and GPU. Where standard decoding produces one token per forward pass, MTP predicts multiple tokens simultaneously, reducing total forward passes for a given output. The result is higher throughput and lower latency with no accuracy trade-off. OpenVINO Model Server adds MTP in preview. To use it: upgrade to 2026.4 and enable MTP in your generation config. If you’re running VLM pipelines and haven’t switched to tree drafting yet, do it now. The EAGLE-3 research https://arxiv.org/html/2503.01840v1 is clear on why it wins at low batch — and low batch is where most local inference workloads live. One Profiler for Everything This one is a quality-of-life win that will feel bigger in practice than it sounds on paper. Intel VTune can now profile CPU, GPU, and NPU execution in a single session via the extended ITT Instrumentation and Tracing Technology integration. Before 2026.4, the NPU was essentially a black box during profiling — you could measure what went in and what came out, but not what happened inside. Now you can identify whether the NPU is the bottleneck in a hybrid pipeline, see how workload dispatch is distributed across compute units, and tune accordingly. For developers building AI PC apps on Core Ultra hardware, this is the missing piece for production-quality optimization. Intel VTune Profiler is free to download; the NPU profiling capability activates automatically once you’re on 2026.4. Windows ML Gold, Node.js ASR, and the Rest Windows ML gold support is now official. Windows ML automatically routes to OpenVINO when it detects an Intel NPU, falls back to the GPU, and then to the CPU. The key word here is “gold”: this is no longer a preview API. If you’ve been building AI PC apps on Windows but waiting for stability guarantees, the wait is over. With 200+ AI PC designs shipping on Intel Core Ultra Series 3 this cycle, targeting the NPU is worth the effort. Node.js gets an ASR pipeline — finally. OpenVINO GenAI 2026.4 exposes ASRPipeline in Node.js with support for Whisper and Qwen3-ASR, including streaming output and performance metrics. The API matches C++ and Python pipelines, so the code patterns translate. This is the first clean path for JavaScript developers to do local speech recognition without a cloud API call. js const { ASRPipeline } = require 'openvino-genai' ; const pipe = new ASRPipeline 'whisper-base', 'NPU' ; const result = await pipe.generate audioBuffer ; Two smaller additions round out the release: DFlash acceleration for Qwen on GPU lower latency via visual-token reduction , Xe3 iGPU optimizations that improve Gemma 4 long-context inference on Core Ultra Series 3, and idle model management in OpenVINO Model Server preview that unloads inactive models to reduce memory pressure in multi-model deployments. How to Upgrade The upgrade is a standard pip update across three packages: pip install --upgrade openvino openvino-tokenizers openvino-genai Verify with: python python -c "import openvino; print openvino. version " Node.js users: update openvino-node to 2026.4.0. Full release notes and migration details are in the official OpenVINO documentation https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html . The GitHub release page https://github.com/openvinotoolkit/openvino/releases/tag/2026.4.0 has the complete changelog including breaking changes. The Intel Community announcement https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/OpenVINO-2026-4-Available-Now/m-p/1759536 covers deployment-specific guidance. Local inference on Intel hardware has been getting meaningfully better with each OpenVINO release this year. 2026.4 is not a rewrite, but EAGLE-3 tree drafting, unified NPU profiling, and stable Windows ML support together represent a genuine step forward for anyone who takes on-device inference seriously.