Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs

Researchers introduced Polestar, a training-free inference framework for diffusion large language models (dLLMs) that uses token representation drift to improve efficiency. Polestar comprises two components—Polestar-Cache for sparse KV-cache refreshes and Polestar-Commit for identifying commit-ready tokens—achieving up to 10.73% accuracy improvement and 3.7x higher throughput on mathematics and coding benchmarks.

arXiv:2607.14107v1 Announce Type: new Abstract: The inference efficiency of diffusion large language models dLLMs is constrained by two challenges: bidirectional attention precludes efficient KV-cache reuse, while increasing decoding parallelism with static confidence thresholds can compromise generation quality. We observe that both challenges arise from a shared phenomenon: as tokens are decoded, their contextual integration through bidirectional attention causes token representations to drift evolve across decoding steps. This insight motivates Polestar, a training-free inference framework that uses token representation drift as a unified signal to jointly address both challenges. Polestar comprises two components: Polestar-Cache, which identifies stale KV-cache positions via drift and performs sparse KV-cache refreshes to enable efficient reuse, and Polestar-Commit, which detects sharp drift events to reliably identify commit-ready tokens. Across mathematics and coding benchmarks on several dLLM families, Polestar sets a new state of the art on the accuracy-throughput Pareto frontier, achieving up to 10.73% accuracy improvement, up to 3.7x higher throughput, and high decoding parallelism of 3.67 tokens per forward pass over existing baselines.