KV Caching for dLLMs is Noise Process Agnostic

The typical language model is autoregressive (AR): it predicts one token at a time, left to right, each token conditioned on the ones before it. This bakes in two limitations: it cannot revise earlier tokens if later context makes them appear wrong, and its latency grows linearly with length, since every token takes its own forward pass.

Diffusion language models avoid these problems. 1 Generation starts from a sequence of pure noise, and at each step the model predicts a cleaner version of every position at once. Repeat this for a fixed number of steps and the noisy sequence resolves into legible text. Because the whole sequence is refined in parallel, the model can revise any position as it goes.