TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

Researchers propose Trajectory-Aware Commit Gating (TACG), a training-free decoder for diffusion language models that uses trajectory-aware signals to decide when to commit tokens, improving accuracy and efficiency on code and math benchmarks.

arXiv:2607.03236v1 Announce Type: new Abstract: Diffusion language models DLLMs generate text by iteratively denoising masked positions, exposing a trajectory of predictive distributions rather than a single instantaneous belief. Most existing decoders ignore this trajectory and commit tokens from the current snapshot alone, conflating confidence with commitment readiness: a transient top-1 peak under incomplete context can be locked in, while candidates with consistent cross-step support are delayed. We propose Trajectory-Aware Commit Gating TACG , a training-free gate-level decoder that anchors token identities to the base posterior and uses trajectory-aware signals only to decide whether the current proposal is ready to commit. TACG combines Temporal Implicit Logits Guidance TILG , which keeps an exponential moving average of past logits as a self-reference and contrasts the current logits against this reference in natural-parameter space, with a History Gate HG that enforces short-term proposal persistence before commitment. Together with a capped extra-promotion budget, these components yield a stability-constrained commit rule without auxiliary networks or extra forward passes. We evaluate TACG on LLaDA, Dream, and LLaDA2-Mini across code HumanEval, MBPP and math GSM8K, MATH500 benchmarks; it typically improves or preserves accuracy while reducing denoising steps and increasing tokens per forward TPF . The code is publicly available at https://github.com/Clarence-CV/TACG-DLLM.