Reviser: Revision-Capable Text Generation via Autoregressive Cursor Actions

Researchers introduced Reviser, a decoder-only Transformer that generates text as a sequence of cursor-relative actions — INSERT(token), MOVE(Δ), or STOP — on a mutable canvas, enabling non-monotonic generation while keeping a next-action interface. In arena evaluations on a continuation benchmark, Reviser was strongly preferred to SEDD and MDLM, with trajectory statistics confirming frequent backward moves and mid-canvas insertions rather than end-append decoding. Reviser was competitive with size-matched autoregressive baselines at the 100M and 300M scales and, under the authors' shared FLOPs convention, required substantially less inference compute than representative multi-pass refinement and diffusion-style baselines.

arXiv:2609.20830v1 Announce Type: new Abstract: Revision-capable generation is appealing because it can insert or revise earlier content, but many non-autoregressive and edit-based approaches obtain this flexibility through repeated sequence-level computation. We propose Reviser, a decoder-only Transformer that generates a response as a sequence of cursor-relative actions on a mutable canvas. At each step, Reviser predicts exactly one action token: INSERT token , MOVE $\Delta$ , or STOP, and is autoregressive over edit-history actions rather than final text order. This design enables genuinely non-monotonic generation while preserving a simple next-action interface. On a continuation benchmark, Reviser is strongly preferred to SEDD and MDLM in our arena evaluations, and trajectory statistics confirm that the model performs frequent backward moves and mid-canvas insertions rather than merely emulating end-append decoding. Against size-matched autoregressive baselines, Reviser is competitive at both the 100M and 300M scales. Under our shared FLOPs convention, Reviser also requires substantially less inference compute than representative multi-pass refinement and diffusion-style baselines.