Per-Token Fixed-Point Convergence in Depth-Recurrent Transformers
A 135M-class depth-recurrent transformer trained on FineWeb-Edu converges to a per-token fixed point, with mean successive-output KL divergence falling from 3.9e-1 at the second loop to 8.5e-6 by the sixteenth. The resea…