# Oxford Reveals How Huginn Rebuilds Thinking Stages on Every Loop

> Source: <https://alphasignal.ai/news/oxford-reveals-how-huginn-rebuilds-thinking-stages-on-every-loop>
> Published: 2026-11-19 12:02:33+00:00

**Recommendation:** Cap loop counts at the trained recurrence range for looped models like Ouro-1.4B, and use input injection, which gave stable fixed points for every normalization type tested, to keep recurrent passes convergent.
More in the playbook "Inference and self-hosting": <https://wpnews.pro/playbooks/inference-serving>

# Oxford Reveals How Huginn Rebuilds Thinking Stages on Every Loop

A NeurIPS paper looks inside looped language models and finds that each pass through the shared block repeats the same processing stages a normal Transformer runs once. Whether those stages hold steady depends on a few design choices.

- A [NeurIPS paper](https://arxiv.org/html/2604.11791v2?utm_source=alphasignal) finds that each loop in a looped LLM repeats the processing stages a feedforward Transformer runs once.
- Each layer in the shared block settles into its own fixed point, so the loop retraces the same path on every pass.
- Huginn, Retrofitted Llama and Parcae converge quickly. Ouro keeps drifting and loses its stages when run past its trained loop count.
- Input injection gives stable fixed points for every normalization type tested. Without it, only pre-norm converges.
- Huginn is the exception with no clear stages; the authors suspect massive activations and test this by ablation.
- The paper releases no new model; the [Parcae code and weights](https://github.com/sandyresearch/parcae?utm_source=alphasignal) it studies are open.

Looped language models apply one block of Transformer layers several times, increasing computation without adding parameters. An Oxford-led team examined their internal behavior in *A Mechanistic Analysis of Looped Reasoning Language Models* ([paper](https://arxiv.org/html/2604.11791v2?utm_source=alphasignal)), accepted at NeurIPS. The researchers tracked hidden states, attention patterns, and task readouts across recurrences to determine whether shared layers recreate the inference stages found in conventional feedforward models.

Most of the models settled into stable, layer-specific cycles and repeated a recognizable sequence of inference stages on every pass. Ouro continued to drift beyond its training range. The distinction matters for test-time scaling: increasing the loop count is predictable only when the recurrent computation converges.

## One block, many passes

A conventional Transformer applies each layer once from bottom to top. Looped architectures reuse a stack of layers, often placing a one-time prelude

 before the recurrent block and a one-time coda

 after it. Input injection can also return the encoded prompt to the block at every recurrence, giving each pass direct access to the original tokens.

- **Re-read models** such as Huginn, Rec-OLMo, and Parcae initialize the recurrent state from noise, inject the prompt on every pass, and train with varying recurrence counts.
- **Ouro** starts from token embeddings and repeatedly applies its full stack without a separate prelude and coda.

Prior interpretability studies of feedforward language models have identified stages of inference

: early layers mix information broadly across tokens, intermediate layers transform and compress representations, and later layers sharpen the output prediction. A looped model could distribute those stages across its full recurrence history or rebuild them inside each pass.

The authors studied Huginn-0125, Ouro-1.4B, Parcae, and Llama models retrofitted with recurrent middle layers. Their controls included feedforward models and a Parcae configuration with the same parameter count trained for a single pass.

## Per-layer cycles emerge

A fixed point is a state left unchanged by another application of the same function. The recurrent models generally did not collapse every internal layer onto one shared state. Instead, the state at each layer converged across recurrences to a layer-specific value. Traversing the block then produced a stable cycle through those values, with every recurrence retracing the same latent-space path.

## This story is for Pro members

You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
