cd /news/artificial-intelligence/oxford-reveals-how-huginn-rebuilds-t… · home › topics › artificial-intelligence › article
[ARTICLE · art-172512] src=alphasignal.ai ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Oxford Reveals How Huginn Rebuilds Thinking Stages on Every Loop

An Oxford-led team's NeurIPS-accepted paper, "A Mechanistic Analysis of Looped Reasoning Language Models," found that each loop in a looped LLM repeats the same processing stages a feedforward Transformer runs once, with each layer in the shared block settling into its own fixed point so the loop retraces the same latent-space path on every pass. Huginn-0125, Retrofitted Llama and Parcae converge quickly, while Ouro-1.4B keeps drifting and loses its stages when run past its trained loop count; Huginn is the exception with no clear stages, which the authors suspect stems from massive activations and test by ablation. Input injection produced stable fixed points for every normalization type tested, whereas without it only pre-norm converged, a distinction the authors tie to test-time scaling: increasing loop count is predictable only when recurrent computation converges.

recommendation

Cap loop counts at the trained recurrence range for looped models like Ouro-1.4B, and use input injection, which gave stable fixed points for every normalization type tested, to keep recurrent passes convergent.

more in playbook: Inference and self-hosting
read3 min views2 publishedNov 19, 2026
Oxford Reveals How Huginn Rebuilds Thinking Stages on Every Loop
Image: Alphasignal (auto-discovered)

A NeurIPS paper looks inside looped language models and finds that each pass through the shared block repeats the same processing stages a normal Transformer runs once. Whether those stages hold steady depends on a few design choices.

  • A NeurIPS paper finds that each loop in a looped LLM repeats the processing stages a feedforward Transformer runs once.

  • Each layer in the shared block settles into its own fixed point, so the loop retraces the same path on every pass.

  • Huginn, Retrofitted Llama and Parcae converge quickly. Ouro keeps drifting and loses its stages when run past its trained loop count.

  • Input injection gives stable fixed points for every normalization type tested. Without it, only pre-norm converges.

  • Huginn is the exception with no clear stages; the authors suspect massive activations and test this by ablation.

  • The paper releases no new model; the Parcae code and weights it studies are open. Looped language models apply one block of Transformer layers several times, increasing computation without adding parameters. An Oxford-led team examined their internal behavior in A Mechanistic Analysis of Looped Reasoning Language Models (paper), accepted at NeurIPS. The researchers tracked hidden states, attention patterns, and task readouts across recurrences to determine whether shared layers recreate the inference stages found in conventional feedforward models.

Most of the models settled into stable, layer-specific cycles and repeated a recognizable sequence of inference stages on every pass. Ouro continued to drift beyond its training range. The distinction matters for test-time scaling: increasing the loop count is predictable only when the recurrent computation converges.

One block, many passes #

A conventional Transformer applies each layer once from bottom to top. Looped architectures reuse a stack of layers, often placing a one-time prelude

before the recurrent block and a one-time coda

after it. Input injection can also return the encoded prompt to the block at every recurrence, giving each pass direct access to the original tokens.

  • Re-read models such as Huginn, Rec-OLMo, and Parcae initialize the recurrent state from noise, inject the prompt on every pass, and train with varying recurrence counts.
  • Ouro starts from token embeddings and repeatedly applies its full stack without a separate prelude and coda.

Prior interpretability studies of feedforward language models have identified stages of inference

: early layers mix information broadly across tokens, intermediate layers transform and compress representations, and later layers sharpen the output prediction. A looped model could distribute those stages across its full recurrence history or rebuild them inside each pass.

The authors studied Huginn-0125, Ouro-1.4B, Parcae, and Llama models retrofitted with recurrent middle layers. Their controls included feedforward models and a Parcae configuration with the same parameter count trained for a single pass.

Per-layer cycles emerge #

A fixed point is a state left unchanged by another application of the same function. The recurrent models generally did not collapse every internal layer onto one shared state. Instead, the state at each layer converged across recurrences to a layer-specific value. Traversing the block then produced a stable cycle through those values, with every recurrence retracing the same latent-space path.

This story is for Pro members #

You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @oxford 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/oxford-reveals-how-h…] indexed:0 read:3min 2026-11-19 · —