cd /news/large-language-models/recurrent-depth-in-llms-the-arithmet… · home › topics › large-language-models › article
[ARTICLE · art-170556] src=pub.towardsai.net ↗ pub= topic=large-language-models verified=true sentiment=· neutral

Recurrent Depth in LLMs: The Arithmetic Behind Looped Transformers

NVIDIA's Dory looped transformer stores 36 layers but executes 96 per token by running a 20-layer core four times, according to an arithmetic breakdown of recurrent-depth models published on piszczek.pl. Fastino's 10 November 2026 report on its 27B GLiDE decision model puts the trade-off at roughly +35% latency for an accuracy gain whose confidence interval excludes zero, after about 110 candidate configurations, with no clear benefit below 1B parameters and a third pass that hurt a causal-reasoning benchmark. The analysis notes Dory's authors report 4 loops generate 1.3–2.5× fewer tokens than 1 loop on agentic benchmarks, while per-token cost rises with executed layers, so the deciding comparison is total cost per correct, checked task.

by read3 min views8 publishedNov 16, 2026

Disclosure: drafted with AI assistance; every number checked against the sources linked at the end.

36 = 96 looks like a typo. For NVIDIA’s Dory it’s a configuration: 36 layers stored, 96 executed per token, same weights throughout.

Looped transformers are usually explained with a diagram and a metaphor. This article does it with arithmetic, because the arithmetic is where most misreadings come from. All numbers are from public papers and model cards; I didn’t benchmark these models myself.

A looped transformer has three parts:

Written as a recurrence (a simplification, not a spec for every model):

h₀      = prelude(x)h_{r+1} = core(h_r, x; θ)        for r = 0 … R−1logits  = coda(h_R)

Two details matter. The core’s weights θ are the same on every pass. And x appears inside the core: NVIDIA’s Dory reinjects the input on each pass, so the loop doesn’t drift away from the question.

Dory’s published layout is 8 prelude layers, a 20-layer core and 8 coda layers.

def depth(R, pre=8, core=20, coda=8):    stored = pre + core + coda    executed = pre + core * R + coda    return stored, executed
for R in range(1, 5):    s, e = depth(R)    print(f"R={R}: stored={s}, executed={e}, ratio={e/s:.2f}")

Weight memory follows the stored column. Compute per token follows the executed column, to a first approximation (attention and KV handling change the exact FLOPs, so treat this as layer arithmetic, not a latency model).

A toy version makes the trade obvious: 8 blocks run 3 times execute 24 blocks while storing 8. That’s the appeal of weight sharing. It doesn’t promise the accuracy of a real 24-block network, because each reused block has to work on a different state every pass, and learning that is the hard part.

Naively, each loop could keep its own attention cache, and cache memory would grow with R. Dory’s paper reports a mode that keeps only the final loop’s KV cache: memory stays at the non-looped footprint, and most of the accuracy gain is retained.

The lesson is general. For looped models, cache growth is an implementation choice. Check the serving mode before sizing hardware.

Fastino’s report on its 27B decision model GLiDE (10 November 2026) is the clearest public price tag:

That’s roughly +35% latency for a gain whose interval excludes zero. Whether it’s worth it depends on what a decision is worth and what response time you can afford. The report also mentions about 110 candidate configurations tried, several failed designs, no clear benefit below 1B parameters, and a third pass that hurt a causal-reasoning benchmark.

Dory’s authors report that 4 loops produce 1.3–2.5× fewer generated tokens than 1 loop on agentic benchmarks. Per-token cost, meanwhile, rises with executed layers (36 → 96 in the layer arithmetic above). Which effect wins is an empirical question per workload. The comparison that settles it is total cost per correct, checked task, including retries.

Repeating a learned function has no built-in guarantee of improvement. Three trajectories are all possible: quality keeps rising with diminishing returns, quality plateaus, or quality peaks and falls. Compute rises in all three.

Practical consequences:

Loops add computation, not information. If the needed fact isn’t in the prompt or the weights, more passes over the same state can’t produce it. Retrieval or a tool call can. Diagnose the failure before picking the fix:

This article expands the arithmetic from my guide Looped Transformers Explained*, originally published on piszczek.pl. The original adds the history from ACT and Universal Transformers to Huginn, a comparison with chain of thought and agent loops, and seven interactive labs where you can change the loop count and watch these numbers move.*

Recurrent Depth in LLMs: The Arithmetic Behind Looped Transformers was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

── more in #large-language-models 4 stories · sorted by recency
── more on @nvidia 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/recurrent-depth-in-l…] indexed:0 read:3min 2026-11-16 · —