Recurrent Depth in LLMs: The Arithmetic Behind Looped Transformers NVIDIA's Dory looped transformer stores 36 layers but executes 96 per token by running a 20-layer core four times, according to an arithmetic breakdown of recurrent-depth models published on piszczek.pl. Fastino's 10 November 2026 report on its 27B GLiDE decision model puts the trade-off at roughly +35% latency for an accuracy gain whose confidence interval excludes zero, after about 110 candidate configurations, with no clear benefit below 1B parameters and a third pass that hurt a causal-reasoning benchmark. The analysis notes Dory's authors report 4 loops generate 1.3–2.5× fewer tokens than 1 loop on agentic benchmarks, while per-token cost rises with executed layers, so the deciding comparison is total cost per correct, checked task. Disclosure: drafted with AI assistance; every number checked against the sources linked at the end. 36 = 96 looks like a typo. For NVIDIA’s Dory it’s a configuration: 36 layers stored, 96 executed per token, same weights throughout. Looped transformers are usually explained with a diagram and a metaphor. This article does it with arithmetic, because the arithmetic is where most misreadings come from. All numbers are from public papers and model cards; I didn’t benchmark these models myself. A looped transformer has three parts: Written as a recurrence a simplification, not a spec for every model : h₀ = prelude x h {r+1} = core h r, x; θ for r = 0 … R−1logits = coda h R Two details matter. The core’s weights θ are the same on every pass. And x appears inside the core: NVIDIA’s Dory reinjects the input on each pass, so the loop doesn’t drift away from the question. Dory’s published layout is 8 prelude layers, a 20-layer core and 8 coda layers. python def depth R, pre=8, core=20, coda=8 : stored = pre + core + coda executed = pre + core R + coda return stored, executed for R in range 1, 5 : s, e = depth R print f"R={R}: stored={s}, executed={e}, ratio={e/s:.2f}" R=1: stored=36, executed=36, ratio=1.00 R=2: stored=36, executed=56, ratio=1.56 R=3: stored=36, executed=76, ratio=2.11 R=4: stored=36, executed=96, ratio=2.67 Weight memory follows the stored column. Compute per token follows the executed column, to a first approximation attention and KV handling change the exact FLOPs, so treat this as layer arithmetic, not a latency model . A toy version makes the trade obvious: 8 blocks run 3 times execute 24 blocks while storing 8. That’s the appeal of weight sharing. It doesn’t promise the accuracy of a real 24-block network, because each reused block has to work on a different state every pass, and learning that is the hard part. Naively, each loop could keep its own attention cache, and cache memory would grow with R. Dory’s paper reports a mode that keeps only the final loop’s KV cache: memory stays at the non-looped footprint, and most of the accuracy gain is retained. The lesson is general. For looped models, cache growth is an implementation choice. Check the serving mode before sizing hardware. Fastino’s report on its 27B decision model GLiDE 10 November 2026 is the clearest public price tag: That’s roughly +35% latency for a gain whose interval excludes zero. Whether it’s worth it depends on what a decision is worth and what response time you can afford. The report also mentions about 110 candidate configurations tried, several failed designs, no clear benefit below 1B parameters, and a third pass that hurt a causal-reasoning benchmark. Dory’s authors report that 4 loops produce 1.3–2.5× fewer generated tokens than 1 loop on agentic benchmarks. Per-token cost, meanwhile, rises with executed layers 36 → 96 in the layer arithmetic above . Which effect wins is an empirical question per workload. The comparison that settles it is total cost per correct, checked task, including retries. Repeating a learned function has no built-in guarantee of improvement. Three trajectories are all possible: quality keeps rising with diminishing returns, quality plateaus, or quality peaks and falls. Compute rises in all three. Practical consequences: Loops add computation, not information. If the needed fact isn’t in the prompt or the weights, more passes over the same state can’t produce it. Retrieval or a tool call can. Diagnose the failure before picking the fix: This article expands the arithmetic from my guide Looped Transformers Explained , originally published on piszczek.pl. The original adds the history from ACT and Universal Transformers to Huginn, a comparison with chain of thought and agent loops, and seven interactive labs where you can change the loop count and watch these numbers move. Recurrent Depth in LLMs: The Arithmetic Behind Looped Transformers https://pub.towardsai.net/recurrent-depth-in-llms-the-arithmetic-behind-looped-transformers-1ec5cfb5499f was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.