# Dory's Loop Transformer: How NVIDIA's 4B Model Beats Gemma 3 27B > Source: > Published: 2026-11-03 00:00:00+00:00 # Dory's Loop Transformer: How NVIDIA's 4B Model Beats Gemma 3 27B NVIDIA's Dory loops a 4B model's weights up to eight times per token, beating Gemma 3 27B on agentic benchmarks. Here's what the data shows. ## What is Dory, and why does it matter? Dory is a 3.97 billion parameter research model from NVIDIA’s Nemotron Labs that reuses the same set of weights multiple times per token instead of relying on a bigger stack of unique layers. NVIDIA calls it a looped transformer, also referred to as a latent recurrent transformer. On its own composite benchmark score, Dory outperforms Gemma 3 27B, a much larger mixture-of-experts model, despite having a fraction of the parameters sitting in GPU memory. The interesting part isn’t that it wins. It’s how it wins, and where it loses, because that split reveals a lot about what looping can and can’t do for model quality. ## TL;DR - **Dory splits its 72-layer structure into three blocks** : a 16-layer input block, a 16-layer output block, and a 40-layer recurrent block that loops. - **The number of loops is set at inference time** , not baked in during training, so the same checkpoint can run as a fast 1-loop model or a slower, deeper 4-loop or 8-loop model. - **More loops consistently improved benchmark win rate against Gemma 3 27B** , going from 1 out of 10 benchmarks at one loop, to 4 out of 10 at two loops, to 5 out of 10 at four loops. - **Dory wins on agentic tasks like instruction following, long context, and tasks resembling SWE-bench verified** , but loses badly on knowledge-heavy benchmarks like GPQA and Humanity’s Last Exam. - **The memory footprint of the weights never changes with more loops** (still 3.9 billion parameters on the GPU), though the KV cache grows unless you switch to a “last loop” caching mode. - **NVIDIA pretrained Dory on 6 trillion tokens** , with the first 4.5 trillion done natively in NVFP4 before switching to 16-bit, then added 300 billion tokens of context extension plus SFT and GRPO reinforcement learning. - **This approach scales compute per token rather than output length** , which is a different lever than the usual trick of just generating longer chains of thought. Auditable. Scalable. ## How does the loop mechanism actually work? A standard transformer runs every layer exactly once, start to finish. Dory instead treats its middle layers as a single reusable block and runs that block repeatedly before handing off to a fixed output block. The architecture breaks down into three parts: an input block (16 layers) that runs once, a recurrent block (40 layers) that loops a configurable number of times, and an output block (16 layers) that also runs once. Each pass through the recurrent block feeds back into the original input representation, similar in spirit to how ResNet architectures add the original input back into each transformation step. The difference is that instead of each block having its own distinct weights, Dory reuses the exact same 40 layers of weights on every pass. Conceptually, each loop re-exposes the model to the original token representation, letting it refine its internal state rather than drift off into something unrelated. Unrolled, a 4-loop pass through Dory behaves like a model with roughly 192 effective layers of depth, even though only 40 of those layers are unique. At one loop, effective depth is about 72 layers. At two loops, around 112. The compute cost scales with the depth, but the parameter count parked in GPU memory stays fixed at just under 4 billion. NVIDIA describes this as “elastic inference compute”: you can dial compute up or down at inference time without touching how much memory the model occupies. Crucially, Dory’s output block was trained to read the state coming out of any loop count. That means one loop, two loops, or eight loops are all valid, already-trained configurations. There’s no need to retrain or fine-tune for a specific number of loops. ## What do the benchmark numbers against Gemma 3 27B actually show? NVIDIA compared Dory’s looped configurations against Gemma 3 27B (a mixture-of-experts model with about 4 billion active parameters per token, so compute-per-token is roughly comparable even though total stored parameters differ enormously). The results scale directly with loop count: - At **one loop** , Dory wins 1 out of 10 benchmarks. - At **two loops** , Dory wins 4 out of 10 benchmarks. - At **four loops** , Dory wins 5 out of 10 benchmarks. The pattern in which benchmarks it wins is the real story. Dory performs well on agentic and instruction-following tasks, long-context handling, and benchmarks resembling SWE-bench verified, the kinds of tasks that reward careful, iterative reasoning about a given context. It performs poorly on knowledge-heavy benchmarks like GPQA and Humanity’s Last Exam, tasks that depend on the model having memorized a large volume of facts rather than reasoning more carefully about what’s already in front of it. *demands*of enterprises. **Passed** *✓*SOC 2 Type 2 *✓*HIPAA *✓*CAIQ attestation *✓*OCSF logging *✓*Role-based access *✓*Self-hosted That split makes sense given the architecture. Looping the same 40 layers more times gives the model more computation to refine its understanding of the current input, but it doesn’t add any new stored knowledge. Gemma 3 27B’s mixture-of-experts design keeps a similar active compute budget per token but has a much larger pool of total weights (its experts) to draw on, which functions as a bigger knowledge reservoir. Dory, at under 4 billion total parameters, simply doesn’t have anywhere near that much room to store facts, so it loses decisively on exam-style, trivia-heavy benchmarks no matter how many times it loops. ## What are the tradeoffs of looping instead of scaling up? The most obvious win is memory. Running Dory at four or eight loops costs more compute and more latency, but the weight footprint on the GPU never grows past roughly 4 billion parameters. That’s a meaningfully different cost curve than training or deploying a dense or mixture-of-experts model with tens of billions of total parameters. The cost that does grow is the KV cache. In “per loop” mode, each loop through the recurrent block keeps its own cache, so more loops require more cache memory at a given context length. NVIDIA also offers a “last loop” mode, where the cache from the most recent loop simply overwrites earlier ones, keeping memory flat across loop counts while reportedly keeping accuracy and output length close to the per-loop version. At a 32K context window, two-loop per-loop caching takes roughly 2GB per sequence, versus about 1.3GB for the last-loop version. That’s the kind of setting worth testing if you’re serving long-context workloads with many concurrent users. On output quality, more loops don’t behave like a simple “turn up reasoning” dial. In testing, four loops often produced more detailed, better-explained answers than one loop, but not reliably across every prompt. Internally, confidence in predicted tokens does increase with more loops. Looking at log probabilities for a specific predicted token, confidence rose from around 15% at one loop up to the 70% range by five or six loops, suggesting the extra passes genuinely sharpen the model’s internal certainty, even if that doesn’t always translate into a visibly better final answer. ## Is Dory practical to deploy today? Dory is explicitly a research release, and running it with variable loop counts currently has a rough edge: tools like vLLM need to be told the loop count at startup, which means serving multiple loop configurations requires running multiple separate instances rather than switching loop count dynamically per request. A sensible setup today looks like running a fast two-loop instance as the default (which appears to match NVIDIA’s own recommendation) and routing only harder, agentic, or long-context requests to a four-loop instance. A router making that decision automatically, rather than a human choosing per prompt, is the obvious next step, and something inference servers could eventually handle natively. NVIDIA has released Dory under its open model license, allowing commercial use, with bf16, MXFP8, and NVFP4 weight formats plus GGUF conversions for local inference. As a 4 billion parameter model, it’s small enough to run on consumer hardware, which is part of why it’s a useful test case even in its current research state. ## Frequently Asked Questions ### What does “looping” mean in a transformer model? It means running the same block of layers multiple times on the same token before passing the result to a final output layer, rather than running a sequence of layers that each have unique weights. The weights are reused on every pass; only the number of passes changes. **1,284** **37** **6** | Vendor | Tier | Risk | Approval NEW | |---|---|---|---| | **Northwind Supply** | Tier 1 | Low | Approved | | **Acme Logistics** | Tier 2 | Medium | Approve | | **Helios Data** | Tier 1 | High | Pending | | **Granite Freight** | Tier 3 | Low | Approved | | **Pinecrest Labs** | Tier 2 | Low | Approve | ### Does Dory actually beat Gemma 3 27B? On NVIDIA’s composite benchmark score, yes, particularly at higher loop counts. But the win is uneven: Dory beats Gemma 3 27B on agentic, instruction-following, and long-context tasks, while losing clearly on knowledge-heavy benchmarks like GPQA and Humanity’s Last Exam. ### Does more looping always improve output quality? No. More loops generally correlate with higher benchmark scores and higher internal token confidence, but for many individual prompts, outputs from different loop counts land on the same answer, and occasionally a lower loop count produces a cleaner result than a higher one. ### Why doesn’t Dory perform well on knowledge-based benchmarks? Looping adds compute per token but doesn’t add stored parameters. Knowledge-heavy tasks depend on how much information a model has memorized in its weights, and Dory’s roughly 4 billion total parameters simply can’t match a model with a much larger total weight pool, even if active compute per token is similar. ### Is Dory related to rumors about GPT-6 using loop transformers? There have been rumors that GPT-6 uses a looped architecture with different loop counts for different model sizes, but that hasn’t been confirmed. Dory is a separate, openly released NVIDIA research model that demonstrates the looped transformer approach in practice, regardless of what larger labs end up shipping.