Dory's Loop Transformer: How NVIDIA's 4B Model Beats Gemma 3 27B NVIDIA's Nemotron Labs released Dory, a 3.97 billion parameter looped transformer that reuses a 40-layer recurrent block up to eight times per token and beats Gemma 3 27B on a composite benchmark score despite far fewer parameters in GPU memory. Dory's benchmark win rate against Gemma 3 27B rose from 1 of 10 benchmarks at one loop to 4 of 10 at two loops and 5 of 10 at four loops, with wins concentrated on agentic tasks such as instruction following, long context and SWE-bench-verified-like tasks, while it loses badly on knowledge-heavy benchmarks including GPQA and Humanity's Last Exam. NVIDIA pretrained Dory on 6 trillion tokens, the first 4.5 trillion natively in NVFP4 before switching to 16-bit, then added 300 billion tokens of context extension plus SFT and GRPO reinforcement learning, and the loop count is set at inference time so one checkpoint runs at 1, 4 or 8 loops without retraining. Dory's Loop Transformer: How NVIDIA's 4B Model Beats Gemma 3 27B NVIDIA's Dory loops a 4B model's weights up to eight times per token, beating Gemma 3 27B on agentic benchmarks. Here's what the data shows. What is Dory, and why does it matter? Dory is a 3.97 billion parameter research model from NVIDIA’s Nemotron Labs that reuses the same set of weights multiple times per token instead of relying on a bigger stack of unique layers. NVIDIA calls it a looped transformer, also referred to as a latent recurrent transformer. On its own composite benchmark score, Dory outperforms Gemma 3 27B, a much larger mixture-of-experts model, despite having a fraction of the parameters sitting in GPU memory. The interesting part isn’t that it wins. It’s how it wins, and where it loses, because that split reveals a lot about what looping can and can’t do for model quality. TL;DR - Dory splits its 72-layer structure into three blocks : a 16-layer input block, a 16-layer output block, and a 40-layer recurrent block that loops. - The number of loops is set at inference time , not baked in during training, so the same checkpoint can run as a fast 1-loop model or a slower, deeper 4-loop or 8-loop model. - More loops consistently improved benchmark win rate against Gemma 3 27B , going from 1 out of 10 benchmarks at one loop, to 4 out of 10 at two loops, to 5 out of 10 at four loops. - Dory wins on agentic tasks like instruction following, long context, and tasks resembling SWE-bench verified , but loses badly on knowledge-heavy benchmarks like GPQA and Humanity’s Last Exam. - The memory footprint of the weights never changes with more loops still 3.9 billion parameters on the GPU , though the KV cache grows unless you switch to a “last loop” caching mode. - NVIDIA pretrained Dory on 6 trillion tokens , with the first 4.5 trillion done natively in NVFP4 before switching to 16-bit, then added 300 billion tokens of context extension plus SFT and GRPO reinforcement learning. - This approach scales compute per token rather than output length , which is a different lever than the usual trick of just generating longer chains of thought. Auditable. Scalable. How does the loop mechanism actually work? A standard transformer runs every layer exactly once, start to finish. Dory instead treats its middle layers as a single reusable block and runs that block repeatedly before handing off to a fixed output block. The architecture breaks down into three parts: an input block 16 layers that runs once, a recurrent block 40 layers that loops a configurable number of times, and an output block 16 layers that also runs once. Each pass through the recurrent block feeds back into the original input representation, similar in spirit to how ResNet architectures add the original input back into each transformation step. The difference is that instead of each block having its own distinct weights, Dory reuses the exact same 40 layers of weights on every pass. Conceptually, each loop re-exposes the model to the original token representation, letting it refine its internal state rather than drift off into something unrelated. Unrolled, a 4-loop pass through Dory behaves like a model with roughly 192 effective layers of depth, even though only 40 of those layers are unique. At one loop, effective depth is about 72 layers. At two loops, around 112. The compute cost scales with the depth, but the parameter count parked in GPU memory stays fixed at just under 4 billion. NVIDIA describes this as “elastic inference compute”: you can dial compute up or down at inference time without touching how much memory the model occupies. Crucially, Dory’s output block was trained to read the state coming out of any loop count. That means one loop, two loops, or eight loops are all valid, already-trained configurations. There’s no need to retrain or fine-tune for a specific number of loops. What do the benchmark numbers against Gemma 3 27B actually show? NVIDIA compared Dory’s looped configurations against Gemma 3 27B a mixture-of-experts model with about 4 billion active parameters per token, so compute-per-token is roughly comparable even though total stored parameters differ enormously . The results scale directly with loop count: - At one loop , Dory wins 1 out of 10 benchmarks. - At two loops , Dory wins 4 out of 10 benchmarks. - At four loops , Dory wins 5 out of 10 benchmarks. The pattern in which benchmarks it wins is the real story. Dory performs well on agentic and instruction-following tasks, long-context handling, and benchmarks resembling SWE-bench verified, the kinds of tasks that reward careful, iterative reasoning about a given context. It performs poorly on knowledge-heavy benchmarks like GPQA and Humanity’s Last Exam, tasks that depend on the model having memorized a large volume of facts rather than reasoning more carefully about what’s already in front of it. demands of enterprises. Passed ✓ SOC 2 Type 2 ✓ HIPAA ✓ CAIQ attestation ✓ OCSF logging ✓ Role-based access ✓ Self-hosted That split makes sense given the architecture. Looping the same 40 layers more times gives the model more computation to refine its understanding of the current input, but it doesn’t add any new stored knowledge. Gemma 3 27B’s mixture-of-experts design keeps a similar active compute budget per token but has a much larger pool of total weights its experts to draw on, which functions as a bigger knowledge reservoir. Dory, at under 4 billion total parameters, simply doesn’t have anywhere near that much room to store facts, so it loses decisively on exam-style, trivia-heavy benchmarks no matter how many times it loops. What are the tradeoffs of looping instead of scaling up? The most obvious win is memory. Running Dory at four or eight loops costs more compute and more latency, but the weight footprint on the GPU never grows past roughly 4 billion parameters. That’s a meaningfully different cost curve than training or deploying a dense or mixture-of-experts model with tens of billions of total parameters. The cost that does grow is the KV cache. In “per loop” mode, each loop through the recurrent block keeps its own cache, so more loops require more cache memory at a given context length. NVIDIA also offers a “last loop” mode, where the cache from the most recent loop simply overwrites earlier ones, keeping memory flat across loop counts while reportedly keeping accuracy and output length close to the per-loop version. At a 32K context window, two-loop per-loop caching takes roughly 2GB per sequence, versus about 1.3GB for the last-loop version. That’s the kind of setting worth testing if you’re serving long-context workloads with many concurrent users. On output quality, more loops don’t behave like a simple “turn up reasoning” dial. In testing, four loops often produced more detailed, better-explained answers than one loop, but not reliably across every prompt. Internally, confidence in predicted tokens does increase with more loops. Looking at log probabilities for a specific predicted token, confidence rose from around 15% at one loop up to the 70% range by five or six loops, suggesting the extra passes genuinely sharpen the model’s internal certainty, even if that doesn’t always translate into a visibly better final answer. Is Dory practical to deploy today? Dory is explicitly a research release, and running it with variable loop counts currently has a rough edge: tools like vLLM need to be told the loop count at startup, which means serving multiple loop configurations requires running multiple separate instances rather than switching loop count dynamically per request. A sensible setup today looks like running a fast two-loop instance as the default which appears to match NVIDIA’s own recommendation and routing only harder, agentic, or long-context requests to a four-loop instance. A router making that decision automatically, rather than a human choosing per prompt, is the obvious next step, and something inference servers could eventually handle natively. NVIDIA has released Dory under its open model license, allowing commercial use, with bf16, MXFP8, and NVFP4 weight formats plus GGUF conversions for local inference. As a 4 billion parameter model, it’s small enough to run on consumer hardware, which is part of why it’s a useful test case even in its current research state. Frequently Asked Questions What does “looping” mean in a transformer model? It means running the same block of layers multiple times on the same token before passing the result to a final output layer, rather than running a sequence of layers that each have unique weights. The weights are reused on every pass; only the number of passes changes. 1,284 37 6 | Vendor | Tier | Risk | Approval