# What Are Next Concept Models? Stanford's Coarse-to-Fine Text AI > Source: > Published: 2026-10-29 00:00:00+00:00 # What Are Next Concept Models? Stanford's Coarse-to-Fine Text AI Stanford's Next Concept Model writes text in rough-to-fine passes instead of word by word. Here's how NCM works, and where its speed claims hold up. ## What is a Next Concept Model? A Next Concept Model (NCM) is a text generation architecture from Stanford that writes an entire answer in a small number of coarse-to-fine passes instead of producing one word at a time. Rather than predicting the next token like GPT-style models, NCM predicts “concepts,” which are learned compression codes that represent a blurry outline of the whole response. Each pass adds more detail until the final pass produces readable text. It is not a diffusion model, though it borrows the same general intuition: sketch first, refine later. ## TL;DR - **NCM generates text in levels** , starting with a tiny, blurry outline of the entire response and adding detail layer by layer until the final pass produces finished text. - **Concepts are not human ideas** , they are codes from a learned vocabulary of about 4,000 entries that a compressor model uses to represent chunks of text at different levels of detail. - **The architecture splits into two parts** : a compressor that turns text into stacked levels of codes, and a generator (a standard transformer) that predicts an entire level of codes in one pass before moving to the next. - **The paper’s headline speed numbers compare NCM only to diffusion models** , not to ordinary autoregressive LLMs, which leaves an open question about real-world speed against the models people actually run. - **Independent testing found NCM is about four times faster than a same-size autoregressive model for single requests** , but slower once many requests are batched together, which matters for server and agentic workloads. - **The widely cited 86% Sudoku-solving result appears to come largely from fill order rather than the NCM architecture itself** , since a comparably sized diffusion model reached similar results once it filled in its most confident cells first. - **NCM has real limitations today** : fixed-size output regardless of answer length, occasional errors with names and numbers, no public training code, and no testing yet at the model sizes people run locally (tens or hundreds of billions of parameters). ## Why are normal language models slow in the first place? Every mainstream chatbot today, including GPT and Gemini-class models, is autoregressive. It writes the first token, appends it to the growing text, then predicts the next one based on everything written so far. A thousand-token answer takes a thousand sequential steps. The bottleneck isn’t the math, it’s memory bandwidth. On every one of those steps, the GPU has to pull the model’s entire set of weights out of memory just to produce a single additional word. For a local model, writing a long response means moving enormous amounts of data through the GPU repeatedly. That’s why a local model can feel slower than a person reading, and why speed on local hardware tracks memory bandwidth more than raw compute power. Diffusion language models (like Mercury from Inception or Google’s diffusion-based text models) tried to fix this by generating an entire page at once and refining it over many passes. The problem: every pass touches every position in the text, and getting clean output usually takes dozens of passes. Cut the passes to save time and quality drops fast. ## How does NCM actually work? NCM borrows an idea from image generation, where some models draw a blurry version of the full picture first and sharpen it over successive steps, so the model never has to commit to fine detail before establishing the overall structure. NCM applies that same logic to text. The architecture has two pieces: **The compressor** learns to store a passage of text as a stack of levels made of codes. The top level holds just a handful of codes representing a blurry outline of the entire passage. Each level below adds more detail, and the bottom level has one code per word. Stack all the levels together and the compressor can reconstruct the original text almost exactly. These codes, drawn from a learned vocabulary of roughly 4,000 entries, are what the paper calls “concepts.” They aren’t semantic ideas in any human sense, just compression units. **The generator** is an ordinary transformer, architecturally similar to what powers today’s LLMs. The difference is in what it predicts: instead of the next single word, it predicts every code in an entire level simultaneously, then moves down to the next level. A model with nine levels takes nine total passes to produce a complete answer, regardless of whether that answer is a sentence or a paragraph. This is the core distinction from diffusion models. A diffusion model redraws every word position on every pass. NCM’s early passes only touch a handful of top-level codes, making them cheap, and each subsequent pass adds only the detail that level is responsible for. ## Is NCM actually faster than normal LLMs? This is the question the original paper leaves unanswered, since every speed comparison in it measures NCM against diffusion models, not against standard autoregressive models of the same size. ## Remy doesn't write the code. It manages the agents who do. Remy runs the project. The specialists do the work. You work with the PM, not the implementers. Independent testing (using the authors’ released code and trained models) found a more nuanced picture. Sent one request at a time, NCM was roughly four times faster than a matched autoregressive model on translation, and close to two times faster on news summarization. That’s a meaningful win for anyone running a single model locally, since a handful of passes beats hundreds or thousands of sequential token predictions. But send 64 requests at once, simulating a real server under load, and the result flips: the autoregressive model comes out ahead. The reason traces back to the same memory-versus-math tradeoff that makes normal LLMs slow in the first place. With one request, the GPU spends most of its time waiting on memory, so doing only nine or eleven passes is a big advantage. With many requests batched together, the GPU loads weights once and spends its time on computation, and NCM ends up doing more total computation because it processes roughly four times more positions than the number of words it ultimately produces. The released code also reprocesses the entire prompt on every single pass, which hurts performance further on longer inputs like full news articles. The practical implication: NCM looks promising for single-user, local inference, but it is not yet a clear win for the batched, multi-agent, server-style workloads that production AI systems increasingly rely on. ## Does NCM really solve Sudoku better than normal LLMs? The paper’s Sudoku result is one of its most eye-catching claims: a standard autoregressive model of a given size solved only a small fraction of hard Sudoku boards, because once it commits to a wrong digit early on, it can’t go back and fix it. A much smaller NCM model solved the large majority of the same boards. Testing this directly raised a real question about where that gap actually comes from. A diffusion model of matching size, trained on the same puzzles and tested on the same boards, solved only about a quarter of them when filling in cells in random order, consistent with the paper. But changing a single rule at inference time, having the model fill in the cell it’s most confident about first rather than a random one, pushed performance up to a level close to NCM’s reported score. Allowing more refinement passes pushed the diffusion model’s score even higher. That suggests the dramatic Sudoku gap in the paper may have more to do with the order cells get filled in than with anything unique to the NCM architecture itself. NCM does get this coarse-to-fine, confidence-ordered behavior built in by default, while a diffusion model needs an extra decoding rule to get similar behavior. That’s a genuine structural advantage, just a narrower one than “NCM solves logic puzzles LLMs can’t.” ## What are NCM’s current limitations? Several constraints matter for anyone considering the architecture beyond research curiosity: - **Fixed-size output.** Every NCM model writes into a fixed-size structure, so even a one-word answer runs through every level and pays the full computational cost. - **Errors with names and numbers.** The compressor sometimes swaps specific details when converting text to codes and back. - **Unproven at scale.** The largest published model is under 1 billion parameters, far smaller than the 27 billion or larger models people commonly run. Whether the speed and quality advantages hold at those sizes is untested. - **No streaming.** Because NCM redraws the entire answer on every pass rather than filling in positions left to right, nothing is readable until the final pass completes, unlike token-by-token models that can stream output as it’s generated. - **Incomplete tooling.** The released code supports evaluation only, with no training code, and the prompt-caching feature doesn’t work reliably yet. - **Mixed quality results.** On actual text tasks like translation and open-ended writing, a standard autoregressive model of the same size still produced better-quality output in the paper’s own tables. ## One coffee. One working app. You bring the idea. Remy manages the project. ## Frequently Asked Questions ### Is NCM a type of diffusion model? No. NCM and diffusion models both generate text in parallel passes rather than word by word, but they work differently. Diffusion models redraw every token position on every pass. NCM predicts coarse-to-fine levels of learned “concept” codes, with early passes touching only a handful of codes and later passes adding detail, which makes its early steps much cheaper. ### What does “concept” mean in Next Concept Models? A concept is not a human idea or topic. It’s a code from a fixed, learned vocabulary (roughly 4,000 entries) that a compressor model uses to represent text at a given level of detail, from a blurry whole-passage summary down to one code per word. ### Is NCM faster than GPT-style models? In limited testing, NCM was roughly four times faster than a matched-size autoregressive model for single requests sent one at a time. But when many requests are batched together, as a real server would do, standard autoregressive models performed better, because NCM ends up doing more total computation per answer. ### Does NCM really outperform LLMs on Sudoku? The reported gap is real in the sense that it reproduces, but independent testing suggests it may come largely from the order in which cells get filled in rather than something unique to the NCM architecture. A diffusion model given a “fill the most confident cell first” rule reached similar performance. ### Can I run NCM models today? The researchers released evaluation code and trained models, so it’s possible to experiment with existing checkpoints. There’s no public training code yet, some caching features are unreliable, and the largest published model is well under 1 billion parameters, so it hasn’t been tested at the scale most people associate with capable local models.