Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

A paper submitted to arXiv on 9 October 2026 introduces Next-Concept Modeling (NCM), a two-stage coarse-to-fine text generation framework that pairs a multiscale residual VQ-VAE with a transformer that predicts all codes within a level in parallel. At matched parameter scales, NCM beats the strongest diffusion language model baseline on generative perplexity on LM1B while running 21x faster, is competitive on OpenWebText at 10x faster, matches the strongest DLM's ROUGE-L on XSum at 31x faster, and outperforms autoregressive and diffusion baselines on Sudoku reasoning across all difficulty levels with fewer function evaluations. The result matters because diffusion language models degrade with fewer sampling steps and process the full sequence at every step, and NCM's parallel per-level prediction cuts inference cost as inference becomes a dominant computational workload.

Computer Science Computation and Language Submitted on 9 Oct 2026 Title:Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation View PDF https://arxiv.org/pdf/2610.14162 HTML experimental https://arxiv.org/html/2610.14162v1 Abstract:Diffusion language models DLMs offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant computational workload. However, DLMs degrade quickly with fewer sampling steps, process the full sequence at every step, and require many passes over the dataset to reach good quality. We introduce Next-Concept Modeling NCM , a two-stage, coarse-to-fine framework: a multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes, and a transformer generates one level at a time, predicting all codes within a level in parallel. At matched parameter scales, NCM achieves better generative perplexity than the strongest DLM baseline on LM1B while running $21\times$ faster. On OpenWebText, NCM is competitive with the strongest DLM while running $10\times$ faster; on XSum, it matches the strongest DLM's ROUGE-L while being $31\times$ faster. On the reasoning task of Sudoku, NCM outperforms autoregressive and diffusion baselines across all difficulty levels, using fewer function evaluations. On the reasoning task of Sudoku, NCM outperforms autoregressive and diffusion baselines across all difficulty levels, using fewer function evaluations. Additional Features Current browse context: cs.CL References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .