{"slug": "building-the-fastest-llms-why-we-re-starting-with-diffusion", "title": "Building the fastest LLMs: why we're starting with diffusion", "summary": "Celeris announced it is building large language models around diffusion rather than autoregressive decoding, aiming to maximize useful intelligence delivered per unit of time. The company cited an NVIDIA-led study, Fast-dLLM, showing up to a 27.6× throughput improvement on open diffusion models with minimal accuracy loss, and the 8B diffusion model LLaDA surpassing GPT-4o on reversal reasoning, as evidence for parallel decoding. Celeris said it is pursuing hybrid approaches that combine sequential and parallel generation in a single system and is investing in neural architecture search and automated research loops.", "body_md": "[← Back to blog](https://celeris.ai/blog)\n\n# Building the world's fastest LLMs: why we're starting with diffusion\n\nAt Celeris, our objective is to **maximize useful intelligence delivered per unit of time**. In this post, we explain how we're pursuing that goal - starting with diffusion - and what becomes possible when it's reached.\n\n## A different starting point\n\nToday the work is split: labs design and train models, and inference providers make them faster after the fact. No one is designing for speed from the ground up. That is what we are doing: new techniques for both training and architecture, built for extreme throughput from the start.\n\nOur current focus is diffusion.\n\nDiffusion models have already demonstrated remarkable parallel-generation properties in other domains. In language, the sequential nature of autoregressive decoding has historically dominated. We see an opportunity to invert that assumption. Our approach draws on the strengths of both paradigms - the coherence of sequential generation and the speed of parallel decoding - and pushes well beyond existing hybrid methods.\n\nThis is not simply \"making diffusion work for text.\" It is about creating training and inference paradigms in which **parallelism is a first-class citizen**.\n\nThe evidence for each of these properties is already public. On raw speed, parallelism gives diffusion decoding headroom that serial generation cannot reach: [Fast-dLLM](https://arxiv.org/abs/2505.22618), an NVIDIA-led study, showed that decoding many tokens per step delivers up to a 27.6× throughput improvement on open diffusion models with minimal accuracy loss - from training-free acceleration alone.\n\nBidirectional context also closes a significant capability gap. Autoregressive models suffer from the [reversal curse](https://arxiv.org/abs/2309.12288): trained that \"A is B\", they fail to infer \"B is A\" - GPT-4 answers 79% of forward questions about celebrity relationships but only 33% of the reversed ones. [LLaDA](https://arxiv.org/abs/2502.09992), an 8B diffusion model that attends to the whole sequence at every step, breaks the pattern, surpassing GPT-4o on reversal reasoning.\n\nAnd refinement changes what inference itself can do. An autoregressive model commits to every token permanently; a diffusion model can revise earlier predictions as the rest of the answer takes shape. [Diffusion-of-Thought](https://arxiv.org/abs/2402.07754) (NeurIPS 2024) showed a small diffusion model self-correcting its reasoning mid-generation and beating a much larger autoregressive model on math tasks in both accuracy and efficiency, with the number of refinement steps acting as a natural dial between latency and reasoning quality.\n\n## What's unique about our approach\n\nWe are working on **hybrid approaches to diffusion** - models that combine sequential and parallel generation inside a single system, spending sequential compute only where language demands it and decoding the rest in parallel. Building this into the architecture and the training objective, rather than bolting one paradigm onto the other at inference time, is the core of our research agenda - and it is what lets a model keep the coherence of autoregression while decoding at diffusion speeds.\n\n## Architecture search as a core capability\n\nHand-designing these systems is insufficient at the frontier. We are investing heavily in neural architecture search and automated research loops that let us systematically explore the space of architectures, training objectives, and inference strategies optimized for latency and intelligence density.\n\nThe goal is not just a single fast model. It is **a research engine** capable of continually discovering better ways to deliver frontier-level capability at microsecond-to-millisecond timescales.\n\n## What this enables\n\nWhen intelligence arrives at true real-time speeds, entire categories of software become possible that are awkward or impossible today:\n\n- Fluid, interruptible voice interfaces that feel instantaneous\n- Agents that reason and act in tight feedback loops\n- Scientific and engineering tools that keep pace with human thought\n- Systems where latency no longer forces a compromise on model quality\n\nWe are still early. Our first models are coming soon. What we can say now is that we are deliberately choosing a harder path - rethinking both training and architecture rather than only optimizing the status quo - because we believe the payoff in capability-per-second will be decisive.\n\nIf you care about the systems that become possible when language models stop being the bottleneck, [we'd love to hear from you](https://celeris.ai/contact).\n\n[Get started](https://console.celeris.ai/)", "url": "https://wpnews.pro/news/building-the-fastest-llms-why-we-re-starting-with-diffusion", "canonical_source": "https://celeris.ai/building-the-worlds-fastest-llms", "published_at": "2026-09-19 10:16:47+00:00", "updated_at": "2026-09-19 10:24:58.269829+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "generative-ai", "ai-infrastructure"], "entities": ["Celeris", "NVIDIA", "Fast-dLLM", "LLaDA", "GPT-4o", "Diffusion-of-Thought", "NeurIPS 2024"], "alternates": {"html": "https://wpnews.pro/news/building-the-fastest-llms-why-we-re-starting-with-diffusion", "markdown": "https://wpnews.pro/news/building-the-fastest-llms-why-we-re-starting-with-diffusion.md", "text": "https://wpnews.pro/news/building-the-fastest-llms-why-we-re-starting-with-diffusion.txt", "jsonld": "https://wpnews.pro/news/building-the-fastest-llms-why-we-re-starting-with-diffusion.jsonld"}}