{"slug": "mercury-3-does-1068-tokens-second-with-a-diffusion-llm-here-s-the-architecture", "title": "Mercury 3 Does 1,068 Tokens/Second With a Diffusion LLM — Here's the Architecture Trick (and How to Try It)", "summary": "Inception Labs released Mercury 3, a diffusion-based large language model with a 1M-token context window that the company reports runs at 1,068 tokens per second on NVIDIA GPUs and is priced at $0.20/$0.70 per million tokens, half price during launch. The model is available through an OpenAI-compatible API on OpenRouter, Baseten and Vercel, letting developers try it by changing a base URL. Unlike autoregressive decoding, which commits tokens serially, diffusion decoding denoises all positions in parallel over K refinement steps, a structure the vendor claims closes the coherence and instruction-following gap that earlier diffusion LMs such as Diffusion-LM, SEDD and LLaDA showed against autoregressive models; the benchmark figures are vendor-reported.", "body_md": "Today Inception Labs dropped **Mercury 3**: a diffusion-based LLM with a 1M-token context window doing **1,068 tokens/sec on NVIDIA GPUs**, claiming to beat GPT-6 Luna and Claude Haiku 5.5 on benchmarks — at $0.20/$0.70 per million tokens, half price during launch. It's live on OpenRouter, Baseten, Vercel, and behind an **OpenAI-compatible API**, so you can try it by changing a base URL.\n\nIf you're a builder, here's the part that matters: this is not \"a faster autoregressive model.\" It's a different generation paradigm, and it changes what your inference code can assume.\n\nAutoregressive decoding: `for i in range(n): token[i] = model(tokens[:i])`. Serial. Unfixable — once token 5 is emitted, it's baked in.\n\nDiffusion decoding: start with a fully masked/noisy sequence of length n, then iteratively denoise **all positions in parallel** over K refinement steps. Position 5 can be revised in step 12 based on what position 500 looks like. The model gets K parallel passes over the whole sequence instead of n serial commits.\n\nThat parallelism is where 1,068 tok/s comes from. On a GPU, parallel refinement across the sequence exploits the hardware's actual strength (wide vector math) instead of its weakness (serial dependency chains).\n\nThree workload classes where diffusion's architecture is *structurally* advantaged:\n\nThe flip side, from the research literature (Diffusion-LM, SEDD, LLaDA): open-ended long-form coherence and precise instruction-following have historically lagged autoregressive models. Mercury 3 claims to close the gap. Claims. Test it yourself.\n\n``` python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https://api.inception.ai/v1\",  # OpenAI-compatible endpoint\n    api_key=\"YOUR_INCEPTION_KEY\",\n)\n\nresp = client.chat.completions.create(\n    model=\"mercury-3\",\n    messages=[{\"role\": \"user\", \"content\": \"Refactor this function...\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\nWhat to actually measure — don't just vibe-check:\n\nOctober has been the month the LLM monoculture cracked: open weights counter-attacking (816B-param models last week), inference prices collapsing (DeepSeek's $0.50 agent yesterday), and now a non-autoregressive architecture shipping at frontier-adjacent quality. Even if Mercury 3's numbers turn out optimistic, the direction is set — and the builders who understand diffusion decoding now will have a head start when (not if) every lab ships one.\n\nTry it this week. Report back what breaks. That's how new architectures earn trust.\n\n*Source: Inception announcement via PANews, Oct 20, 2026. Benchmark figures are vendor-reported. Written by Shivam Kumar (VisionQuantech).*", "url": "https://wpnews.pro/news/mercury-3-does-1068-tokens-second-with-a-diffusion-llm-here-s-the-architecture", "canonical_source": "https://dev.to/shivaysinghrajput/mercury-3-does-1068-tokenssecond-with-a-diffusion-llm-heres-the-architecture-trick-and-how-to-45d", "published_at": "2026-10-20 04:35:01+00:00", "updated_at": "2026-10-20 04:47:15.215015+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-research", "generative-ai", "ai-infrastructure"], "entities": ["Inception Labs", "Mercury 3", "NVIDIA", "OpenRouter", "Baseten", "Vercel", "GPT-6 Luna", "Claude Haiku 5.5"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/mercury-3-does-1068-tokens-second-with-a-diffusion-llm-here-s-the-architecture", "markdown": "https://wpnews.pro/news/mercury-3-does-1068-tokens-second-with-a-diffusion-llm-here-s-the-architecture.md", "text": "https://wpnews.pro/news/mercury-3-does-1068-tokens-second-with-a-diffusion-llm-here-s-the-architecture.txt", "jsonld": "https://wpnews.pro/news/mercury-3-does-1068-tokens-second-with-a-diffusion-llm-here-s-the-architecture.jsonld"}}