Mercury 3 Does 1,068 Tokens/Second With a Diffusion LLM — Here's the Architecture Trick (and How to Try It)

Inception Labs released Mercury 3, a diffusion-based large language model with a 1M-token context window that the company reports runs at 1,068 tokens per second on NVIDIA GPUs and is priced at $0.20/$0.70 per million tokens, half price during launch. The model is available through an OpenAI-compatible API on OpenRouter, Baseten and Vercel, letting developers try it by changing a base URL. Unlike autoregressive decoding, which commits tokens serially, diffusion decoding denoises all positions in parallel over K refinement steps, a structure the vendor claims closes the coherence and instruction-following gap that earlier diffusion LMs such as Diffusion-LM, SEDD and LLaDA showed against autoregressive models; the benchmark figures are vendor-reported.

Today Inception Labs dropped Mercury 3 : a diffusion-based LLM with a 1M-token context window doing 1,068 tokens/sec on NVIDIA GPUs , claiming to beat GPT-6 Luna and Claude Haiku 5.5 on benchmarks — at $0.20/$0.70 per million tokens, half price during launch. It's live on OpenRouter, Baseten, Vercel, and behind an OpenAI-compatible API , so you can try it by changing a base URL. If you're a builder, here's the part that matters: this is not "a faster autoregressive model." It's a different generation paradigm, and it changes what your inference code can assume. Autoregressive decoding: for i in range n : token i = model tokens :i . Serial. Unfixable — once token 5 is emitted, it's baked in. Diffusion decoding: start with a fully masked/noisy sequence of length n, then iteratively denoise all positions in parallel over K refinement steps. Position 5 can be revised in step 12 based on what position 500 looks like. The model gets K parallel passes over the whole sequence instead of n serial commits. That parallelism is where 1,068 tok/s comes from. On a GPU, parallel refinement across the sequence exploits the hardware's actual strength wide vector math instead of its weakness serial dependency chains . Three workload classes where diffusion's architecture is structurally advantaged: The flip side, from the research literature Diffusion-LM, SEDD, LLaDA : open-ended long-form coherence and precise instruction-following have historically lagged autoregressive models. Mercury 3 claims to close the gap. Claims. Test it yourself. python from openai import OpenAI client = OpenAI base url="https://api.inception.ai/v1", OpenAI-compatible endpoint api key="YOUR INCEPTION KEY", resp = client.chat.completions.create model="mercury-3", messages= {"role": "user", "content": "Refactor this function..."} , print resp.choices 0 .message.content What to actually measure — don't just vibe-check: October has been the month the LLM monoculture cracked: open weights counter-attacking 816B-param models last week , inference prices collapsing DeepSeek's $0.50 agent yesterday , and now a non-autoregressive architecture shipping at frontier-adjacent quality. Even if Mercury 3's numbers turn out optimistic, the direction is set — and the builders who understand diffusion decoding now will have a head start when not if every lab ships one. Try it this week. Report back what breaks. That's how new architectures earn trust. Source: Inception announcement via PANews, Oct 20, 2026. Benchmark figures are vendor-reported. Written by Shivam Kumar VisionQuantech .