# Mercury 3 Does 1,068 Tokens/Second With a Diffusion LLM — Here's the Architecture Trick (and How to Try It)

> Source: <https://dev.to/shivaysinghrajput/mercury-3-does-1068-tokenssecond-with-a-diffusion-llm-heres-the-architecture-trick-and-how-to-45d>
> Published: 2026-10-20 04:35:01+00:00

Today Inception Labs dropped **Mercury 3**: a diffusion-based LLM with a 1M-token context window doing **1,068 tokens/sec on NVIDIA GPUs**, claiming to beat GPT-6 Luna and Claude Haiku 5.5 on benchmarks — at $0.20/$0.70 per million tokens, half price during launch. It's live on OpenRouter, Baseten, Vercel, and behind an **OpenAI-compatible API**, so you can try it by changing a base URL.

If you're a builder, here's the part that matters: this is not "a faster autoregressive model." It's a different generation paradigm, and it changes what your inference code can assume.

Autoregressive decoding: `for i in range(n): token[i] = model(tokens[:i])`. Serial. Unfixable — once token 5 is emitted, it's baked in.

Diffusion decoding: start with a fully masked/noisy sequence of length n, then iteratively denoise **all positions in parallel** over K refinement steps. Position 5 can be revised in step 12 based on what position 500 looks like. The model gets K parallel passes over the whole sequence instead of n serial commits.

That parallelism is where 1,068 tok/s comes from. On a GPU, parallel refinement across the sequence exploits the hardware's actual strength (wide vector math) instead of its weakness (serial dependency chains).

Three workload classes where diffusion's architecture is *structurally* advantaged:

The flip side, from the research literature (Diffusion-LM, SEDD, LLaDA): open-ended long-form coherence and precise instruction-following have historically lagged autoregressive models. Mercury 3 claims to close the gap. Claims. Test it yourself.

``` python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inception.ai/v1",  # OpenAI-compatible endpoint
    api_key="YOUR_INCEPTION_KEY",
)

resp = client.chat.completions.create(
    model="mercury-3",
    messages=[{"role": "user", "content": "Refactor this function..."}],
)
print(resp.choices[0].message.content)
```

What to actually measure — don't just vibe-check:

October has been the month the LLM monoculture cracked: open weights counter-attacking (816B-param models last week), inference prices collapsing (DeepSeek's $0.50 agent yesterday), and now a non-autoregressive architecture shipping at frontier-adjacent quality. Even if Mercury 3's numbers turn out optimistic, the direction is set — and the builders who understand diffusion decoding now will have a head start when (not if) every lab ships one.

Try it this week. Report back what breaks. That's how new architectures earn trust.

*Source: Inception announcement via PANews, Oct 20, 2026. Benchmark figures are vendor-reported. Written by Shivam Kumar (VisionQuantech).*