We tend to blame compute limits for bottlenecked AI agents. But I wanted to test a different theory — what if the real friction is simply that autoregressive models force our data into a single-file line?
To see how this architectural difference actually plays out in practice, I built a rigorous cache-busting benchmarking suite and ran a series of benchmarks comparing the emerging Inception Mercury-2 (a Diffusion LLM) alongside a respected industry standard, Google Gemini 3.6 Flash (an Autoregressive LLM).
The results didn’t just show an incremental upgrade. They proved that the math on agentic workflows has fundamentally changed.
To understand the latency measurements in this suite, one must be aware of the core architectural differences driving the models.
LLM models like Gemini 3.6 Flash, GPT-5 and Claude 4 operate on a traditional Autoregressive Decoder Only Transformer architecture.
Models like Inception’s Mercury-2 introduce a radically different paradigm — Diffusion Language Modeling. Drawing inspiration from continuous diffusion models (like Stable Diffusion for images), Diffusion Language Models treat text generation as an iterative denoising process rather than a sequential prediction.
Autoregressive LLMsgeneratetokenssequentially from left to right, appending each predicted token to the growing context until EOS.Diffusion LLMsbegin withmasked tokensand iteratively refine multiple positions toward a complete sequence, illustratingparallel sequence refinement rather than strictly token-by-token generation.
To understand how these architectures truly perform, we have to look under the hood at the “engine speed.” For this benchmark, I tracked three core metrics: Time-To-First-Token (TTFT), End-to-End Latency (E2E), and Tokens-Per-Second (TPS).
However, measuring speed requires context.
To test this, the benchmark suite evaluated both models across five distinct scenarios — Real-Time Chat, Structured JSON Extraction, Agentic Tool Calling, Long Context Retrieval, and Complex Logic (Chain-of-Thought). Instead of testing models on static, memorizable datasets, this suite dynamically generates multi-hop logic puzzles, massive 500K-token scavenger hunts, and rigid JSON extraction schemas in real-time.
To ensure the testing methodologies align with standard enterprise AI evaluations, the suite draws direct inspiration from the following industry baselines:
Here is the data that should make every AI leaders rethink their stack:
Why is it so fast?
In blocking mode (like JSON extraction or Tool Calling), an autoregressive model makes you wait for sequential chunks. With Inception’s parallel diffusion, the first token arrives at the exact same millisecond as the last token. The TTFT is physically 0.00s.
Does this benchmark render the autoregressive architecture not good vs diffusion modelling architecture? No. It simply redefines each of these role in the modern AI stack.
When a problem requires deep, step-by-step logical deduction, the forced left-to-right sequential generation is actually a feature, not a bug.
However, for structural and agentic tasks, the diffusion architecture dominates. Here is the full breakdown of the engine speed tests:
Where Inception (Mercury-2) Wins For low-latency API pipelines, streaming user experiences, and rigid extractions, parallel diffusion removes the sequential bottleneck.
Where Gemini (3.6 Flash) Wins In tasks requiring brute-force processing over massive text blocks and thousands of hidden “thinking” tokens, the autoregressive transformer still holds the edge.
Across massive varied benchmarked tokens in the experiments, the diffusion architecture was 83% cheaper to run.For specific benchmarking scenarios, Mercury-2 cost a total of $0.03920, while Gemini 3.6 Flash cost $0.23655 for the exact same workloads.
We need to stop burning cash and latency on 1-million-token autoregressive context windows for everyday routing, formatting, and extraction.
The benchmarking data suggests a shift toward highly modular, high-speed architectures. Instead of forcing a single massive model to act as a slow, monolithic brain, this points to a few emerging patterns for production AI:
By playing to the strengths of parallel generation, we can build systems that are fundamentally faster and cheaper
Other Recommended Articles
I Benchmarked Inception vs. Gemini: The Math on Agentic Workflows Just Changed Forever was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.