# Mercury 3: Why Diffusion LLMs Could Change How We Build AI Agents > Source: > Published: 2026-10-21 02:35:39+00:00 When choosing an AI model, developers often focus on reasoning quality, coding benchmarks, and context length. But there is another factor that becomes increasingly important as AI applications move beyond chat: latency. A chatbot can sometimes get away with waiting a few seconds for an answer. An AI agent that searches the web, calls tools, reads documents, writes code, and repeats the process several times has a different problem. Every model call adds latency. Those delays accumulate. This is where Mercury 3, the latest diffusion-based language model from Inception Labs, becomes interesting. Rather than simply competing on traditional language-model benchmarks, Mercury 3 explores a different approach to generating text, with a strong emphasis on speed and latency-sensitive workloads. Let's look at what makes it different, where it could be useful, and what developers should test before adopting it. **1. What Makes Mercury 3 Different?** Most modern large language models use autoregressive generation. In simplified terms, the model predicts the next token based on the tokens that came before it, then continues generating the sequence step by step. This approach has become incredibly capable, but generating long responses can take time because the output is produced sequentially. Mercury 3 takes a different approach: diffusion-based language generation. Instead of committing to the output strictly one token at a time, diffusion language models iteratively generate and refine multiple tokens in parallel. A useful analogy is the difference between writing a document one word at a time and drafting larger portions before refining them through multiple passes. The analogy isn't a complete description of the underlying mathematics, but it captures the key idea: parallel refinement can change the relationship between output length and generation time. Inception Labs has explored this approach in its earlier Mercury research paper, which describes parallel token generation and its implications for inference efficiency. Mercury 3 brings this approach to a more capable model designed for real-world applications. **2. The Numbers Behind Mercury 3** According to Inception's official announcement, Mercury 3 offers: 1,068 tokens per second reported on NVIDIA GPUs. 1 million tokens of context. Support for agentic workflows and real-time applications. Reported performance improvements over several speed-optimized models on selected benchmarks. Inception also reports that Mercury 3 runs approximately 4–8 times faster than the comparison models used in its announcement. These numbers are promising, but they need context. The reported throughput is not a guarantee that every API request will return at that speed. Actual performance depends on the hardware, serving infrastructure, prompt length, output length, concurrency, and the way a particular workload is evaluated. Likewise, benchmark comparisons do not establish that Mercury 3 is better at every task. For developers, the more useful question is: Can Mercury 3 complete my particular task faster without sacrificing the quality I need? That is the question worth testing. **3. Why Speed Matters More for AI Agents** The benefits of faster inference become more obvious when we look at how agents work. Consider an agent that researches a topic across several websites. A simplified workflow might look like this: Understand the user's request. Decide which sources to investigate. Search the web and inspect the results. Summarize the findings. Identify missing information and search again. Produce a final answer. The model may be called several times throughout this process. Some steps depend on the results of earlier steps, so they cannot all run simultaneously. Even if each individual call is only slightly faster, reducing latency across repeated calls can improve the overall experience. This makes fast inference particularly interesting for: Web research and search agents. Coding agents that repeatedly inspect and modify code. Subagents responsible for narrow, well-defined tasks. Interactive applications where users expect immediate feedback. Voice interfaces where long pauses disrupt conversations. Inception describes these as important application areas for Mercury 3, and its launch announcement includes examples from web navigation, interactive education, real-time search, and customer support. The broader lesson is that model speed is not merely a benchmark number. It can influence how an entire application feels and how much work an agent can complete within a given time budget. **4. Where Mercury 3 Could Fit in a Coding Workflow** Coding agents are a particularly interesting use case because they often make repeated model calls. Imagine an agent working on an unfamiliar codebase. It might need to inspect several files, locate a bug, generate a patch, review the changes, run tests, and fix any failures. Not every step requires the same kind of intelligence. Some tasks require deep architectural reasoning. Others involve summarizing files, extracting relevant code, generating a small change, or processing tool output. A useful architecture could assign different roles to different models: Task Potential model role Understand the overall task Strong reasoning model Summarize files and tool output Fast model Generate a focused code change Fast coding-oriented model Review a complex architectural decision Validate the final result Tests and independent checks Mercury 3 is worth evaluating for the latency-sensitive parts of this workflow. That does not mean it should automatically replace the model responsible for the entire coding task. A fast subagent that produces unreliable output can create additional review work and erase its speed advantage. The goal is to identify where faster inference improves the complete workflow, not just one step. **5. Is a Faster Model Always Better?** Not necessarily. There are several trade-offs developers should consider before adopting a diffusion-based LLM. **Output speed versus end-to-end latency** Tokens per second measures generation throughput, not the entire request lifecycle. Network latency, queueing, prompt processing, tool execution, and time to first token can all affect the user's experience. For interactive applications, measure the time from submitting a request to receiving a useful result. **Speed versus task success** A model that generates an answer quickly but makes more mistakes may require additional calls, retries, or human intervention. For coding agents, measure whether the generated code passes tests. For research agents, check factual accuracy and source quality. For structured extraction, measure schema validity and field-level correctness. **Throughput versus cost per completed task** The lowest price per token does not necessarily produce the lowest cost for a successful outcome. A better comparison includes: Input and output token costs. Number of model calls. Retry frequency. Tool execution costs. Human review effort. Total time required to complete the task. This is especially important for agents, where a small difference in reliability can have a significant effect over many steps. **Long context versus useful context** A million-token context window is valuable when an application genuinely needs to process large amounts of information. However, a large context window does not automatically guarantee that a model will find every important detail or reason correctly across a long document. Test it against your own documents, repositories, and retrieval workflows. **6. How I Would Evaluate Mercury 3 in Practice** Rather than relying entirely on published benchmarks, I would run a small evaluation using representative production tasks. Start with three categories: **Test A: Short, latency-sensitive requests** Use search-result summarization, classification, or small coding subtasks. Measure time to first token, total response time, and output quality. **Test B: Multi-step agent workflows** Run the same research or coding task with Mercury 3 and another model. Keep the prompts, tools, task limits, and evaluation criteria consistent. Record total completion time, number of model calls, retries, and whether the final result meets the requirements. **Test C: Long-context tasks** Provide a large document set or codebase and ask the model to locate specific information, explain relationships, or produce a structured result. Evaluate both accuracy and latency. Do not treat the maximum context window as a substitute for a task-level evaluation. For a fair comparison, repeat the tests under similar conditions and report median latency as well as slower requests, such as the 95th percentile. The result should be a practical model-selection decision, not just a leaderboard ranking. **7. Why Model Choice Matters More as AI Applications Mature** The AI ecosystem has traditionally focused heavily on finding the strongest general-purpose model. But real applications have different requirements. A research agent may prioritize reasoning quality. A voice assistant may prioritize responsiveness. A coding agent may need a combination of fast execution, reliable tool use, and strong code generation. There is no reason to assume that one model must be the best choice for every component. This creates an opportunity for model routing: use different models for different tasks, based on their capabilities, latency, and cost. The challenge is making that choice practical for developers. Managing different providers, API formats, authentication methods, and billing systems adds operational complexity. It can also make it harder to compare models using the same application and workload. At [ApiHub](https://www.apihub.ink/), we've made Mercury 3 available alongside other AI models through a unified API. Developers can use the platform's free credits to experiment with different models and evaluate which ones work best for their applications. The point isn't to assume Mercury 3 is the right model for everything. It's to make it easier to test alternatives and choose based on real requirements. **Final Thoughts** Mercury 3 is interesting not simply because of its reported generation speed, but because it demonstrates another direction for building language models. Diffusion-based generation could be particularly useful in applications where model calls happen repeatedly and latency compounds, including search agents, coding subagents, and real-time interactive systems. There are still important questions to answer through independent evaluations and workload-specific testing. Speed alone does not guarantee reasoning quality, reliability, or lower total cost. But developers now have another architecture worth experimenting with. The most useful question is no longer just which model is smartest. It's which model can complete each part of your workflow reliably, quickly, and at the right cost. That's where model choice becomes an engineering decision rather than a leaderboard contest. **Further Reading** Introducing Mercury 3 — Inception Labs Mercury: Ultra-Fast Language Models Based on Diffusion — Research Paper Inception Releases Mercury 3, Its Flagship Diffusion LLM — Unite.AI Explore AI models on ApiHub