# Microsoft ThinkingBox Exposes the AI Agent Reliability Gap > Source: > Published: 2026-09-03 01:10:27+00:00 Your AI agent confirmed the booking, resolved the ticket, and updated the record. You watched the transcript. Everything looked fine. There was just one problem: it may have done none of that. Microsoft’s new open-source benchmark, **ThinkingBox**, just put hard numbers on exactly how often agents that look successful actually aren’t — and those numbers should make you rethink every agent you have shipped to production. ## One Success Isn’t Reliability ThinkingBox-Bench ran 507 stateful business tasks through 12 AI models, 20 times each. The results expose what researchers are calling the **discovery-reliability gap**. The best-performing model succeeded on 65.36% of single attempts. Sounds acceptable. Then look at the other number: that same model succeeded on *all* 20 consecutive runs of the same task only 25.25% of the time. That nearly 40-point spread is the gap between “can do it” and “always does it.” In production, you need the second number. Demos give you the first. The benchmark covers five business domains — retail, hospitality, auto insurance, neobank IT, and consulting IT/HR support — selected precisely because they require agents to gather missing information across multiple turns, follow domain policies, coordinate dependent tools, and leave the correct persistent state behind. Real work, not synthetic puzzles. ## Why Transcripts Lie The more troubling finding is how these failures look. Of all failed trials in ThinkingBox-Bench, 80.88% ended cleanly — no crash, no exception thrown, no error logged. The agent calmly reported completing the task. It just hadn’t actually done it, or had done part of it and stopped. This is the exact failure mode that every traditional eval misses. Most agent benchmarks grade what the model *says* it did. ThinkingBox checks what it *actually changed in the database*. That distinction is everything. An agent that confidently confirms a support ticket is resolved, while leaving the ticket open in the backend, passes a transcript check and fails a state check. Only one of those matters in production. ## What ThinkingBox Actually Does The framework runs agents inside isolated MCP-compatible tool sessions. Each session starts with a known initial state, has a simulated user available to answer follow-up questions, and ends with automated assertions that verify the actual backend state against the expected outcome. Nothing is inferred from what the agent said — only what changed. The benchmark package and execution harness are decoupled, so you can update your agent implementation without invalidating past benchmark runs. That matters if you plan to use this iteratively, which you should. ThinkingBox ships with a bundled `cloud_drive` scenario so you can verify your setup without downloading the full benchmark dataset first: ``` uv run tb mcp-start # start the MCP session proxy uv run tb infer # run inference against tasks uv run tb pp # pretty-print results ``` The full benchmark dataset lives in a separate repository and covers all five business domains. Tools are defined as MCP servers, which means the same definitions work across Claude, Copilot, and any other MCP-compatible agent platform. ## The Production Reality ThinkingBox did not discover a new problem — it finally gave the problem a precise measurement. Independent research already shows that [between 70% and 95% of AI agents fail in production](https://www.fiddler.ai/blog/ai-agent-failure-rate), and that roughly 88% of enterprise agents that work in controlled demos break in real workflows. Gartner projects that fewer than one in three enterprise agents will meet their stated reliability targets in their first 12 months. Enterprises with full automated eval coverage saw a 9% production rollback rate. Those without evals: 47%. A five-fold difference, almost entirely attributable to catching failures before they reach users. The industry has been measuring the wrong thing. Single-run success rates, demo transcripts, and human preference scores all measure “can it do this once, in a controlled setting.” [Pass^k](https://www.philschmid.de/agents-pass-at-k-pass-power-k) — the probability that an agent succeeds on every one of k independent trials — is what actually corresponds to production reliability. ThinkingBox is the first practically deployable tool built around that metric for stateful business workflows. ## Run It Before You Ship ThinkingBox is MIT-licensed and available now at [github.com/microsoft/thinkingbox](https://github.com/microsoft/thinkingbox). The full methodology is in the [arXiv paper](https://arxiv.org/abs/2608.19741), and the original announcement is on [Microsoft’s Command Line blog](https://commandline.microsoft.com/thinkingbox-bench-agent-benchmarking/). If you are building agents for production — not for demos, not for internal tooling a human double-checks, but for real automated workflows — run ThinkingBox before you ship. Not because 65% isn’t impressive. Because 25% is what you’re actually promising your users.