Microsoft and Hugging Face released ThinkingBox on October 3rd, a benchmark that grades AI agents on the final state they leave in business systems rather than on their transcripts or tool calls. The …
Microsoft and Hugging Face released ThinkingBox, a benchmark that grades AI agents on the terminal backend state and side effects they leave behind rather than on their generated text, running 507 sta…
Microsoft's new open-source benchmark ThinkingBox reveals a significant reliability gap in AI agents, finding that the best-performing model succeeded on only 25.25% of tasks across 20 consecutive run…
Microsoft released ThinkingBox, an open-source benchmark and sandbox, on August 19, 2026, revealing that the best AI models, such as GPT-5.4, achieve only 25.25% reliability across 20 attempts on busi…
Microsoft released ThinkingBox, an open-source sandbox framework that tests AI agent reliability by verifying actual database changes rather than transcripts, paired with ThinkingBox-Bench. Testing 12…