Google Research released ToolGrad, an ACL 2026 Findings framework that generates tool-use datasets by building a verified API chain first and then writing the matching user query, reaching a 99.8% pas…
Researchers introduced the Korean Open Public API Benchmark (KOPA-Bench) with 145 real-world tasks to measure multi-step tool-calling performance of open-source LLM agents on live Korean government AP…
A new arXiv preprint (2608.18171v1) finds that looped language models improve compositional tool calling, with accuracy on multi-step tool use generally increasing with recurrent depth. The study, whi…
A developer has open-sourced a framework-agnostic testing methodology for AI agents, including a 61-source benchmark map and 58 universal test blocks across seven tiers, with full coverage of the OWAS…
Researchers at arXiv identified a silent policy-violation failure mode in tool-using LLM agents, where tools execute forbidden state transitions without error signals. In the τ²-bench airline domain, …
Researchers at an undisclosed institution found that multi-teacher on-policy distillation for agentic language models can cause over-calling of tools, where models call tools on examples that should b…
A developer analyzed the open-source ecosystem for reducing LLM costs, focusing on three cost lines: cached input, uncached/written input, and output. The key insight is that tools must preserve the m…
Kitchen Rush, a new benchmark for evaluating large language model tool-calling, measures both accuracy and latency by simulating an Overcooked-style kitchen where thinking time directly impacts game p…