SearchHorizonBench: A Benchmark for Search Value in AI Agents

SearchHorizonBench was introduced as a benchmark for learning and evaluating the value of additional search in AI agents, addressing the gap between answer-success probabilities, which capture question difficulty, and the decision of whether another search is worth its cost. The benchmark is built on execution experience and targets the estimation of search's additional benefit rather than question difficulty alone.

AI agents must decide when another search is worth its cost. Answer-success probabilities capture question difficulty, while this decision requires estimating the additional benefit of search. We introduce SearchHorizonBench, a benchmark for learning and evaluating search value from execution experi