Is Your Long-Horizon Benchmark Really Deep? Measuring and Synthesizing Depth in Terminal Tasks
A new analysis finds that long-horizon terminal-task benchmarks commonly measure task length by the number of turns a model needs to pass and difficulty by labels an LLM assigns during generation, and the authors report …