JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-Making

Researchers introduced JockeyBench, a benchmark and evaluation protocol that tests LLM agents on repeated event forecasting using historical Japanese horse racing, where agents must extract predictive signals from noisy evidence and evaluate their value across successive decisions. The protocol combines time-gated evaluation with the racing data, according to the paper's description.

Repeated event forecasting requires extracting predictive signals from noisy evidence and evaluating their value across successive decisions. We introduce JockeyBench, a benchmark and evaluation protocol that studies this problem through historical Japanese horse racing. The protocol combines time-g