Infrathon: Benchmarking Language Agents for Real-World, Long-Horizon AI Infra Optimization

Researchers introduced Infrathon, a benchmark of diverse, long-horizon AI infrastructure optimization tasks spanning multiple levels, to test whether frontier agents can continually improve real-world AI infrastructure over extended periods. The team evaluated six models across 150 runs, each with a 24-hour budget.

In this work, we study whether frontier agent can continually improve real-world AI infrastructure over long horizons. We introduce Infrathon, a benchmark of diverse, long-horizon AI infrastructure optimization tasks across different levels. We evaluate six models across 150 runs with 24-hour budget