Apple releases RSI-Inference: evaluating agents for self-improving LLM inference

Apple ML Research released RSI-Inference Bench, a benchmark built from real-world vLLM and SGLang tasks that tests whether AI agents can improve the systems running AI models, spanning day-0 model support, inference optimization and bug fixes. Apple reported that after spending $100K in API tokens and 50K GPU hours, the best agent reached 26.7% pass@1, leaving the benchmark unsaturated. The release gives inference engineers a verifiable-reward evaluation covering the full inference stack, with binary and continuous scoring.