Measure First, Then Allocate: Calibrated Rollout Budgets for MCP Agents in Reinforcement Learning

A new arXiv paper (2610.28785v1) introduces BRACE (Bank-calibrated Rollout Allocation under Correlated Episodes), a method that forecasts which rollout groups will contain mixed rewards before spending generation budget on reinforcement-learning training for Model Context Protocol (MCP) agents. Across three Qwen3-VL model scales, BRACE exceeded the strongest plotted baseline peaks with fewer new training rollouts, with historical-bank collection counted separately, and it also led compared methods on the BFCL and NESTFUL benchmarks without further training. The paper pairs BRACE with Discord, a harness-evaluation method using exact paired inference and decision-preserving early stopping, and reports that live MCP experiments expose a retention-cost trade-off.

arXiv:2610.28785v1 Announce Type: new Abstract: Training tool-using language models requires costly interaction, yet identical rewards within a rollout group provide no group-relative learning signal. We use reward disagreement to guide harness evaluation and the allocation of rollouts for policy training in Model Context Protocol MCP environments. We first introduce Discord, which tests harness edits using exact paired inference and decision-preserving early stopping: only disagreeing paired outcomes favor one harness over another. This insight motivates prospective allocation in BRACE Bank-calibrated Rollout Allocation under Correlated Episodes . BRACE combines historical task evidence, uncertainty in task success rates, and a correlated reward model to forecast mixed-reward groups before allocating generation budget. Controlled harness experiments favor Discord, while live MCP experiments expose a retention-cost trade-off. At three Qwen3-VL model scales, BRACE exceeds the strongest plotted baseline peaks with fewer new training rollouts; historical-bank collection is counted separately. It also leads the compared methods on BFCL and NESTFUL without further training. Ablations probe the bank, estimator, and allocation score; prospective evaluation checks forecast calibration. These results support forecasting reward contrast before generation.