AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories

AgentBrew, an offline training framework from researchers publishing on arXiv (2609.05837v1), learns tool-use policies from a single batch of raw interaction trajectories without task verifiers or iterative on-policy rollouts. On three real-world MCP applications — GitHub, Notion, and PostgreSQL — AgentBrew improved Qwen3-32B by +8.7 Acc / +9.7 Score on average, surpassing Qwen3-235B (+2.3 / +4.4) and outperforming rejection sampling (+5.9 / +10.3). The framework uses retrospective task inference to reconstruct an aligned instruction per trajectory and PMI-based credit assignment to weight policy training, with code available at https://github.com/alphatogo/AgentBrew.

arXiv:2609.05837v1 Announce Type: new Abstract: LLM-based agents are increasingly deployed in real-world applications through tool-use APIs, yet training them for specific environments remains fundamentally difficult: real-world applications provide no pre-defined tasks or verifiers, no faithful simulators, and limited budget for large-scale environment interaction. In this paper, we propose \textbf{AgentBrew}, an offline training framework that learns effective tool-use policies from a single batch of raw interaction trajectories, without task verifiers or iterative on-policy rollouts. The agent first explores the target environment to collect a raw trajectory corpus without quality filtering. To extract training signal from this noisy corpus, \emph{retrospective task inference} reconstructs an aligned instruction for each trajectory based on its actual outcome, and \emph{PMI-Based credit assignment} decomposes the trajectory's total information about the inferred instruction into additive per-action credits via pointwise mutual information PMI . These credits weight the policy training objective, amplifying informative actions while suppressing ineffective ones. On three real-world MCP applications GitHub, Notion, PostgreSQL , AgentBrew improves Qwen3-32B by +8.7 Acc / +9.7 Score on average, surpassing Qwen3-235B +2.3 / +4.4 and outperforming rejection sampling +5.9 / +10.3 . These results demonstrate that fine-grained offline learning can recover useful supervision from raw trajectories that filtering-based approaches would discard. The code is available at https://github.com/alphatogo/AgentBrew