Learning Stateful Predictive Knowledge From Experience

Researchers propose Stateful Knowledge Learning (SKL), a method that trains large language model agents to maintain explicit, state-anchored predictive knowledge instead of relying on trajectory-level reflection, and report that SKL outperforms reflection-based paradigms on WebShop, ScienceWorld, and ChessPuzzles benchmarks.

arXiv:2607.28638v1 Announce Type: new Abstract: As large language model LLM agents increasingly learn from experience, they primarily rely on trajectory-level reflection to extract insights. Viewed through the lens of predictive knowledge, we argue that this approach operates on episodic hindsight rather than predictive foresight, yielding brittle, path-dependent heuristics. To address this, we propose Stateful Knowledge Learning SKL . SKL shifts the agent's focus from trajectory-level summarization to maintaining Stateful Knowledge: explicit, declarative predictive assessments anchored to state. We first demonstrate a motivating example showing how stateful knowledge provides granularity, enhances generalization, and enables knowledge bootstrapping. To further scale up the idea, we introduce two algorithms via self-distillation SKL-SD and reinforcement learning SKL-RL , training agents to autonomously extract state-grounded predictive knowledge from experience and learn to leverage it for policy making. Experiments on interactive environments WebShop, ScienceWorld and a complex reasoning task ChessPuzzles demonstrate that equipping models with the inherent ability to learn stateful predictive knowledge significantly outpaces current reflection-based training paradigms.