{"slug": "learning-to-control-llm-agent-harnesses-with-offline-reinforcement-learning", "title": "Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning", "summary": "Researchers formalize the execution harness of LLM agents as a learnable control layer using offline reinforcement learning, training a lightweight controller with advantage-weighted regression to improve verification behavior and final task quality across six domains and two benchmarks.", "body_md": "arXiv:2607.05458v1 Announce Type: new\nAbstract: Large language model (LLM) agents are usually improved by changing prompts, models, or hand-written workflows, while the execution harness around the model is treated as fixed infrastructure. We argue that this harness is itself a learnable control layer. We formalize harness operation as a finite-horizon Harness MDP, where a lightweight controller selects structural execution actions while the LLM executor remains frozen. The controller is trained from offline rollouts using advantage-weighted regression with only terminal task-rubric rewards. We also separate final task quality from a post-hoc Harness Maturity Score, which measures whether the harness follows reliable execution patterns rather than only whether the final answer is correct. This separation gives a finite-buffer view of harness learning: final-quality gains require high-return support in the offline buffer, while process behavior can shift whenever it aligns with advantage-weighted actions. Across six controlled domains and two public-benchmark adapters, the learned controller consistently improves verification behavior and selectively improves final task quality, with the largest gains on adapted tau-bench retail, adapted AgentBench DB-Bench, and coding with a calibrated structural verifier. Ablations against behavior cloning and Forced CHECK show that the gains are not explained by imitation or by simply adding checks. These results identify harness control as a learnable layer for frozen LLM agents, while showing that offline support limits when better process control becomes better final answers.", "url": "https://wpnews.pro/news/learning-to-control-llm-agent-harnesses-with-offline-reinforcement-learning", "canonical_source": "https://arxiv.org/abs/2607.05458", "published_at": "2026-07-08 04:00:00+00:00", "updated_at": "2026-07-08 04:16:28.311775+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-agents", "ai-research"], "entities": ["arXiv", "tau-bench", "AgentBench", "DB-Bench"], "alternates": {"html": "https://wpnews.pro/news/learning-to-control-llm-agent-harnesses-with-offline-reinforcement-learning", "markdown": "https://wpnews.pro/news/learning-to-control-llm-agent-harnesses-with-offline-reinforcement-learning.md", "text": "https://wpnews.pro/news/learning-to-control-llm-agent-harnesses-with-offline-reinforcement-learning.txt", "jsonld": "https://wpnews.pro/news/learning-to-control-llm-agent-harnesses-with-offline-reinforcement-learning.jsonld"}}