{"slug": "autosaddler-automatic-harness-optimization", "title": "AutoSaddler: Automatic Harness Optimization", "summary": "Researchers proposed AutoSaddler, an automatic harness optimization framework that improves LLM agent reliability on long-horizon tasks by treating harness updates as code patches derived from failure traces. In experiments on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0, AutoSaddler improved agent performance by 9.0, 9.6, and 10.0 percentage points over base harnesses, respectively. The study highlights deep debugging, targeted modifications, and generalization-aware selection as key ingredients for effective harness optimization.", "body_md": "# Computer Science > Artificial Intelligence\n\n[Submitted on 24 Aug 2026]\n\n# Title:AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces\n\n[View PDF](/pdf/2608.23041)\n\n[HTML (experimental)](https://arxiv.org/html/2608.23041v1)\n\nAbstract:LLM agents remain unreliable on long-horizon tasks, where small local failures can compound over extended interactions and lead to overall task failure. Although external harnesses can substantially improve robustness, harness design remains a manual and expensive process that requires searching over a large space of prompts, tool configurations, and control logic. We propose AutoSaddler, an automatic harness optimization framework that formulates harness improvement as an offline learning problem and iteratively updates the harness using failure signals from mini-batches. AutoSaddler combines failure-trace diagnosis, structured patch generation that treats the harness as code, and validation-based update selection. Experiments on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 show that AutoSaddler substantially improves agent performance over the corresponding base harnesses, achieving gains of 9.0, 9.6, and 10.0 percentage points, respectively. Ablation studies further suggest that effective harness optimization benefits from three ingredients: deep debugging rather than shallow reflection, targeted modifications rather than unconstrained editing, and generalization-aware selection rather than trajectory-specific repair. Together, these results suggest that automatic harness optimization is a promising path toward more performant and reliable agent systems.\n\n### Current browse context:\n\ncs.AI\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/autosaddler-automatic-harness-optimization", "canonical_source": "https://arxiv.org/abs/2608.23041", "published_at": "2026-08-28 13:21:38+00:00", "updated_at": "2026-08-28 13:48:48.623435+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-agents"], "entities": ["AutoSaddler", "GAIA2", "SWE-Bench Pro", "Terminal-Bench 2.0"], "alternates": {"html": "https://wpnews.pro/news/autosaddler-automatic-harness-optimization", "markdown": "https://wpnews.pro/news/autosaddler-automatic-harness-optimization.md", "text": "https://wpnews.pro/news/autosaddler-automatic-harness-optimization.txt", "jsonld": "https://wpnews.pro/news/autosaddler-automatic-harness-optimization.jsonld"}}