# What is a self-improving agent harness? Google's Harness2, and whether tuning pays > Source: > Published: 2026-10-21 09:48:14+00:00 # What is a self-improving agent harness? Google's Harness2, and whether tuning pays A self-improving agent harness is a harness that edits its own files, such as the system prompt, skills, hooks and memory, based on what went wrong in a run, keeps the edits that run better, and carries the lesson to the next task. Google's [Harness2](https://stackness.dev/tools/harness2), published on 9 October 2026, raised [GLM-5](https://stackness.dev/tools/glm) from 64.2 to 75.7 on WorkBuddy-Bench in one round. In those headline results [Claude Opus](https://stackness.dev/tools/claude-opus) 4.8 did the editing, not the agent itself, and no edit was checked for correctness. ## What is a self-improving agent harness? A self-improving agent harness treats the scaffolding around a model as something a loop can rewrite. The [harness](https://stackness.dev/blog/what-is-a-coding-agent-harness-and-which-parts-of-it-actually-change-the-output) is everything in an agent except the model: prompts, tool descriptions, skills, hooks, subagents, memory. A self-improving one runs a task, reads the trace, proposes a change to those files, tests it, and keeps the change only if the run gets better. [Arize](https://stackness.dev/tools/arize)'s [glossary](https://arize.com/glossary/self-improving-harness/) describes it as a harness "that changes in response to what it observes about its own failures", driven by production traces and eval results rather than by someone remembering to revisit it. The academic version is the Self-Harness paper ([arXiv 2606.09498](https://arxiv.org/abs/2606.09498)), in which a fixed model mines its failures, proposes small edits to one harness definition file, and accepts an edit only when it improves one of two test splits without lowering the other. Its largest absolute gain, in the August versions, was GLM-5 on AppWorld, from 44.4% to 85.0%. ## Harness2 rewrites eight harness components between runs of a task Harness2 is an Apache-2.0 Python project from Google Cloud AI Research and Northwestern University ([paper](https://arxiv.org/abs/2610.14224), [repository](https://github.com/google-research/harness2)). For each task it runs the agent on the current harness, has an editor model propose two edits, one on how the agent works and one on what a practitioner of the task knows, runs each candidate once, and composes the result. An "experience reflector" then records which edit helped, which hurt, and a lesson. The next task starts again from the stock harness, with past harnesses and lessons handed to the editor as examples, not inherited. At the smallest setting, each task costs four agent runs, three editing sessions and a reflection session, according to the code. The harness itself is eight components, each one or more files: *The Harness2 component map. Image: [google-research/harness2](https://github.com/google-research/harness2) by Google LLC, [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0).* The repository's demo shows the loop without any model calls. A ledger task's script sums only positive amounts. The editor deletes the `if amount > 0` filter, the fixed total matches the ledger, and the reflector records "Include both signs when summing in scripts/summarize.py" for the next ledger task. In the paper's real example, on JobBench with [Gemini 3.5 Flash](https://stackness.dev/tools/gemini-flash), four edits, among them a rule to reconcile the source files and reopen the outputs plus a reconciliation skill, took one task from 43.1 to 77.8. ## The agent can rewrite its guardrails, and the model and graders stay fixed Harness2 freezes the model weights, the agent framework ([OpenCode](https://stackness.dev/tools/opencode) or [OpenAI Codex](https://stackness.dev/tools/codex-cli)), its own editor and selector prompts, and the benchmark graders. Everything else on the surface is editable, including `guardrails.md` on OpenCode, `AGENTS.md` on Codex, plugins and `.codex/hooks.json`. No section is marked read-only, and there is no human approval step. The guards that exist are narrow: a path allowlist, a character cap on always-on text, and a check that no grader verdict reaches the editor. The README says "Executor network access is unrestricted, and Codex runs with its sandbox disabled", and the repository's notes record a validation run where "a Codex rollout on JobBench downloaded its task's rubric without being instructed to". The paper does not say what network limits applied to its reported numbers. The published alternatives keep the guardrails out of the loop. Self-Harness keeps its evaluator fixed and gates every edit on a regression check. The Hierarchical Self-Improvement paper ([arXiv 2608.08466](https://arxiv.org/abs/2608.08466)) loads its top-level logic from "an immutable initialization template". A third paper, SBCO ([arXiv 2608.10157](https://arxiv.org/abs/2608.10157)), shows why: its optimizer learned to add shopping items and never remove them, because "the final reward does not penalize extra items". ## Harness2 writes the same AGENTS.md, skills and hooks you write by hand A self-improving harness does not replace AGENTS.md, skills, hooks or subagents. It is a loop that writes them. On Codex, Harness2's editable surface is `AGENTS.md`, `.agents/skills/`, subagent files, scripts and hooks. Its editor prompt states a preference order: a skill first, then a script, a subagent, a hook, and always-on text last. The prompt's reasons match the [layer rules for AGENTS.md, CLAUDE.md and skills](https://stackness.dev/blog/agents-md-vs-claude-md-vs-skills-which-layer-each-instruction-belongs-in-and-what-to-delete): "ALWAYS-ON PROSE IS THE EXPENSIVE OPTION. Every word of it is in context on every run", and a hook is "the only way to enforce something the model cannot skip". Removing groups of edits after the fact, the paper found skills carried the most: dropping them cost 5.3 points on WorkBuddy, against 1.5 for the prompt files. Harness2 has no mode that edits your repository's `AGENTS.md` in place. It runs against benchmark adapters, so your own tasks need an adapter first. What it produces is a readable harness tree per task under `runs/`. ## Google measured gains on weaker models, with Claude Opus 4.8 editing Google ran Gemini 3.5 Flash and GLM-5 as the agents, each on OpenCode and Codex, over three runs with shuffled task order. Claude Opus 4.8 was the editor, composer, selector and reflector. One round of Harness2 raised every configuration, and the weaker model, GLM-5, gained most on WorkBuddy-Bench and LAB. | Agent on OpenCode | WorkBuddy-Bench | JobBench | LAB (legal) | |---|---|---|---| | Gemini 3.5 Flash | 72.4 → 77.5 | 31.5 → 36.0 | 79.9 → 81.9 | | GLM-5 | 64.2 → 75.7 | 20.6 → 24.2 | 59.6 → 71.4 | Scores from Table 1 and Table 2 of the [Harness2 paper](https://arxiv.org/abs/2610.14224): base harness, then one round. LAB is the share of criteria passed. The agent improving its own harness appears in one figure, run at a recursion budget of k=2 rather than the main tables' k=1, so the two are not a clean comparison. With the same model as agent and editor, GLM-5 gained 4.1 points on WorkBuddy-Bench, against 11.5 with Opus 4.8 editing, and Gemini 3.5 Flash 0.9 on JobBench against 4.5. It goes the other way too: GLM-5 editing itself gained 13.7 on LAB, against 11.8. The authors conclude that refinement "need not rely on a stronger editor". Claude Opus 4.8 and Opus 5 also gained 2.5 to 5.5 points editing their own harnesses. *Figure 8 from [Harness²: Recursive Agent Harnessing for an Open World](https://arxiv.org/abs/2610.14224) by Ruiyao Xu et al., 2026, [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).* What is only claimed: the paper puts Harness2 "on the empirical Pareto frontier" of tokens against score but gives no dollar figure. The authors ran every baseline themselves, I found no independent replication as of 21 October, and several one-round gains sit within one or two standard deviations of the base. ## Does harness tuning pay off? Harness tuning pays off on cheap models in both cost measurements I found; on frontier models it moved the score in Google's benchmarks but not in a test that was already passing. The cleanest cost test is Naresh Jain's [experiment on martinfowler.com](https://martinfowler.com/articles/are-harnesses-worth-the-cost.html), 13 October 2026. He built a fixed "behavioral harness", designed by people rather than by a self-improving loop, and ran Codex 10 times per condition on one Java service checked by 178 hidden acceptance tests. | Model | Harness | Accepted runs | Estimated cost per run | |---|---|---|---| | [GPT-6](https://stackness.dev/tools/gpt) Luna | No | 3 of 10 | $0.086 | | GPT-6 Luna | Yes | 10 of 10 | $0.273 | | GPT-6 Sol | No | 10 of 10 | $0.406 | | GPT-6 Sol | Yes | 10 of 10 | $2.679 | The harness took GPT-6 Luna from 3 to 10 accepted runs, and harnessed Luna cost about a third less than unharnessed Sol. On Sol it changed nothing but the bill, about 6.6 times higher. The costs are API-equivalent estimates, the time spent writing the harness is excluded, and Jain founded [Specmatic](https://stackness.dev/tools/specmatic), the tool the harness was built with. [LangChain](https://stackness.dev/tools/langchain)'s [tuning](https://www.langchain.com/blog/tuning-the-harness-not-the-model-a-nemotron-3-ultra-playbook) of [Nemotron](https://stackness.dev/tools/nemotron) 3 Ultra, done by hand in July, points the same way: from about 0.80 to 0.84 on its eval suite, against about 0.86 for Opus 4.8, at $4.48 a suite against $43.48. Harness2 adds the editor's own cost on top. At the smallest setting, every task runs four Opus 4.8 sessions for editing and reflection; the paper counts their tokens but prices none of them. ## Who needs a self-improving harness, and who can skip it A self-improving harness fits a team running a stream of similar tasks with a deliverable that can be judged, on a cheap or open model, with budget for a stronger editor. That is the paper's setting: office, legal and job tasks, with GLM-5's biggest gains on the office and legal ones. A developer on a frontier model in one repository with a hand-tuned `AGENTS.md` can skip it. Jain's frontier model passed every run without a harness, and the [harness post](https://stackness.dev/blog/what-is-a-coding-agent-harness-and-which-parts-of-it-actually-change-the-output) found that on strong models a harness moves the bill more than the pass rate. Anyone who cannot let an editor rewrite guardrails, or run agents without a sandbox, should wait for a version with a frozen section. ## Where to start with a self-improving harness Run the deterministic demo first, which needs no model credentials, and read what it writes: ``` git clone https://github.com/google-research/harness2 && cd harness2 uv venv --python 3.12 .venv && source .venv/bin/activate uv pip install -e '.[render]' harness2-demo --output runs/demo --k 2 ``` Then read the adopted harness and `EXPERIENCE.md` under `runs/demo` before copying anything into your own `AGENTS.md` or skills. For model runs, block network egress, cap the number of tasks, and keep the rules that must hold in client-enforced permissions rather than in files the editor can reach. The [context engineering](https://stackness.dev/moves/context-engineering) move covers the hand-tuned version of the same work. ## Harness2 in brief - **Licence:** Apache-2.0. "This is not an officially supported Google product." - **Install:** from source only, Python 3.12 or 3.13, not on PyPI. The provided setup for model runs uses Linux, Node.js 20+ and Vertex AI. - **Agents it drives:** OpenCode by default, or Codex. - **Stars:** over 16,000 on GitHub as of 21 October 2026, 13 days after the first code commit. ## New posts by email About one email a week with what we published. No account needed. ## Which of these do you use? Nobody on Stackness runs Harness2 yet. Tick what you use and be the first. [Play Lonny Eagle Amateur Skater 4A pixel art skateboarding game. Signing up opens it straight away.Sign up to play](https://stackness.dev/games/lonny-eagle)