Google open sources Harness2, letting AI agents rewrite their own playbook

Google Cloud AI Research released Harness2 on October 9 under an Apache 2.0 license, a method that lets an AI agent rewrite its own harness — instructions, tools, memory and routines — while leaving model weights frozen. The Propose-Probe-Compose mechanism needs no ground-truth answer key: it proposes two candidate harness edits, probes both under matched conditions against a reference run, then keeps only changes the behavioral contrast supports. On WorkBuddy-Bench a single round of harness editing added 11.5 points over baseline and five parallel recursive branches added 3.9 more; on JobBench the first pass added 4.5 points and recursion 4.2 more; on the legal-agent benchmark LAB the partial pass rate rose 11.8 percentage points before recursion and 6.7 after. The team is led by Tomas Pfister with Ruiyao Xu, Yanfei Chen and Rujun Han, and the paper used Gemini 3.5 Flash and GLM-5 as executor agents but Claude Opus 4.8 as the editor model proposing and composing harness changes.

Google Cloud AI Research released Harness2 on October 9, under an Apache 2.0 license, built to let an AI agent rewrite the scaffolding around it rather than wait for engineers to patch it by hand. Here's what makes this one worth paying attention to: it doesn't touch the model at all. The weights stay frozen. Everything Harness2 changes sits in the layer wrapped around the model, the instructions, the tools, the memory, the routines that decide what the agent tries next. Google Cloud AI Research published the underlying paper on arXiv on October 9 and put the code up at github.com/google-research/harness2. The team is led by Tomas Pfister, alongside Ruiyao Xu, Yanfei Chen, Rujun Han and several others. The mechanism is called Propose-Probe-Compose, and it's designed to work without a ground-truth answer key, which is the usual blocker for self-improvement on open-ended tasks. First the system proposes two candidate edits to its own harness, one aimed at a general weakness, one aimed at something specific to the task at hand. Then it probes both candidates by actually running them under matched conditions, comparing the resulting behavior against a reference run. Only then does it compose: it keeps whatever changes the behavioral contrast actually supports, and throws out the rest. No labeled correctness signal required. Just the difference between how the agent behaved before and after. The numbers are concrete enough to check. On WorkBuddy-Bench, which covers office, web and coding tasks, a single round of harness editing added 11.5 points over the baseline. Running the process recursively at five parallel branches added another 3.9 points on top. On JobBench, a benchmark built around professional knowledge work, the first pass added 4.5 points and recursion added 4.2 more. On LAB, a legal-agent benchmark, the gain was 11.8 percentage points in partial pass rate before recursion and 6.7 points after. These aren't huge jumps in absolute terms. But they're the kind of gain a team would normally spend weeks of manual prompt and tool engineering chasing. There's a detail in the paper worth sitting with. Google ran its executor agents on Gemini 3.5 Flash and GLM-5. But it used Claude Opus 4.8 as the editor model that proposes and composes the harness changes. That's a Google research paper choosing Anthropic's model to do the actual harness surgery. Not a small thing to admit in print. Stripe's $7.5 billion OpenRouter deal hands it the AI model dial https://startupfortune.com/stripes-75-billion-openrouter-deal-hands-it-the-ai-model-dial/ OpenRouter raised $113 million at a $1.3 billion valuation about three months before the deal leaked, meaning Stripe's reported price values it five to six times higher than that round. - Stripe acquires OpenRouter for seven point five billion https://startupfortune.com/stripes-75-billion-openrouter-deal-hands-it-the-ai-model-dial/ - how AI model routing affects developer API choices https://startupfortune.com/stripes-75-billion-openrouter-deal-hands-it-the-ai-model-dial/ Every major lab already treats its own agent scaffolding as a competitive asset it doesn't fully open up. Anthropic has spent the past year hardening Claude Code's internal harness, and OpenAI has done the same with Codex, both iterating on planning loops, tool routing and memory management largely behind closed doors. What Google just did is publish a working method for doing that iteration automatically, and hand it to anyone with a GitHub account. That's a different move from either company's approach so far. And it changes who gets to compete on agent quality: not just labs with large post-training budgets, but anyone willing to run Propose-Probe-Compose against their own stack. It also lands in the middle of a live argument about where agent self-improvement is actually heading. Industry discussion this year, including a widely read essay from Lilian Weng, former OpenAI VP of AI safety and now a co-founder of Thinking Machines Lab, has increasingly framed harness engineering, not weight updates, as the most plausible near-term path toward agents that get better at their own design. Harness2 is the first big-name release that makes that framing concrete rather than theoretical. It also isn't magic. Separate research this year on self-improving agent harnesses found that systems tend to land one solid structural fix and then stall. They lack any internal model of why that first fix worked, so they can't reliably compound it into a second. Vague success criteria make it worse: agents chasing a soft objective tend to drift toward reward hacking, tuning their behavior to look better on the metric rather than actually getting better at the task. Harness2's Probe step is a direct answer to that failure mode: it demands an observed behavioral contrast before accepting any change. Though the paper's own numbers show recursion adding smaller and smaller increments at each step, not compounding gains. None of this rewrites model weights, and nobody involved is claiming otherwise. Letting an agent edit its own tools, prompts and planning loop is one thing. Publishing the code for anyone to run on their own stack is another. Together, that's a concrete step past the model-weight self-improvement debate everyone has been having in the abstract. The byline list didn't include a safety team. That's probably the detail worth watching next. Also read: Stripe's $7.5 billion OpenRouter deal hands it the AI model dial https://startupfortune.com/stripes-75-billion-openrouter-deal-hands-it-the-ai-model-dial/ • Microsoft Drops Out of Nscale's Flagship UK Data Center Project https://startupfortune.com/microsoft-drops-out-of-nscales-flagship-uk-data-center-project/ • How to Structure the AI Agent Pilot to Production Handoff https://startupfortune.com/how-to-structure-the-ai-agent-pilot-to-production-handoff/ This article is posted in AI News https://startupfortune.com/category/ai/ , check it out for more related stories. Join the discussion Open in the community → https://startupfortune.com/community/ Almost there. Sign in and your reply posts straight away.