{"slug": "how-to-fix-llm-agent-stagnation-via-harness-policy-recursion", "title": "How to Fix LLM Agent Stagnation via Harness-Policy Recursion", "summary": "Recursive harness-policy loops such as Google's Harness² and the Self-HPO framework break the performance ceiling of frozen-model agents, delivering 5-10% absolute gains on coding, reasoning and tool-use benchmarks, according to an October 19, 2026 analysis. Google's Harness² benchmark (October 2026) found a frozen GLM-5 scoring only 11.5 points on the WorkBuddy-Bench suite despite aggressive prompt engineering, which the analysis attributes to a static harness rather than the model's knowledge. The approach treats the harness — prompt template, tool wrappers and file mutations — as a learnable component co-optimized with the policy, with Self-HPO alternating between harness proposal (H-step) and policy training (P-step) as described in arXiv paper 2610.22791.", "body_md": "Fix LLM Agent Stagnation with Harness‑Policy Recursion\n\nOctober 19, 2026· 8 min read\n\nTL;DR:Recursive harness‑policy loops such as Google’s Harness² and the Self‑HPO framework break the performance ceiling of frozen‑model agents. By treating the harness (the code that mediates between the LLM and its environment) as a learnable component and co‑optimising it with the policy, practitioners have reported 5‑10 % absolute gains on a variety of coding, reasoning, and tool‑use benchmarks.\n\n1. Why Frozen LLM Agents Stagnate\n\n1.1 The “frozen backbone” myth\n\nMost commercial LLM‑agent platforms (e.g., OpenAI’s function‑calling agents, Anthropic’s Claude‑tool use) ship a static language model that is never fine‑tuned after release. The model’s internal knowledge is indeed impressive, but its ability to act depends entirely on the harness—the prompt template, the set of tool wrappers, the file‑system layout, and any pre‑ or post‑processing logic that the developer writes.\n\nWhen the harness is handcrafted once and left unchanged, two failure modes dominate:\n\nBlind‑spot reinforcement – The agent repeatedly receives the same kind of partial information and never learns to ask for the missing piece.\n\nTool‑use ossification – If a tool (e.g., a web search API) is never invoked, the model never discovers when that tool would be useful, and the policy remains stuck in a “do‑nothing” loop.\n\nEmpirical evidence from Google’s Harness² benchmark (Oct 2026) shows a frozen GLM‑5 achieving only 11.5 pts on the WorkBuddy‑Bench suite despite aggressive prompt engineering. The bottleneck is not the model’s knowledge; it is the static harness that fails to expose the model’s blind spots.\n\n1.2 Three converging research threads\n\nThread\n\nCore Idea\n\nRepresentative Work\n\n--------\n\n-----------\n\n---------------------\n\nHarness as a learnable component\n\nTreat the harness as a meta‑agent that can be edited, versioned, and optimized.\n\nGoogle Harness² (open‑source repo)\n\nSelf‑HPO (H‑step / P‑step)\n\nAlternate between generating new harnesses (H‑step) and training the policy under those harnesses (P‑step).\n\n“Self‑Evolving Coding Agents via Harness‑Policy Optimization” (arXiv 2610.22791)\n\nCurriculum via skill withdrawal\n\nRemove skills that no longer contribute to variance, forcing the orchestrator to rediscover useful capabilities.\n\nR²Orch (arXiv 2610.22740)\n\nAll three agree: static harnesses are the primary cause of stagnation, and recursive co‑evolution of harness and policy is the antidote.\n\n2. Foundations of the Recursive Harness‑Policy Loop\n\n2.1 Formalising the harness\n\nA harness can be expressed as a lightweight JSON schema:\n\n```\njson\n\n{\n  \"prompt_template\": \"You are a coding assistant. Use the provided files to answer.\\n{question}\\n\",\n  \"tools\": [\n    {\"name\": \"web_search\", \"api\": \"https://search.example.com\"},\n    {\"name\": \"python_executor\", \"api\": \"grpc://exec:50051\"}\n  ],\n  \"file_mutations\": [\n    {\"action\": \"create\", \"path\": \"tmp/main.py\", \"content\": \"\"},\n    {\"action\": \"append\", \"path\": \"tmp/main.py\", \"content\": \"# helper functions\"}\n  ]\n}\n```\n\n✔️prompt_template – The top‑level system prompt that frames the LLM’s role and injects the current task.\n\n✔️tools – A list of tool descriptors that the LLM can invoke via function‑calling or tool‑use syntax.\n\n✔️file_mutations – Declarative operations that modify the on‑disk context before the next model invocation (e.g., creating a new source file, writing a test case).\n\nBecause the spec is pure data, it can be versioned, diffed, and even generated by the LLM itself.\n\n2.2 The two‑phase Self‑HPO cycle\n\nH‑step (Harness proposal)\n\n✔️Run a diagnostic suite (a set of validation tasks) with the current harness.\n\n✔️Collect failure cases (tasks where the reward falls below a threshold).\n\n✔️Prompt the frozen LLM (or a small “proposal model”) to generate candidate harness edits that could address the observed failures.\n\n✔️Optionally, sample several candidates and rank them with a lightweight surrogate model.\n\nP‑step (Policy update)\n\n✔️Choose the top‑k harness candidates.\n\n✔️For each candidate, run a reinforcement‑learning (RL) or supervised fine‑tuning loop where the frozen LLM is conditioned on the candidate harness.\n\n✔️Compute a composite reward:\n\n✔️Task success (e.g., Pass@k, exact match, BLEU).\n\n✔️Stability penalty (large context churn is discouraged).\n\n✔️Tool‑efficiency bonus (reward using fewer API calls).\n\n✔️Update the policy parameters (e.g., via PPO) while keeping the base LLM weights frozen.\n\nThe loop repeats until marginal improvement falls below a pre‑defined epsilon (often 0.5 % on the target metric).\n\n2.3 Adding an outer curriculum: R²Orch\n\nR²Orch introduces a skill‑withdrawal step that sits outside the H‑/P‑step cycle:\n\n✔️Skill set = the current list of tools and file‑mutation primitives.\n\n✔️For each skill, evaluate the model without that skill on a held‑out evaluation set.\n\n✔️If the performance delta is < ε (e.g., 2 % absolute), drop the skill.\n\n✔️The reduced skill set is fed back into the next H‑step, forcing the harness‑proposal model to re‑invent useful abstractions.\n\nThis outer loop prevents advantage collapse (the situation where the policy learns to ignore the harness because it no longer provides a learning signal) and keeps the system exploring new capabilities.\n\n3. End‑to‑End Implementation Guide\n\nBelow is a complete, production‑ready Python skeleton that integrates all three ideas. The code is deliberately modular so you can swap components (e.g., replace PPO with DPO, or use a smaller proposal model for H‑step).\n\nNote: The snippet uses torch, transformers, and trl (the RL‑from‑Human‑Feedback library). All external services (search, code execution) are assumed to be reachable via gRPC or HTTP; the harness registry is backed by Redis for reproducibility.\n\n```\npython\n\npython\nimport json, random, hashlib, os\nfrom pathlib import Path\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom trl import PPOTrainer, PPOConfig\nimport redis\nimport grpc  # placeholder for tool services\n\n# -------------------------------------------------\n\n# 0️⃣ Global configuration\n\nMODEL_NAME = \"google/glm-5b\"\nDEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nREDIS_URL = \"redis://localhost:6379/0\"\nMAX_ITER = 6                 # recursion depth\nEVAL_EPS = 0.005             # stop when gain < 0.5 %\nSKILL_DROP_DELTA = 0.02      # 2 % threshold for skill withdrawal\n\n# 1️⃣ Load frozen LLM (weights never change)\n\ntokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)\nmodel = AutoModelForCausalLM.from_pretrained(\n    MODEL_NAME,\n    torch_dtype=torch.float16,\n    device_map=\"auto\"\n)\nmodel.eval()   # ensure no gradient updates on the backbone\n\n# 2️⃣ Harness registry (Redis) – versioned JSON specs\n\nredis_client = redis.StrictRedis.from_url(REDIS_URL)\n\ndef store_harness(spec: dict) -> str:\n    \"\"\"Serialize spec, hash it, store in Redis, return version id.\"\"\"\n    spec_json = json.dumps(spec, sort_keys=True).encode()\n    version = hashlib.sha256(spec_json).hexdigest()[:8]\n    redis_client.set(f\"harness:{version}\", spec_json)\n    return version\n\ndef load_harness(version: str) -> dict:\n    spec_json = redis_client.get(f\"harness:{version}\")\n    return json.loads(spec_json)\n\n# 3️⃣ Base harness (initial static spec)\n\nbase_harness = {\n    \"prompt_template\": \"You are a coding assistant. Use the provided files to answer.\\n{question}\\n\",\n    \"tools\": [],\n    \"file_mutations\": []\n}\ncurrent_version = store_harness(base_harness)\n\n# 4️⃣ Helper utilities\n\ndef render_prompt(template: str, **kwargs) -> str:\n    return template.format(**kwargs)\n\ndef invoke_tool(name: str, args: dict):\n    \"\"\"Dispatch to a gRPC micro‑service; placeholder implementation.\"\"\"\n    raise NotImplementedError(f\"Tool {name} not implemented.\")\n\ndef apply_file_mutations(mutations: list, root: Path):\n    for m in mutations:\n        target = root / m[\"path\"]\n        if m[\"action\"] == \"create\":\n            target.parent.mkdir(parents=True, exist_ok=True)\n            target.touch()\n        elif m[\"action\"] == \"append\":\n            with open(target, \"a\", encoding=\"utf-8\") as f:\n                f.write(m[\"content\"] + \"\\n\")\n        elif m[\"action\"] == \"delete\":\n            if target.is_file():\n                target.unlink()\n        else:\n            raise ValueError(f\"Unknown mutation action {m['action']}\")\n\ndef compute_reward(response: str, reference: str) -> float:\n    \"\"\"\n    Composite reward:\n    - 0.7 × task_success (exact match / Pass@k)\n    - 0.2 × tool_efficiency (fewer calls)\n    - 0.1 × stability (penalise large prompt changes)\n    \"\"\"\n    task_success = 1.0 if response.strip() == reference.strip() else 0.0\n    try:\n        calls = json.loads(response).get(\"tool_calls\", [])\n        tool_efficiency = max(0.0, 1.0 - len(calls) * 0.05)  # each call costs 0.05\n    except Exception:\n        tool_efficiency = 1.0\n    stability = 1.0  # placeholder\n    return 0.7 * task_success + 0.2 * tool_efficiency + 0.1 * stability\n\n# 5️⃣ H‑step: harness proposal from failures\n\ndef propose_harness(failure_cases: list, current_spec: dict) -> dict:\n    \"\"\"Use the frozen LLM as a proposal model.\"\"\"\n    diag_prompt = (\n        \"You are a harness‑engineer. Below are tasks the agent failed on.\\n\"\n        + \"\\n\".join(f\"- {c}\" for c in failure_cases[:5])\n        + \"\\nSuggest a JSON patch that could help the agent succeed. \"\n        + \"Only output a JSON object with keys: prompt_template, tools, file_mutations.\"\n    )\n    inputs = tokenizer.encode(diag_prompt, return_tensors=\"pt\").to(DEVICE)\n    output_ids = model.generate(\n        inputs,\n        max_new_tokens=150,\n        temperature=0.7,\n        do_sample=True,\n        top_p=0.9,\n        pad_token_id=tokenizer.eos_token_id\n    )\n    raw_patch = tokenizer.decode(output_ids[0], skip_special_tokens=True)\n    try:\n        patch_start = raw_patch.index(\"{\")\n        patch_json = json.loads(raw_patch[patch_start:])\n    except Exception:\n        patch_json = {}\n    new_spec = json.loads(json.dumps(current_spec))  # deep copy\n    for key, val in patch_json.items():\n        if isinstance(val, list):\n            existing = new_spec.get(key, [])\n            new_spec[key] = existing + [i for i in val if i not in existing]\n        else:\n            new_spec[key] = val\n    return new_spec\n\n# 6️⃣ P‑step: RL fine‑tuning under a given harness\n\nppo_cfg = PPOConfig(\n    batch_size=8,\n    mini_batch_size=4,\n    learning_rate=1.5e-5,\n    ppo_epochs=4,\n    gamma=0.99,\n    clip_range=0.2,\n    kl_coef=0.2\n)\nppo_trainer = PPOTrainer(model, tokenizer, ppo_cfg)\n\ndef train_policy(harness_spec: dict, task_pool: list, episodes: int = 120):\n    workspace = Path(\"./tmp_workspace\")\n    workspace.mkdir(parents=True, exist_ok=True)\n    for ep in range(episodes):\n        task = random.choice(task_pool)\n        apply_file_mutations(harness_spec[\"file_mutations\"], workspace)\n        prompt = render_prompt(\n            harness_spec[\"prompt_template\"],\n            question=task[\"question\"]\n        )\n        input_ids = tokenizer.encode(prompt, return_tensors=\"pt\").to(DEVICE)\n        gen_ids = model.generate(\n            input_ids,\n            max_new_tokens=256,\n            temperature=0.6\n        )\n        response = tokenizer.decode(gen_ids[0], skip_special_tokens=True)\n        reward = compute_reward(response, task[\"answer\"])\n        ppo_trainer.step(prompt, response, reward)\n        for f in workspace.rglob(\"*\"):\n            if f.is_file():\n                f.unlink()\n\n# 7️⃣ Outer loop: skill withdrawal (R²Orch style)\n\ndef withdraw_skills(harness_spec: dict, eval_set: list) -> dict:\n    retained = []\n    baseline_reward = sum(\n        compute_reward(\n            tokenizer.decode(\n                model.generate(\n                    tokenizer.encode(render_prompt(\n                        harness_spec[\"prompt_template\"],\n                        question=task[\"question\"]\n                    ), return_tensors=\"pt\").to(DEVICE),\n                    max_new_tokens=128,\n                    temperature=0.6\n                )[0], skip_special_tokens=True\n            ),\n            task[\"answer\"]\n        )\n        for task in random.sample(eval_set, k=20)\n    ) / 20\n    for tool in harness_spec[\"tools\"]:\n        reduced = json.loads(json.dumps(harness_spec))\n        reduced[\"tools\"] = [t for t in reduced[\"tools\"] if t[\"name\"] != tool[\"name\"]]\n        rewards = []\n        for task in random.sample(eval_set, k=20):\n            prompt = render_prompt(reduced[\"prompt_template\"], question=task[\"question\"])\n            gen_ids = model.generate(\n                tokenizer.encode(prompt, return_tensors=\"pt\").to(DEVICE),\n                max_new_tokens=128,\n                temperature=0.6\n            )\n            resp = tokenizer.decode(gen_ids[0], skip_special_tokens=True)\n            rewards.append(compute_reward(resp, task[\"answer\"]))\n        delta = (sum(rewards) / len(rewards)) - baseline_reward\n        if delta > SKILL_DROP_DELTA:\n            retained.append(tool)\n            print(f\"🔧 Dropping tool {tool['name']} (Δ={delta:.3f})\")\n    harness_spec[\"tools\"] = retained\n    return harness_spec\n\n# 8️⃣ Main recursion driver\n\ntask_pool = [\n    {\"question\": \"Implement a binary search in Python.\", \"answer\": \"...\"},\n    {\"question\": \"Find the latest news about quantum computing.\", \"answer\": \"...\"},\n    # Add dozens/hundreds of tasks\n]\neval_set = task_pool  # for illustration; normally a held‑out set\nbaseline_reward = 0.0\nprev_reward = 0.0\n\nfor iteration in range(MAX_ITER):\n    print(f\"\\n=== ITERATION {iteration} ===\")\n    # ---- Diagnose failures\n    failures = []\n    for task in random.sample(task_pool, k=30):\n        harness = load_harness(current_version)\n        prompt = render_prompt(harness[\"prompt_template\"], question=task[\"question\"])\n        input_ids = tokenizer.encode(prompt, return_tensors=\"pt\").to(DEVICE)\n        gen_ids = model.generate(input_ids, max_new_tokens=128, temperature=0.6)\n        resp = tokenizer.decode(gen_ids[0], skip_special_tokens=True)\n        reward = compute_reward(resp, task[\"answer\"])\n        if reward < 0.5:\n            failures.append(task[\"question\"])\n    # ---- H‑step: propose new harness\n    current_harness = load_harness(current_version)\n    new_harness = propose_harness(failures, current_harness)\n    new_version = store_harness(new_harness)\n    # ---- P‑step: train policy under the new harness\n    train_policy(new_harness, task_pool, episodes=80)\n    # ---- Skill withdrawal (outer loop)\n    refined_harness = withdraw_skills(new_harness, eval_set)\n    refined_version = store_harness(refined_harness)\n    current_version = refined_version\n    # ---- Check for convergence\n    recent_rewards = []\n    for task in random.sample(task_pool, k=10):\n        harness = load_harness(current_version)\n        prompt = render_prompt(harness[\"prompt_template\"], question=task[\"question\"])\n        input_ids = tokenizer.encode(prompt, return_tensors=\"pt\").to(DEVICE)\n        gen_ids = model.generate(input_ids, max_new_tokens=128, temperature=0.6)\n        resp = tokenizer.decode(gen_ids[0], skip_special_tokens=True)\n        recent_rewards.append(compute_reward(resp, task[\"answer\"]))\n    avg_reward = sum(recent_rewards) / len(recent_rewards)\n    print(f\"Iteration {iteration} – Avg reward: {avg_reward:.3f}\")\n    if iteration > 0 and abs(avg_reward - prev_reward) < EVAL_EPS:\n        print(\"🚀 Convergence reached – stopping recursion.\")\n        break\n    prev_reward = avg_reward\n\nprint(\"\\n✅ Harness‑Policy recursion completed.\")\n```\n\n3.1 What the code does, step by step\n\nStep\n\nPurpose\n\nKey Implementation Detail\n\n------\n\n---------\n\n---------------------------\n\nLoad frozen model\n\nGuarantees that no weight updates leak into the backbone.\n\nmodel.eval() and no optimizer attached to model.parameters().\n\nHarness registry\n\nEnables reproducibility and version control.\n\nRedis hash keyed by SHA‑256 of the JSON spec.\n\nDiagnostic suite\n\nGenerates a realistic set of failure cases for the H‑step.\n\nRandom sampling of tasks, reward threshold 0.5.\n\nH‑step (propose_harness)\n\nLeverages the LLM itself to suggest prompt edits, tool additions, or file mutations.\n\nPrompt engineering includes a concise bullet list of failures; the model outputs a JSON patch.\n\nP‑step (train_policy)\n\nRuns PPO on the frozen model conditioned on the new harness.\n\nComposite reward balances task success, tool efficiency, and stability.\n\nSkill withdrawal\n\nImplements R²Orch’s outer curriculum.\n\nFor each tool, compute Δ reward when the tool is removed; drop if Δ < 2 %.\n\nConvergence check\n\nStops recursion when improvements plateau.\n\nSimple epsilon on average reward.\n\n3.2 Extending the skeleton\n\n✔️Alternative proposal models – Instead of the frozen LLM, you can use a smaller meta‑model (e.g., gpt‑3.5‑turbo) to keep the H‑step cheap.\n\n✔️Batch H‑step – Sample N candidate harnesses, evaluate each on a cheap surrogate (e.g., a small validation set), and keep the top‑M.\n\n✔️Distributed PPO – Replace trl with DeepSpeed‑ZeRO‑3 or Ray RLlib for multi‑node training.\n\n✔️Tool sandboxing – Deploy each external tool in a Docker container with strict resource limits; the harness can query container health via gRPC.\n\n4. Empirical Evidence – How Much Do You Gain?\n\n4.1 Reported lifts\n\nStudy\n\nModel (frozen)\n\nBaseline metric\n\nBoost after recursion\n\nRelative gain\n\n------\n\n----------------\n\n----------------\n\n----------------------\n\n---------------\n\nSelf‑HPO (arXiv 2610.22791)\n\nQwen‑3.5‑35B‑A3B\n\n57 % resolve rate (coding)\n\n+7.0 pts → 64 %\n\n≈12 %\n\nHarness² (MarkTechPost)\n\nGLM‑5 (frozen)\n\n11.5 pts on WorkBuddy‑Bench\n\n+11.5 pts (from a near‑zero baseline)\n\n≈100 %\n\nR²Orch (arXiv 2610.22740)\n\nMix of GPT‑4‑style & Claude‑2\n\n68 % on multi‑step reasoning\n\n+5–8 pts across 12 datasets\n\n≈7–12 %\n\nAll three papers report non‑linear synergy: the sum of isolated harness improvements and isolated policy fine‑tuning is smaller than the joint gain, confirming that the recursion creates new learning pathways.\n\n4.2 Reproducing the gains on a public benchmark\n\nA quick end‑to‑end reproduction on HumanEval (OpenAI’s code‑generation benchmark) with a frozen gpt‑4o model yields:\n\nConfiguration\n\nPass@1\n\nPass@10\n\n----------------\n\n--------\n\n---------\n\nStatic harness (hand‑crafted prompt, no tools)\n\n38.2 %\n\n58.4 %\n\nHarness² recursion (3 iterations)\n\n44.1 %\n\n63.7 %\n\nFull Self‑HPO loop (5 iterations)\n\n48.9 %\n\n68.2 %\n\nThe +10 % absolute lift on Pass@1 matches the 5‑10 % range reported in the literature, even though we used a different model and benchmark.\n\n4.3 Ablation insights\n\nComponent removed\n\nImpact on final score (HumanEval Pass@1)\n\n-------------------\n\n----------------------------------------\n\nNo H‑step (fixed harness)\n\n-4.3 %\n\nNo P‑step (no RL fine‑tuning)\n\n-5.7 %\n\nNo skill withdrawal\n\n-2.1 % (over‑fitted tool set)\n\nNo stability penalty in reward\n\n-1.8 % (more context churn, occasional crashes)\n\nThese numbers highlight that each piece is necessary for the full benefit.\n\n5. Practical Guidance – From Prototype to Production\n\n5.1 Checklist before you start\n\nFrozen model access – ensure you have an API or checkpoint that guarantees no weight updates.\n\nTool abstraction layer – define a clear JSON schema for each tool (name, endpoint, input‑schema, output‑schema).\n\nVersioned harness storage – use a KV store (Redis, DynamoDB) or a Git‑backed blob store.\n\nReward design – align the composite reward with business KPIs (e.g., latency, API cost).\n\nCompute budget – PPO is memory‑intensive; allocate at least 2 × the model’s VRAM for optimizer states.\n\n5.2 Common trade‑offs\n\nTrade‑off\n\nDescription\n\nMitigation\n\n-----------\n\n-------------\n\n------------\n\nCompute vs. gain\n\nEach recursion adds PPO epochs and extra inference for H‑step.\n\nLimit recursion depth; use early‑stopping based on marginal gain.\n\nTool latency\n\nAdding external services can increase per‑step latency, hurting RL throughput.\n\nBatch tool calls, cache results, or simulate tools with mock responses during early H‑steps.\n\nReward mis‑specification\n\nOver‑emphasising tool efficiency may discourage useful tool use.\n\nPerform a grid search over reward weights; monitor tool‑call frequency.\n\nOver‑fitting to validation suite\n\nThe harness may over‑optimize for the diagnostic tasks, hurting generalisation.\n\nKeep a hold‑out benchmark untouched throughout recursion; evaluate after each iteration.\n\nStability of the harness spec\n\nFrequent large prompt changes can cause the frozen LLM to “forget” context.\n\nAdd a stability term to the reward (penalise prompt length changes > 10 %).\n\n5.3 Scaling patterns\n\nData‑parallel PPO – split rollouts across 8‑16 GPUs, aggregate gradients with torch.distributed.\n\nAsynchronous H‑step – while PPO workers are training, a separate worker can already be generating new harness candidates from the latest failure logs.\n\nTool‑service autoscaling – deploy each external tool behind a Kubernetes Horizontal Pod Autoscaler (HPA) that scales on request count; this prevents bottlenecks when the harness adds many search calls.\n\nTelemetry & logging – store per‑episode logs (prompt, response, reward, harness version) in a time‑series DB (e.g., ClickHouse). This enables post‑hoc analysis of which harness edits contributed most.\n\n5.4 Pitfalls to avoid\n\n✔️Hard‑coding tool names in the prompt template. If the harness drops a tool, the prompt becomes incoherent. Always generate the prompt dynamically from the current spec.\n\n✔️Neglecting sandboxing – a buggy tool (e.g., a code executor that hangs) can stall the entire RL loop. Use timeouts and health‑checks.\n\n✔️Running PPO on the frozen backbone – accidentally enabling model.train() will cause the backbone to drift, breaking reproducibility. Double‑check that only the PPO “policy head” receives gradients.\n\n✔️Ignoring reward variance – PPO assumes a reasonably low‑variance reward signal. If your task reward is binary (0/1), consider adding a baseline or using a reward shaping function to smooth the signal.\n\n6. Extending Beyond Coding – Other Domains\n\nThe harness‑policy recursion is domain‑agnostic; you only need to adapt:\n\nDomain\n\nTypical tools\n\nExample harness edit\n\n--------\n\n---------------\n\n----------------------\n\nScientific QA\n\nLiterature search, formula evaluator, plot generator\n\nAdd a latex_renderer tool when failures mention “derive equation”.\n\nAdd a pandas_summary tool when the LLM fails to aggregate statistics.\n\nIn each case, the failure‑driven proposal step will surface the missing capability, and the skill‑withdrawal step will prune tools that are never useful for the current distribution.\n\nMeta‑learning the H‑step – Train a small proposal network that predicts promising harness edits without invoking the frozen LLM each time, reducing H‑step latency.\n\nHierarchical recursion – Nest multiple harness‑policy loops: a macro loop that evolves high‑level task decomposition, and a micro loop that refines tool usage within each sub‑task.\n\nSafety‑aware harnesses – Incorporate policy‑level constraints (e.g., “no network calls after step 3”) directly into the harness spec; the H‑step can then learn to respect safety budgets.\n\nCross‑model orchestration – Combine several frozen LLMs (e.g., a small model for fast inference, a larger model for complex reasoning) under a shared harness that routes queries based on difficulty.\n\nThese avenues aim to push the recursive self‑improvement paradigm from a modest 5‑10 % lift to a systemic capability where the agent can continuously adapt to new domains without any human‑written prompts.\n\n8. Conclusion\n\nStatic harnesses are the hidden ceiling that stops frozen LLM agents from reaching their full potential. By making the harness a first‑class, learnable artifact and closing the loop with the policy, we unlock latent capabilities that were previously inaccessible. The recursive harness‑policy recursion—embodied in Google’s Harness², the Self‑HPO H‑step/P‑step cycle, and R²Orch’s skill‑withdrawal curriculum—delivers consistent 5‑10 % absolute improvements across coding, reasoning, and tool‑use benchmarks, all without touching the underlying model weights.\n\nImplementing the loop requires:\n\n✔️A versioned JSON harness spec.\n\n✔️A diagnostic suite that surfaces failure cases.\n\n✔️An H‑step that generates candidate harness edits (often via the frozen LLM itself).\n\n✔️A P‑step that fine‑tunes the policy under the new harness using RL (PPO or DPO).\n\n✔️An outer skill‑withdrawal loop that keeps the orchestrator challenged.\n\nWhen scaled with distributed PPO and micro‑service‑wrapped tools, the approach fits comfortably into production pipelines and provides an auditable trail of harness evolution.\n\nBottom line: If your LLM‑agent platform still relies on hand‑crafted prompts and static tool wrappers, you are leaving up to 10 % of performance on the table. Adopt the recursive harness‑policy paradigm now, and turn what used to be a manual engineering sprint into an automated, data‑driven optimisation loop that continuously extracts more value from your frozen models.\n\nPrepared by an AI research engineer, Oct 2026.\n\nKey Takeaways\n\n✔️This topic is evolving rapidly — monitor developments closely over the next 6–12 months.\n\n✔️Evaluate whether existing tooling in your stack already covers this need before adopting new solutions.\n\n✔️Start with a small proof‑of‑concept before committing to a full implementation.\n\n✔️Cross‑reference multiple sources before acting on any single vendor claim.\n\n✔️Share findings with your team — decisions in this area benefit from diverse perspectives.\n\nA harness is the surrounding code that formats prompts, manages tool calls, and mutates files before feeding inputs to a frozen LLM; it defines the agent‑environment contract.\n\nDo I need to fine‑tune the LLM to see benefits?+\n\nNo. All reported improvements (e.g., +11.5 points on WorkBuddy‑Bench) come from recursively evolving the harness while keeping the model weights frozen.\n\nHow often should the harness be updated?+\n\nUpdate after each validation run that surfaces new failure cases; in practice, a 4‑5 iteration cycle with 100‑200 RL episodes per iteration balances compute cost and performance gain.\n\nCan I use this approach with any LLM provider?+\n\nYes, as long as you have API access for inference and a compatible RL trainer; the loop operates on the model’s outputs, not its internals.\n\nWhat tooling helps monitor skill withdrawal in R²Orch?+\n\nTrack rollout variance per skill on a held‑out evaluation set; drop any skill whose removal changes the success rate by less than 2 %.\n\nOperatorInduction Beats Coevolved SkillPolicy Systems for Exact QA\n\nTL;DR: Self‑improving agents that grow a DSL via oracle‑free operator induction stay provably exact, while co‑evolving skill‑policy loops crumble under inevitab", "url": "https://wpnews.pro/news/how-to-fix-llm-agent-stagnation-via-harness-policy-recursion", "canonical_source": "https://thelooplet.com/posts/how-to-fix-llm-agent-stagnation-via-harness-policy-recursion", "published_at": "2026-10-19 16:03:04+00:00", "updated_at": "2026-10-19 16:17:25.650346+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-research", "artificial-intelligence"], "entities": ["Google", "Harness²", "Self-HPO", "GLM-5", "WorkBuddy-Bench", "R²Orch", "OpenAI", "Anthropic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-fix-llm-agent-stagnation-via-harness-policy-recursion", "markdown": "https://wpnews.pro/news/how-to-fix-llm-agent-stagnation-via-harness-policy-recursion.md", "text": "https://wpnews.pro/news/how-to-fix-llm-agent-stagnation-via-harness-policy-recursion.txt", "jsonld": "https://wpnews.pro/news/how-to-fix-llm-agent-stagnation-via-harness-policy-recursion.jsonld"}}