{"slug": "self-improving-agent-systems-a-unified-view", "title": "Self-Improving Agent Systems: A Unified View", "summary": "Researchers propose a unified framework for self-improving agent systems, distinguishing between model-parameter changes and persistent external-state changes such as prompts, tools, workflows, and skills. The framework decomposes the improvement system into active agent, updater, lineage, evaluation, and governance components, arguing that most practical progress today comes from improving model-external state while keeping the foundation model frozen.", "body_md": "Self-Improving Agent Systems: A Unified View of Algorithms, Skills, and Harnesses\n\nyigeng & GPT 5.6\n\n2026-07-12\n\n  \n  \n  {\nMachine Learning\n}\n  \n  \n  \n  \n  \n  \n[self-improvement,\n\nAgent]\n\nSelf-improvement is often discussed as if it meant a model rewriting its own weights. That is one possible endpoint, but it is not where most practical progress is happening today. Deployed agents are compound systems: a foundation model is surrounded by prompts, tools, workflows, memory, Skills, sub-agents, evaluators, and runtime policies. Changing any persistent part of that system can change what the agent is able to do next.\n\nThis article develops one framework for reading a fast-growing and easily fragmented literature. The central claim is simple:\n\nSelf-improvement is an evidence-driven, persistent state transition whose effect on future behavior can be evaluated.\n\nThe primary difference between prompt evolution, workflow evolution, Skill evolution, Harness evolution, and self-referential agent evolution is therefore not how ambitious their names sound. It is \nwhat state they allow to change\n. A second, independent question is \nwho constructs the successor\n. Evidence, search topology, update timing, and governance form additional axes.\n\nThe result is a deliberately bounded account of self-improvement. Most systems covered here keep the foundation model frozen and improve model-external state. Joint model–Harness adaptation appears only as a boundary case. The paper-to-paper arrows below should be read as conceptual relationships unless an explicit methodological inheritance is stated.\n\n1. Preliminaries\n\n1.1 Scope, Thesis, and Unit of Analysis\n\nThe unit of analysis in this article is not an isolated language model. It is a deployed agent system together with the machinery that proposes, evaluates, archives, and promotes changes.\n\nFor predominantly frozen-model settings, a useful engineering decomposition is\n\n\\[\nA_t=(\\theta_t,H_t),\n\\]\nwhere \\(\\theta_t\\) denotes model parameters and \\(H_t\\) denotes persistent model-external state. In most of the systems discussed here, \\(\\theta_{t+1}=\\theta_t\\) while \\(H_{t+1}\\neq H_t\\). The model is unchanged, yet the behavior of the deployed system may improve because its instructions, control flow, tools, reusable capabilities, or runtime have changed.\n\nThis framing excludes two broad areas from the main discussion. First, parameter-only methods such as ordinary fine-tuning are outside scope unless they are explicitly coupled to Harness adaptation. Second, one-off reflection, best-of-\\(N\\) answer selection, and temporary in-context adaptation are not persistent self-improvement by themselves. They may generate useful evidence, but they do not automatically create a better future system.\n\nIt is helpful to describe the whole improvement system as\n\n\\[\n\\mathcal{E}_t=(A_t,U_t,L_t;V,B),\n\\]\nand then separate its functional roles:\n\nSymbol\n\nRole\n\nTypical realizations\n\n\\(A_t=(\\theta_t,H_t)\\)\n\nActive agent system\n\nModel plus prompts, workflow, tools, Skills, memory, and runtime\n\n\\(U_t\\)\n\nUpdater\n\nPrompt optimizer, coding agent, Refiner, MetaAgent\n\n\\(L_t\\)\n\nLineage or archive\n\nBest-so-far state, Pareto pool, search tree, population, open archive\n\n\\(V\\)\n\nEvaluation and promotion\n\nTests, validation tasks, preference gates, rollout metrics\n\n\\(B\\)\n\nGovernance boundary\n\nPermission enforcement, credential boundaries, sandboxing, budgets, provenance, audit, rollback\n\nThis is a functional decomposition, not a claim that the sets are always disjoint. Hyperagents, for example, embeds part of the update mechanism inside its editable program, so \\(U_{\\text{embedded}}\\subseteq H_t\\), while parent selection, evaluation, and other outer-loop functions remain external. The distinction still matters: a system can expose a broad editable surface while keeping its evaluator and authorization enforcement protected. If the candidate can rewrite both its behavior and the standard used to judge that behavior, a higher score no longer establishes improvement.\n\n1.2 Agent Systems as Persistent Editable State\n\nA deployed agent is better viewed as a stateful program than as a single model call. The model produces actions, but the surrounding system decides what context reaches it, which tools exist, what results persist, how errors are recovered, and when another model call is made.\n\n```\nflowchart LR\n    X[\"Task or environment\"] --> H[\"Harness H_t\"]\n    T[\"Frozen or slowly changing model θ_t\"] --> H\n    H --> Y[\"Actions, artifacts, and trajectory τ_t\"]\n    Y --> E[\"Evidence: outcomes, traces, failures, cost\"]\n    E --> U[\"Updater U_t\"]\n    U --> C[\"Candidate persistent state H'\"]\n    C --> V[\"Protected evaluation V\"]\n    V -->|\"promote\"| H\n    V -->|\"admit as safe candidate\"| L[\"Lineage / archive L_t\"]\n    V -->|\"invalid or unsafe\"| Q[\"Rejected-candidate buffer\"]\n    B[\"Governance B\"] -.->|constrains edits and deployment| U\n    B -.->|protects| V\n```\n\nThis diagram already reveals two different meanings of “self.” At the system level, the deployed system improves when some persistent part of it changes. At the stricter self-referential level, the current agent, a parent version, or an editable MetaAgent participates in constructing its successor. The first does not imply the second.\n\nIt also reveals why edit scope cannot be treated as a single measure of autonomy. A fixed external optimizer may search over an entire agent codebase, while a self-referential updater may be allowed to change only one prompt. Breadth of the target and identity of the updater are independent.\n\n1.3 What Is a Skill?\n\nAnthropic defines Agent Skills operationally as organized folders of instructions, scripts, and resources that an agent can discover and load dynamically. At minimum, a Skill is a directory centered on a \nSKILL.md\n file containing metadata and instructions. More complex Skills may include scripts, references, templates, or assets. Their key systems idea is \nprogressive disclosure\n: lightweight metadata supports discovery, the main instructions are loaded when relevant, and additional files are accessed only when needed (\nAnthropic, 2025\n).\n\nFor this article, a \nSkill\n is a reusable, persistent, and portable capability artifact with five conceptual parts:\n\nPart\n\nQuestion\n\nTrigger and metadata\n\nWhen should the agent discover or invoke it?\n\nProcedure\n\nWhat workflow, heuristics, or domain rules should it follow?\n\nResources\n\nWhich scripts, references, assets, or dependencies support execution?\n\nValidation contract\n\nHow can correct or useful behavior be checked?\n\nVersion and provenance\n\nWhere did it come from, what changed, and how can it be rolled back?\n\nA Skill is not merely a prompt because it is intended to be discoverable, reusable, and lifecycle-managed. It is not identical to a tool because it may teach the agent when and how to compose multiple tools. It is not the whole Harness because it normally packages a task-scoped capability rather than controlling the global execution runtime.\n\nThe word “skill” is used much more broadly elsewhere in the literature. To keep the comparison coherent, this article does not use it to mean an implicit memory policy, a repository operating system, or a multi-agent orchestration policy.\n\n1.4 What Is a Harness?\n\nA \nHarness\n is the model-external scaffold that determines what the model sees, what actions it can take, how state is maintained, and how execution is orchestrated. Depending on the system, its editable surfaces may include:\n\nsystem prompts and context construction;\n\ntools, tool schemas, middleware, and action filters;\n\nSkills and Skill retrieval;\n\nmemory, files, logs, and artifact stores;\n\nsub-agent definitions and delegation;\n\nworkflow, retries, stopping rules, and error recovery;\n\npermission-request policies, capability routing, and runtime policies.\n\nThe shorthand “Agent \\(\\approx\\) Model + Harness” is useful engineering intuition, not a literal identity. A Harness is defined here by its system role, not by its file format. It may be represented as text, configuration, a graph, a directory, executable code, or a mixture of these.\n\nThat definition also requires a boundary. A Harness may decide when to request a capability or which tool route to use, but credential enforcement, final authorization rules, hidden tests, immutable scoring paths, sandboxing, audit logs, and ultimate rollback authority should not be editable by the candidate at the same optimization level. They belong to the governance envelope \\(B\\), even if implemented in code next to the Harness.\n\n1.5 Self-Improvement, Evolution, and Self-Reference\n\nThe terminology in this area is overloaded, so I will use three terms with different scopes.\n\nTerm\n\nMeaning in this article\n\nSelf-improvement\n\nThe broad research program of using feedback to create persistent system changes intended to improve future behavior; a demonstrated improvement additionally requires independent outcome evidence\n\nEvolutionary search\n\nThe subset of this program that explicitly uses variation, selection, lineages, populations, Pareto sets, search trees, or open archives\n\nSelf-reference\n\nThe target agent, a parent version, or an editable MetaAgent participates in constructing its successor\n\n“Skill Evolution” and “Harness Evolution” are retained as broad field labels in the chapter titles because the literature uses them that way; not every included method implements evolutionary search in the stricter algorithmic sense. These definitions allow component-scoped methods to be described precisely. Prompt and Skill evolution expose comparatively narrow edit surfaces; Harness evolution may range from one executable constraint to the full model-external scaffold. Narrow scope does not imply weak impact. Likewise, a broad program search is not necessarily self-referential if the optimizer that produces the target remains fixed.\n\nThere is therefore no defensible one-dimensional ladder from “low” to “high” self-evolution. Edit scope, updater identity, evidence quality, search topology, update timing, parameter coupling, and outer-loop mutability are separate dimensions. The rest of the article uses those dimensions as a roadmap.\n\n2. Self-Improving Agent Systems\n\n2.1 Why Self-Improvement Matters\n\nModern foundation models are rarely deployed alone. Their behavior depends on prompts, workflows, tools, memory, context management, sub-agents, and the runtime that connects them. Improving an agent therefore does not necessarily require changing model weights. For closed-weight systems in particular, persistent model-external state is often the only practical learning substrate available to application builders.\n\nThis matters economically and scientifically. A transient insight is paid for again in every context window. A persistent improvement can be reused across tasks, users, and deployments. External artifacts are also comparatively easy to inspect, diff, test, transfer, and roll back. In the near term, this makes model-external self-improvement less like science fiction and more like automated systems engineering.\n\nThe long-term motivation should still be stated carefully. Google DeepMind’s report \nFrom AGI to ASI\n identifies recursive improvement as one of four possible pathways from AGI to ASI, alongside further scaling, paradigm shifts, and large-scale multi-agent collectives. The report also emphasizes frictions and open questions along each pathway. It is therefore evidence that recursive improvement is a serious research hypothesis—not evidence that today’s agent loops are sufficient, monotonic, or destined to produce ASI.\n\nThe papers in this chapter study bounded versions of a more concrete question:\n\nWhat must be editable, observable, and independently verifiable for useful changes to accumulate?\n\nMost freeze the foundation model and modify prompts, workflows, or agent code. Some use a fixed optimizer; others let the current agent or one of its historical versions construct a successor. Automated optimization is not automatically self-reference, and self-reference is not automatically recursive intelligence amplification.\n\n2.2 An Operational Definition and the Minimal Loop\n\nTwo statements should be kept separate:\n\nSelf-improvement process.\n Evidence from executions, evaluations, or the environment produces a persistent candidate change to editable state with the aim of improving future behavior.\n\nValidated improvement claim.\n Independent outcome evaluation shows that the changed system improves future behavior relative to an appropriate baseline.\n\nThe first describes an update process; the second establishes that the process succeeded. Before the second condition is met, “self-modification” or “adaptation” is the more precise description of an individual change.\n\nThe framework has four requirements:\n\nPersistence.\n The change survives beyond the immediate answer or trajectory.\n\nEvidence.\n Execution, evaluation, or environmental feedback informs the change.\n\nAn effective update operator.\n Some mechanism converts evidence into a bounded candidate delta.\n\nOutcome evaluation.\n Future behavior is compared with a relevant baseline before improvement is claimed.\n\nThe minimal loop is:\n\n```\nflowchart LR\n    Z[\"Persistent editable state Z_t\"] --> X[\"Execute tasks\"]\n    X --> E[\"Collect traces, outcomes, cost, and failures\"]\n    E --> D[\"Diagnose reusable patterns\"]\n    D --> P[\"Propose a bounded update\"]\n    P --> M{\"Activation mode\"}\n    M -->|\"candidate or shadow\"| V[\"Independent evaluation\"]\n    M -->|\"in place\"| N[\"Next live interaction\"]\n    V -->|\"reject or roll back\"| Z\n    V -->|\"archive\"| L[\"Candidate archive / lineage\"]\n    V -->|\"promote\"| Z2[\"Active state Z_(t+1)\"]\n    N --> E\n    L -->|\"future parent\"| D\n    Z2 --> X\n```\n\nThis loop separates several operations that are often conflated:\n\nCandidate generation is not acceptance.\n An edit shows that a system can change, not that it improved.\n\nArchive admission is not deployment promotion.\n A low-scoring branch may be valuable as a stepping stone without being safe enough to serve users.\n\nDiagnostic feedback is not final authority.\n Adaptive critics can guide search, but a protected evaluation path must anchor deployment claims.\n\nImmediate online edits are possible.\n A held-out gate for every update is the stronger deployment design, not a descriptive requirement satisfied by every paper. When an edit enters a live trajectory directly, it is an adaptation until later outcome evidence supports an improvement claim; recovery mechanisms become more important.\n\nA one-off reflection that improves the current answer may be useful adaptation, but it is not persistent self-improvement under this definition. Best-of-\\(N\\) sampling chooses an answer, not a successor system. Similarly, generating one prompt, Skill, or program is construction unless subsequent evidence can revise it.\n\n2.3 A Two-Axis Roadmap with Orthogonal Overlays\n\nThe most stable primary descriptor is the \neditable set\n, not a linear ladder of artifact names. Let\n\n\\[\nH_t=\\{\n\\text{prompts, workflows, tools, Skills, memory, context, runtime}\n\\}\n\\]\ndenote the model-external scaffold. A method declares an editable subset\n\n\\[\n\\Omega_{\\text{method}}\\subseteq\\operatorname{components}(\\theta,H,U,L),\n\\]\nwhile evaluation \\(V\\) and governance \\(B\\) normally remain protected. The artifact representation—text, graph, directory, configuration, or executable code—is orthogonal to the semantic update object.\n\nWithin the Agent-design branch, prompt \\(\\rightarrow\\) workflow \\(\\rightarrow\\) code-defined target Agent \\(\\rightarrow\\) full Agent scaffold remains a useful analytical edit-surface narrative. It is not the taxonomy for the whole article. Skills are portable components that may sit inside a Harness; a broad Agent program and a broad Harness can overlap in implementation. Chapters 3 and 4 therefore re-index the same framework around declared Skill and Harness boundaries rather than continuing a size hierarchy.\n\nThe second axis is the \nupdater–target relationship\n:\n\n\\[\n\\text{fixed external optimizer}\n\\rightarrow\n\\text{lineage member edits a successor}\n\\rightarrow\n\\text{editable MetaAgent}.\n\\]\nFour further overlays should remain orthogonal:\n\nEvidence and evaluation:\n labels, tests, execution traces, opaque oracles, self-preference, deployment outcomes, or open-world verification.\n\nSearch topology:\n greedy replacement, best-history search, Pareto pools, MCTS, populations, or open archives.\n\nUpdate timing:\n between experiments, between deployments, or in place during one continuous trajectory.\n\nParameter coupling:\n frozen model, alternating model–Harness updates, or joint adaptation of \\((\\theta,H)\\).\n\nThe following table locates the systems in this chapter. It is a mechanism-level map, not a claim that each paper inherited from the preceding one.\n\nSystem\n\nPrimary persistent update\n\nUpdater–target relationship\n\nSearch and evidence\n\nTiming\n\nGEPA\n\nPrompts in a compound AI system\n\nFixed prompt optimizer\n\nTrace reflection; Pareto pool\n\nBetween evaluations\n\nAFlow\n\nWorkflow nodes, operators, edges, and prompts\n\nFixed workflow optimizer\n\nMCTS; execution score\n\nBetween evaluations\n\nADAS\n\nCode-defined target agent\n\nFixed MetaAgent\n\nArchive; validation score; runtime repair\n\nBetween evaluations\n\nSICA\n\nComplete coding-agent framework\n\nHistorical Agent generates a successor\n\nBest-history selection; score, cost, time\n\nEpisodic\n\nDGM\n\nCoding-agent codebase\n\nArchived parent produces a descendant\n\nOpen archive; empirical benchmark validation\n\nEpisodic\n\nHyperagents\n\nTaskAgent and MetaAgent in one program\n\nEditable MetaAgent modifies itself and TaskAgent\n\nDGM-style archive; multi-domain evaluation\n\nEpisodic\n\nLearning Beyond Gradients\n\nHeuristic software system\n\nCoding agent maintains external program state\n\nReward, logs, replay, tests\n\nIterative\n\nAlphaEvolve\n\nExternal algorithm or program\n\nFixed evolutionary coding pipeline\n\nProgram database; automated evaluators\n\nEpisodic\n\nUpdate object constrains the search space and the available validation tools, but does not determine them. Code can be compiled and unit-tested, yet code evolution may still use weak benchmarks. A prompt may be hard to interpret causally, yet a protected held-out evaluation can provide a strong promotion gate.\n\n2.4 Evolving Agent Artifacts with a Fixed Optimizer\n\nThe first family treats an agent artifact as an optimizable object while keeping the optimizer outside the target lineage.\n\nGEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning\n operates at the prompt layer. It samples trajectories from a compound AI system—including reasoning, tool calls, tool outputs, and evaluator feedback—then asks a reflection model to diagnose failures in natural language and propose prompt updates. Complementary candidates are retained through Pareto selection. The central insight is that a trace contains richer credit-assignment information than a final scalar reward. In this taxonomy, GEPA is persistent, feedback-driven component evolution, but its prompt optimizer is not itself being evolved.\n\nAFlow: Automating Agentic Workflow Generation\n widens the editable surface from instructions to control flow. It represents an agentic workflow as LLM-invoking nodes connected by code-defined edges, then uses Monte Carlo Tree Search to modify prompts, operators, and execution structure. Each search node is a runnable workflow. The move from GEPA to AFlow is therefore a conceptual expansion from text to executable orchestration, not a documented implementation lineage.\n\nAutomated Design of Agentic Systems\n, or ADAS, widens the space again by representing target agents as code. Meta Agent Search reads an archive of earlier designs, proposes and implements a new agent, repairs runtime errors, and evaluates the result. Code can express prompts, tools, workflows, verification, and multi-agent interaction, so ADAS should not be reduced to workflow search.\n\nThe common boundary is decisive: the optimized artifact changes, but the optimizer that designs it remains fixed. ADAS may search a broader target than a self-editing prompt method while still being less self-referential.\n\n2.5 From Agent Design to Self-Referential Evolution\n\nThe transition to self-reference is mainly a change in updater identity. The question is no longer only whether an optimizer can discover a better agent, but whether an agent from the evolving lineage can participate in producing its descendant.\n\nA Self-Improving Coding Agent\n, abbreviated SICA, treats a complete coding-agent framework as editable software. Historical versions are evaluated, and a high-utility Agent analyzes the archive and implements the next candidate. The editable surface can include tools, sub-agent use, prompts, context management, and control logic. Its utility includes cost and execution time in addition to benchmark performance, discouraging “improvements” that merely spend more.\n\nThe conceptual move from ADAS to SICA is not “workflow becomes program”; both already use code. It is \nfixed MetaAgent designs a target \\(\\rightarrow\\) an Agent from the lineage constructs its successor\n. In its experimental setting, SICA shows that a multi-component coding-agent framework can become its own software-engineering object.\n\nThe \nDarwin Gödel Machine\n, or DGM, adds open-ended search over multiple lineages. It samples parents from an archive of coding agents, modifies their code, empirically evaluates descendants, and retains a growing tree of candidates. A temporarily weak branch can remain available as a stepping stone instead of forcing every generation to descend from the current best.\n\nIt is useful to compare SICA with DGM as best-history search versus a branching open archive. That relationship is analytical; it should not be read as a verified implementation inheritance. DGM’s own ablations support the value of both self-modification and open-ended exploration within its experimental setting.\n\nThe Hyperagents paper motivates coding as a favorable first domain for self-reference because task performance and self-modification draw on the same broad competence: software engineering. A coding Agent that becomes better at understanding repositories may also become better at modifying its own scaffold. This is a structural motivation, not a generally proven law, and the alignment does not hold automatically elsewhere. A paper-review Agent can become better at review without becoming better at editing the system that implements review.\n\nHyperagents\n explicitly extends DGM to address this limitation. It combines a TaskAgent, which solves domain tasks, with a MetaAgent, which reads evaluation evidence and modifies both the TaskAgent and itself. Both live in one editable program. The lineage can therefore accumulate better task behavior and better mechanisms for diagnosing evaluations, maintaining memory, tracking performance, and generating future changes.\n\nThe paper instantiates this idea as DGM-Hyperagents, making DGM \\(\\rightarrow\\) Hyperagents the clearest methodological continuation in this chapter. Its contribution is deeper than adding non-coding domains: the procedure responsible for producing descendants becomes part of the editable state, and cross-domain experiments test whether that improvement procedure can transfer.\n\nEven here, “recursive self-improvement” should be used carefully. Evaluators, task distributions, resource limits, sandboxing, and parts of parent selection remain external. Hyperagents make more of the updater editable; they do not eliminate the trust root.\n\n2.6 Beyond the Agent Boundary\n\nThe same mutation–evaluation loop can improve programs that are not the Agent’s own scaffold. These systems form an adjacent branch. They are not the next step in the DGM \\(\\rightarrow\\) Hyperagents methodological continuation, and SICA is used only as a conceptual comparison to DGM.\n\nLearning Beyond Gradients\n argues that a coding agent can make a heuristic software system learn by reading failures, editing policies and state detectors, adding tests, inspecting logs or replays, and retaining useful memory. The persistent state is not a neural network or merely an Agent class; it is a maintained software system whose future behavior changes through evidence-driven revision. Its evidence boundary should be kept explicit: this is an author project page and \nrepository\n, not a peer-reviewed experimental paper.\n\nAlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery\n provides an evaluator-centric version of program evolution. An evolutionary coding pipeline samples from a program database, asks LLMs to generate code changes, runs automated evaluators, and returns scores and feedback to the database. Its applications include algorithm discovery and infrastructure optimization.\n\nAlphaEvolve is adjacent to Agent self-improvement rather than a strict instance. The optimized program is normally an external solution, not the coding agent’s own execution or update machinery. Its relevance lies in transferable mechanisms: diff-based mutation, population management, multi-objective evaluation, and evaluator-first problem formulation.\n\nA useful boundary test is:\n\nIf the evolved program persistently controls how the Agent will perceive, act, remember, or improve in future runs, it belongs to Agent or Harness evolution. If it is an external task solution, it is adjacent program evolution.\n\n2.7 Synthesis and Transition to Component-Scoped Improvement\n\nThe Agent literature can be read as two interacting expansions.\n\n```\nflowchart LR\n    P[\"Prompt · GEPA\"] -.->|conceptual edit-surface expansion| W[\"Workflow · AFlow\"]\n    W -.->|conceptual edit-surface expansion| C[\"Code-defined target Agent · ADAS\"]\n    C -.->|conceptual updater shift| S[\"Self-editing framework · SICA\"]\n    S -.->|conceptual search-topology contrast| D[\"Open archive · DGM\"]\n    D -->|\"explicit methodological extension\"| H[\"TaskAgent + editable MetaAgent · Hyperagents\"]\n    X[\"Fixed external optimizer\"] -.->|updater class| P\n    X -.->|updater class| W\n    X -.->|updater class| C\n    Y[\"Lineage member edits successor\"] -.->|updater class| S\n    Y -.->|updater class| D\n    Z[\"Successor-generation mechanism is editable\"] -.->|updater class| H\n```\n\nThe conceptual relationships and their evidence status are:\n\nRelationship used here\n\nEvidence status\n\nGEPA \\(\\rightarrow\\) AFlow \\(\\rightarrow\\) ADAS as widening edit scope\n\nAnalytical ordering, not claimed paper inheritance\n\nADAS \\(\\rightarrow\\) SICA as fixed designer \\(\\rightarrow\\) lineage self-edit\n\nAnalytical comparison\n\nSICA and DGM: best-history versus branching archive\n\nConceptual contrast, not asserted implementation inheritance\n\nDGM \\(\\rightarrow\\) Hyperagents\n\nExplicit methodological extension\n\nLearning Beyond Gradients and AlphaEvolve\n\nAdjacent expansions beyond Agent-scaffold evolution\n\nThis framing explains why Skill and Harness evolution deserve their own chapters. They re-index the framework around declared component boundaries and expose more specific engineering questions. A Skill is a portable capability artifact with a lifecycle. A Harness is the broader runtime that discovers, composes, executes, and governs those artifacts.\n\nConversely, several systems called Agent self-improvement—especially SICA, DGM, and Hyperagents—can also be described as broad Harness evolution when the model is frozen. The Agent label emphasizes \nwho modifies whose successor\n. The Harness label emphasizes \nwhich model-external execution surfaces are editable\n. Keeping those questions separate is the key to unifying the literature without flattening it.\n\n3. Skill Evolution: Improving Externalized Capabilities\n\n3.1 What Evolves: Skills as Persistent Artifacts\n\nThis chapter uses Skill in the narrow, artifact-centric sense established above. The important point is not the Markdown format itself, but that a capability is externalized into a persistent, reusable package instead of being reconstructed in every context window. Anthropic also explicitly anticipates agents eventually creating, editing, and evaluating Skills themselves (\nAnthropic, 2025\n).\n\nWe can represent a versioned Skill as\n\n\\[\nS_t=(m_t,i_t,c_t,r_t,q_t,d_t,p_t),\n\\]\nwhere \\(m_t\\) contains metadata and trigger conditions, \\(i_t\\) root instructions, \\(c_t\\) executable scripts, \\(r_t\\) references or assets, \\(q_t\\) tests and validation contracts, \\(d_t\\) dependencies, and \\(p_t\\) provenance and version history. Not every Skill needs every field, but making them explicit clarifies what an update may change and what must remain auditable.\n\nSkill evolution is then a persistent transition:\n\n\\[\n\\widetilde S_{t+1}=U(S_t,E_t), \\qquad\nS_{t+1}=\n\\begin{cases}\n\\widetilde S_{t+1}, & \\text{if }V(\\widetilde S_{t+1},S_t)=1,\\\\\nS_t, & \\text{otherwise}.\n\\end{cases}\n\\]\nHere \\(E_t\\) is evidence from failures, successful trajectories, external resources, tests, or deployment outcomes. \\(U\\) proposes a candidate, while \\(V\\) decides whether it should persist.\n\nThis makes a Skill \nparameter-like but not parametric\n. It changes future behavior and can accumulate across tasks, yet remains readable, portable, versionable, and rollbackable. The central question is not merely how to write a good instruction file. It is how an agent can discover, construct, validate, maintain, and deploy reusable external capability.\n\n3.2 Where Skills Come From: Failures, Trajectories, and Open-World Evidence\n\nSkill acquisition begins with abstraction. A trace contains task-specific actions, errors, and observations; a reusable Skill must extract what remains useful on future tasks. The following systems are best organized by the evidence available for that abstraction, not by publication order.\n\nEvoSkill\n starts from labeled execution failures. It diagnoses a missing capability, proposes a new Skill or an edit to an existing one, and materializes the proposal as a structured Skill folder. Its key move is to convert “the Agent failed this task” into “the Agent lacks this reusable procedure.” Candidate programs are retained on a Pareto frontier when their Skills improve held-out performance.\n\nTrace2Skill\n addresses a different bottleneck: one trajectory rarely contains enough evidence to define a robust procedure. It analyzes a diverse pool of successful and failed executions in parallel, extracts trajectory-local lessons, and consolidates recurring failures and workarounds into a coherent Skill directory. The result is not raw episodic memory retrieved at inference time, but a declarative artifact intended to transfer across tasks and models. Discovery identifies \nwhat is missing\n; distillation determines \nhow repeated experience should be compressed\n.\n\nWithin this article’s minimal loop, Trace2Skill primarily contributes the acquisition and distillation stage. The paper need not be read as instantiating a full recurrent deployment loop in which every produced artifact is repeatedly revised by later feedback.\n\nCoEvoSkills\n moves to settings in which a complex multi-file Skill must be built while the true evaluator exposes only opaque pass/fail outcomes. A Skill generator iteratively revises the artifact, while an information-isolated surrogate verifier develops executable checks and provides denser diagnostics. If the surrogate passes but the external oracle fails, the verifier—not only the Skill—is updated. Acquisition becomes inseparable from learning how to test.\n\nOpenSkill\n removes still more of the assumed learning loop. During construction, it does not rely on curated Skills, successful target-task trajectories, target answers, or a ready-made verifier. It retrieves knowledge and verification anchors from documentation, repositories, and the web, synthesizes candidate Skills, and builds virtual tasks for refinement. Target-task ground truth is reserved for final evaluation.\n\nFigure 1. OpenSkill uses open-world sources to build both a Skill and a leakage-free virtual practice environment. Source: \nYan et al., 2026\n.\n\nThe conceptual progression is a change in evidence conditions:\n\n\\[\n\\text{labeled failures}\n\\rightarrow\n\\text{trajectory pools}\n\\rightarrow\n\\text{opaque outcomes}\n\\rightarrow\n\\text{open-world evidence and self-built practice}.\n\\]\nThis is not a simple strongest-to-weakest supervision ranking. An opaque oracle and an open-world verification anchor provide different information. What matters is whether the available evidence can be converted into a stable procedure without smuggling benchmark answers into the update loop.\n\n3.3 The Skill Lifecycle: Optimize, Maintain, and Deploy\n\nCreating a candidate Skill is only the beginning. A deployed capability must be tested, registered, retrieved at the right time, revised when it fails, and retired when it becomes redundant or harmful.\n\n```\nflowchart LR\n    E[\"Failures, trajectories, and resources\"] --> P[\"Propose or revise Skill\"]\n    P --> V[\"Execute tests and evaluation tasks\"]\n    V --> G{\"Promotion gate\"}\n    G -->|\"reject\"| R[\"Repair, buffer, or archive\"]\n    R --> P\n    G -->|\"accept\"| M[\"Merge, version, and register\"]\n    M --> X[\"Retrieve and execute\"]\n    X --> D[\"Collect deployment evidence\"]\n    D --> P\n```\n\nSkillOpt\n isolates the optimization stage. It treats a Skill document as trainable external state, converts scored rollouts into bounded add, delete, or replace operations, and accepts an edit only when it strictly improves a held-out selection score. A rejected-edit buffer and explicit edit budgets make the process closer to controlled optimization than unconstrained rewriting. The final artifact is loaded normally, so optimization adds no extra model call at deployment.\n\nMUSE-Autoskill\n broadens the problem from optimizing one artifact to managing a long-lived capability. It creates Skills on demand, tests and registers them, retrieves them for later tasks, attaches experience to individual Skills, and supports refinement, merging, and pruning. A Skill library improves only if useful packages can be found and maintained without unbounded duplication.\n\nSkillClaw\n extends this lifecycle into multi-user deployment. It aggregates interactions across users and time, identifies recurring successes and failures, proposes refinements or new capabilities, and maintains the results in a shared repository. An improvement discovered by one user can propagate system-wide. The same property raises the risk: a bad local edit becomes a supply-chain change once synchronized to everyone.\n\nThe conceptual expansion is\n\n\\[\n\\text{single-artifact optimization}\n\\rightarrow\n\\text{lifecycle management}\n\\rightarrow\n\\text{shared deployment evolution}.\n\\]\nThese works address successive lifecycle bottlenecks; the sequence is not a claim of direct citation inheritance.\n\n3.4 Verification: When Does Change Become Improvement?\n\nThe governing principle is:\n\nAn edit creates a candidate; only evidence-backed acceptance establishes improvement.\n\nProposal evidence and promotion evidence should be separated whenever possible. The former may be rich and adaptive—reflection, diagnostics, retrieved documentation, or generated tests. The latter should be independent enough to prevent the updater from approving its own mistakes.\n\nSystem\n\nEvidence used to propose changes\n\nPersistent output\n\nVerification or promotion signal\n\nEvoSkill\n\nLabeled failures\n\nSkill folders and candidate frontier\n\nHeld-out frontier performance\n\nTrace2Skill\n\nSuccessful and failed trajectories\n\nConsolidated Skill directory\n\nStructural guardrails and held-out transfer evaluation; no strict gate for every patch\n\nSkillOpt\n\nScored rollout batches\n\nOptimized Skill document\n\nStrict improvement on a selection split\n\nMUSE-Autoskill\n\nTasks, runtime errors, Skill-level experience\n\nTested Skill packages and library state\n\nUnit tests and runtime feedback\n\nCoEvoSkills\n\nOutputs and surrogate diagnostics\n\nMulti-file Skill bundle\n\nSurrogate tests plus opaque external oracle\n\nSkillClaw\n\nCross-user deployment trajectories\n\nShared Skill repository\n\nDelayed real-environment validation before promotion\n\nOpenSkill\n\nOpen resources and verification anchors\n\nTask-oriented Skill set\n\nSelf-built virtual tests; hidden ground truth only for final evaluation\n\nThe table exposes a verification ladder:\n\n\\[\n\\text{held-out score}\n\\rightarrow\n\\text{programmatic tests}\n\\rightarrow\n\\text{surrogate diagnostics + opaque oracle}\n\\rightarrow\n\\text{delayed deployment outcomes}\n\\rightarrow\n\\text{self-built virtual verification}.\n\\]\nThese mechanisms should not be collapsed into one “supervision strength” number. They differ in coverage, independence, cost, and error sensitivity.\n\nTwo evidence boundaries are especially important. CoEvoSkills is not fully unsupervised: its surrogate provides actionable feedback, but a real opaque oracle still anchors correctness. OpenSkill’s supervision-free claim applies to construction, not to final benchmark evaluation. As verification becomes more autonomous, proxy drift, incomplete coverage, false acceptance, and self-confirmation become the dominant failure modes.\n\n3.5 Synthesis and Bridge to Harnesses\n\nSkill evolution is attractive because it offers a fast, inspectable adaptation layer. A Skill can be diffed, tested, transferred to another model, canary-deployed, and rolled back without changing model weights. But a high-quality artifact can still fail if the surrounding runtime retrieves it at the wrong time, allocates insufficient context, routes tools incorrectly, mishandles dependencies, or violates its execution contract.\n\nThe boundary between Skill and Harness is determined by system role rather than file extension. A script packaged inside a task-scoped Skill remains part of that Skill. Global policies for retrieval, context construction, tool routing, scheduling, permission requests, memory, recovery, and sub-agent orchestration belong to the Harness—even if stored as Markdown. Credential enforcement and maximum permission ceilings remain in the governance boundary.\n\nContinual Harness\n makes this boundary concrete. During one reset-free trajectory, its Refiner can edit the system prompt, sub-agents, Skills, and memory in place. Individual navigation Skills may be repaired and reused, and the paper separately analyzes Skill quality with a direct pathfinding oracle. Yet the overall update object is the composite runtime state \\(H_t=(p_t,G_t,K_t,M_t)\\), not the Skill library alone. It is Harness evolution with Skill evolution as one component.\n\nThis gives us the transition: \nSkill evolution changes what reusable capability is available; Harness evolution changes how the Agent operates with that capability.\n\n4. Harness Evolution: Improving the Agent Scaffold\n\nA model does not act directly on the world. It receives context through a runtime, calls tools through an interface, stores selected state, delegates work, retries failures, and decides when to stop. Harness evolution asks whether that execution system can improve while the underlying model remains fixed—or, at the boundary, while model and Harness adapt together.\n\nEvidence convention.\n Paper-linked mechanism descriptions below are paper-reported facts. The coordinate assignments and conceptual arrows are this article’s mechanism-level synthesis; they do not imply a citation, chronological, or implementation lineage.\n\n4.1 A Coordinate System for Harness Improvement\n\nWith a fixed model \\(M\\), task \\(x\\), and Harness \\(H\\), execution produces a trajectory\n\n\\[\n\\tau \\sim p_M(H,x).\n\\]\nChanging \\(H\\) changes what the model observes, which actions it can take, what persists, and how failures are handled. “The Harness changed” is still too coarse to characterize an improvement method. To align with the global notation, we describe a fixed-model Harness update with six coordinates:\n\nCoordinate\n\nQuestion\n\nTypical choices\n\n\\(\\Omega_H\\subseteq H\\)\n\nWhich Harness surface is editable?\n\nAction verifier, prompt, tool, Skill, memory, stateful program, full scaffold\n\n\\(U\\)\n\nWho proposes it?\n\nExternal optimizer, target model, specialized Refiner\n\n\\(E\\)\n\nWhat evidence drives it?\n\nEnvironment errors, rewards, traces, failure clusters, self-preference\n\n\\(T\\)\n\nWhen does it become active?\n\nBetween evaluations, between deployments, or inside one continuous run\n\n\\(V\\)\n\nHow is it evaluated or accepted?\n\nInterface checks, held-out tests, Pareto selection, preference gate, online outcome\n\n\\(B\\)\n\nWhat remains immutable?\n\nCredential and permission ceilings, sandbox, hidden evaluator, budgets, audit, rollback\n\nThis is a Harness-focused projection of the global framework, not the complete taxonomy. Search topology \\(L\\)—greedy, population, Pareto, or open archive—and parameter coupling \\(P_\\theta\\)—frozen, alternating, or joint—remain overlays.\n\n```\nflowchart LR\n    X[\"Task or environment\"] --> R[\"Rollout\"]\n    H[\"Active Harness H\"] --> R\n    R --> E[\"Evidence E\"]\n    E --> U[\"Updater U\"]\n    U --> C[\"Candidate H'\"]\n    C --> A{\"Activation mode T\"}\n    A -->|\"staged\"| VP[\"Pre-activation validation V\"]\n    VP -->|\"promote\"| H\n    VP -->|\"safe but exploratory\"| L[\"Archive L\"]\n    VP -->|\"reject\"| Q[\"Rejected-candidate buffer\"]\n    A -->|\"in place\"| H\n    H --> O[\"Observed future outcomes\"]\n    O --> VA[\"Post-activation evaluation V\"]\n    VA -->|\"retain\"| H\n    VA -->|\"roll back\"| K[\"Checkpointed prior Harness\"]\n    K --> H\n    B[\"Immutable boundary B\"] -.->|constrains| U\n    B -.->|protects| VP\n    B -.->|protects| VA\n```\n\nThe last two coordinates are particularly important. An updater can generate candidates without having final deployment authority. Staged methods evaluate before activation; online methods may activate first and evaluate later. A robust design keeps at least one evaluation and governance layer outside editable state; otherwise, the system may “improve” by weakening its test or suppressing failures.\n\n4.2 What Is Updated: From Executable Code to Agent Scaffolding\n\nAt the narrowest end, \nAutoHarness\n synthesizes executable code that prevents invalid interaction with an environment. The paper studies action filters that generate legal candidates, action verifiers that reject illegal model outputs, and code policies that can choose actions without invoking the model at decision time. A language model mutates candidate programs, while search uses legality and reward feedback to decide what to refine.\n\nThe important point is not simply “the artifact is code.” AutoHarness externalizes deterministic rule tracking—a failure mode language models handle unreliably—into an executable mechanism. Its editable subset \\(\\Omega_H\\) is narrow, its environmental evidence \\(E\\) is comparatively hard, and legality can be tested directly. The trade-off is that the learned mechanism is usually environment-specific.\n\nMeta-Harness\n expands \\(\\Omega_H\\) from a local action function to a complete stateful program. Its Harness controls what to store, retrieve, and present to the model, allowing coupled changes to context construction, memory updates, tool use, and orchestration.\n\nIts distinctive design is the evidence interface exposed to the proposer. Instead of compressing each experiment into a short summary, Meta-Harness stores every candidate’s source, score, and raw execution traces in a filesystem. An agentic proposer inspects this history with ordinary file tools, creates a new program, and submits it to interface validation and task evaluation. A population supports Pareto selection when objectives such as accuracy and context cost conflict.\n\nFigure 2. Meta-Harness exposes the source, traces, and scores of all prior candidates through a filesystem rather than compressing them into a single prompt. Source: \nLee et al., 2026\n.\n\nEnd-to-end search can discover interactions that independent prompt edits would miss, but it is harder to attribute improvement when many behaviors are coupled inside one program.\n\nAgentic Harness Engineering\n, or AHE, addresses this problem with three forms of observability. \nComponent observability\n represents prompts, tools, middleware, Skills, sub-agent configuration, and memory as file-level edit units. \nExperience observability\n distills very long trajectories into layered evidence. \nDecision observability\n requires every edit to predict its intended fixes and possible regressions; the next evaluation compares those predictions with observed task-level changes and can roll back at file granularity.\n\nThe resulting conceptual expansion is\n\n\\[\n\\text{local executable constraint}\n\\rightarrow\n\\text{full stateful program}\n\\rightarrow\n\\text{componentized and observable scaffold}.\n\\]\nThis is not a direct paper lineage. It highlights a trade-off: end-to-end program search explores coupled strategies, while componentized engineering improves diagnosis, ownership, falsifiability, and rollback.\n\n4.3 Who Performs the Update: Self-Referential Harness Improvement\n\nThe next distinction concerns \\(U\\), not the representation of \\(H\\). Many optimizers rely on a separate, often stronger coding Agent. \nSelf-Harness\n instead asks whether the target model can identify its own model-specific weaknesses and propose changes to the Harness it will subsequently use.\n\nIts loop has three stages: \nWeakness Mining\n collects execution traces and clusters verifier-grounded failure patterns; \nHarness Proposal\n invokes the same target model in a proposer role to generate bounded modifications; \nProposal Validation\n tests candidates on both held-in and held-out splits. An edit is promoted only if it does not regress on either split and improves at least one.\n\nFigure 3. Self-Harness changes updater identity: the target model participates in reshaping its own operating Harness. Source: \nZhang et al., 2026\n.\n\nThis is self-referential at the updater coordinate, but it is not unconstrained self-approval. The evaluator, hidden split, acceptance rule, permission ceiling, and deployment authority remain outside editable state. Self-Harness changes \\(U\\) while protecting \\(V\\) and \\(B\\).\n\nThe trade-off is structural. The target model is well positioned to reveal model-specific failure modes a generic optimizer may miss. Its diagnoses are also bounded by the same competence and biases that produced those failures. The held-out gate is not incidental machinery; it is what converts self-proposed change into demonstrated improvement.\n\n4.4 What Evidence Drives the Update: Learning from Unlabeled Histories\n\nReal deployments accumulate trajectories, but users rarely provide a ground-truth score for every coding, research, or operational task. \nRetrospective Harness Optimization\n, or RHO, asks whether a Harness can improve from historical trajectories without validation labels during optimization.\n\nRHO selects a difficult and diverse coreset of past tasks, re-solves each task several times, and analyzes grouped rollouts. Self-validation identifies questionable tool calls, assumptions, and stopping decisions; self-consistency exposes consequential disagreement. The diagnostics guide several candidate updates, which are compared through pairwise self-preference. If no candidate has positive mean preference over the baseline, the Harness remains unchanged.\n\nFigure 4. RHO converts unlabeled deployment histories into grouped rollouts, diagnoses disagreement and errors, and selects a candidate through self-preference. Source: \nPan et al., 2026\n.\n\nThis changes the supervision regime, not the definition of a Harness. “No ground-truth labels during optimization” also does not remove the need for independent final evaluation. The paper reports its scientific results on held-out benchmark tests with external grading; its label-free claim applies to the optimization loop, not to final evaluation. Self-preference is a proxy for task utility. A model may prefer confidence, verbosity, or internal consistency even when an external objective would rank the behavior lower.\n\n4.5 When Does Improvement Happen? Reset-Free Online Adaptation\n\nThe methods above largely separate execution and improvement into rounds. \nContinual Harness: Online Adaptation for Self-Improving Foundation Agents\n adds the temporal axis: updates occur inside one ongoing trajectory without resetting the environment.\n\nIts running state is\n\n\\[\nH_t=(p_t,G_t,K_t,M_t),\n\\]\nwhere \\(p_t\\) is the system prompt, \\(G_t\\) the sub-agent collection, \\(K_t\\) the Skill library, and \\(M_t\\) persistent memory. A fixed meta-tool API bounds CRUD operations over these components. After a warm-up, a Refiner periodically reads a recent trajectory window and diagnoses navigation loops, failed tool calls, stalled objectives, missed exploration, and reusable successful action sequences. The edited Harness becomes active on the next interaction step.\n\n```\nsequenceDiagram\n    participant E as Persistent environment\n    participant A as Agent using H_t\n    participant R as Same-model Refiner\n    participant S as Component stores\n    A->>E: Act and observe\n    E-->>A: New state\n    A->>S: Append trajectory and memory\n    loop After warm-up, every F steps\n        R->>S: Read recent trajectory window\n        R->>R: Diagnose loops, failures, stalls, and reusable success\n        R->>S: Edit prompt, sub-agents, Skills, or memory\n        S-->>A: Activate H_(t+1) immediately\n    end\n    A->>E: Continue without reset\n```\n\nThe Agent and Refiner roles use the same foundation model, though with different contexts and invocation times. Continual Harness therefore overlaps with full-scaffold evolution in \\(\\Omega_H\\), same-model self-improvement in \\(U\\), and trajectory-driven learning in \\(E\\). Its distinctive contribution is \\(T\\): failure, repair, and reuse occur in the same persistent run.\n\nThat temporal regime complicates validation. The online loop applies edits in place rather than running a held-out acceptance test for every candidate. This statement is an inference from the reported method, not a separate claim by the authors. Future behavior mixes the effects of the changed Harness, advancing environment state, accumulated memory, and changing task difficulty. The paper reports successful in-loop Skill repair, but its results also suggest a method- and domain-dependent capability floor: a weaker model can underperform the minimalist baseline because it cannot reliably exploit or improve the growing scaffold.\n\nThe paper includes a second loop at the boundary of pure Harness evolution. Rollout windows from an open-source model are scored by a process reward model, low-reward behavior is relabeled by a frontier teacher, and soft supervised fine-tuning updates the model before the next reset-free training iteration. The environment persists while both \\(\\theta\\) and \\(H\\) change at different timescales.\n\nThis is best described as model–Harness co-learning. The primary loop updates \\(H\\) through online refinement; the outer loop updates \\((\\theta,H)\\) and depends on an external teacher. The \nofficial implementation\n exposes the Refiner and persistent component stores.\n\n4.6 Synthesis: Harness Evolution versus Agent Self-Reference\n\nThese papers do not form a single ladder; they change different coordinates.\n\nSystem\n\nEditable \\(\\Omega_H\\)\n\nUpdater \\(U\\)\n\nProposal evidence \\(E\\)\n\nActivation \\(T\\)\n\nEvaluation \\(V\\)\n\nAutoHarness\n\nAction verifier, filter, or code policy\n\nExternal synthesis/search loop\n\nLegality and reward feedback\n\nBetween search trials\n\nEnvironment legality and task reward\n\nMeta-Harness\n\nComplete stateful program\n\nAgentic proposer\n\nPrior source, scores, and raw traces\n\nBetween evaluations\n\nInterface checks, task score, Pareto selection\n\nAHE\n\nFile-level scaffold components\n\nEvolving coding Agent\n\nLayered traces and change manifests\n\nRound-based\n\nPredicted-versus-observed effects and rollback\n\nSelf-Harness\n\nBounded model-specific deltas\n\nTarget model in proposer role\n\nVerifier-grounded failure clusters\n\nStaged candidate\n\nHeld-in and held-out regression gate\n\nRHO\n\nPersistent tools, prompts, and Skills\n\nSame backbone in several roles\n\nUnlabeled history and grouped rollouts\n\nRetrospective, between deployments\n\nSelf-preference internally; held-out external benchmark for the scientific claim\n\nContinual Harness\n\nPrompt, sub-agents, Skills, and memory\n\nSame-model Refiner\n\nRecent trajectory window\n\nMid-trajectory, in place\n\nNo per-edit held-out gate is reported; later trajectory outcomes\n\nAcross all six systems, \\(B\\) should protect permission ceilings, credentials, evaluator integrity, resource limits, audit, and rollback authority. Their search topology \\(L\\) and parameter-coupling mode \\(P_\\theta\\) remain additional overlays.\n\nTwo conclusions follow.\n\nFirst, \nHarness evolution\n and \nself-referential Agent evolution\n answer different questions. Harness evolution asks which model-external execution surface changes. Self-reference asks whether the target system participates in constructing its successor. A method can belong to both.\n\nSecond, broadening the editable surface is only one direction of progress:\n\n\\[\n\\text{narrow code}\n\\rightarrow\n\\text{full scaffold}\n\\rightarrow\n\\text{observable components}.\n\\]\nUpdater identity, evidence, timing, validation, and governance remain orthogonal. Weakening external supervision or activating edits faster may increase autonomy, but it also increases the burden on evaluation, attribution, checkpointing, rollback, and immutable trust boundaries.\n\nJoint model–Harness learning pushes this burden further. It must distinguish gains caused by parameter updates, Harness updates, teacher relabeling, and a changing trajectory distribution. The central question is not whether an Agent can rewrite its scaffold. It is whether useful changes can accumulate while \\(V\\) remains informative and \\(B\\) remains outside the system’s ability to redefine success.\n\n5. Future Challenges\n\nThe reported studies demonstrate, within their benchmark and deployment settings, that an Agent can turn experience into persistent changes to prompts, workflows, Skills, code, and Harness state. Three bottlenecks determine whether those loops become reliable learning systems rather than increasingly elaborate self-editors:\n\nIs the system optimizing the right objective when ground truth is absent?\n\nDoes an update that passes a local test remain useful over time?\n\nHow should fast external adaptation interact with slower parameter learning?\n\nThese questions are sequential. A durable update is not valuable if the objective is wrong, and a good objective is not enough if improvements decay or the learning state becomes impossible to maintain.\n\n5.1 Improving Toward What? Evaluation Without Ground Truth\n\nWhen ground truth exists, the basic protocol is familiar: use one split to generate or select updates, keep another split hidden for evaluation, and prevent the updater from editing the scoring path. The difficult case is open-world deployment, where outcomes may be delayed, partially observed, multi-objective, or never reduced to a clean reward.\n\n“No reward” does not necessarily mean “no evidence.” A real system may still observe hard constraints, task completion artifacts, user corrections, downstream consequences, disagreement among independent evaluators, or whether a decision survives later review. The problem is that these signals answer different questions and may all be biased.\n\nEvidence source\n\nWhat it can establish\n\nMain failure mode\n\nHard constraints and executable checks\n\nValidity, safety invariants, interface compliance\n\nNecessary behavior may still be low quality\n\nDelayed environment or user outcomes\n\nWhether behavior eventually produced value\n\nSparse feedback, confounding, and long delay\n\nHuman or user preference\n\nQuality dimensions that are hard to formalize\n\nInconsistency, cost, and preference manipulation\n\nMultiple independent critics or models\n\nDisagreement and uncertainty\n\nCorrelated model bias and shared blind spots\n\nSelf-generated tests and self-preference\n\nDense diagnostic signal for search\n\nCircular self-approval and Goodharting\n\nCounterfactual or causal comparison\n\nWhether the update, rather than context, caused the gain\n\nExpensive or impossible in a changing environment\n\nThe key design rule is to separate \nproposal evidence\n from \npromotion authority\n. Self-reflection, self-preference, generated tests, and surrogate verifiers can guide candidate search. They should not become the sole reason a high-impact update is deployed. Promotion should combine the strongest available external anchors and retain an uncertainty-aware option to abstain, defer, or request human review.\n\nIn many environments there is also no single “correct direction.” Task utility, cost, latency, safety, user preference, maintainability, and exploration may conflict. Collapsing them into one reward hides value judgments and invites reward hacking. A better formulation maintains a vector objective or Pareto frontier and makes the protected constraints explicit:\n\n\\[\nJ(H)=\\bigl(\n\\text{task utility},\n-\\text{cost},\n-\\text{risk},\n\\text{maintainability}\n\\bigr).\n\\]\nThe central research question is therefore not merely how to synthesize a reward. It is how to construct a \nfalsifiable evaluation contract\n: what evidence would show that the proposed direction is wrong, which signals are allowed to adapt, and which authority remains outside the self-improvement loop?\n\n5.2 From Local Validation to Durable Improvement\n\nSuppose the first problem is solved and a candidate passes its validation gate. That still proves only a local statement: the update helped on the sampled tasks, under the current model, Harness, environment, and budget. It does not show that the change will remain useful after the task distribution shifts, other components evolve, users behave differently, or the artifact library grows.\n\nLocal success can fail to become durable improvement in at least four ways:\n\nevaluation overfitting:\n the edit memorizes recurring benchmark structure or exploits a verifier blind spot;\n\ndistribution shift:\n future tasks exercise capabilities absent from the validation set;\n\ncomponent interference:\n a later Skill, model, or routing change invalidates the original gain;\n\nmaintenance debt:\n local patches accumulate until context cost, coupling, or regression risk outweighs their benefit.\n\nDurability must therefore be evaluated at multiple horizons.\n\nHorizon\n\nQuestion\n\nAppropriate evidence\n\nCandidate-level\n\nDoes the delta fix its target failure without an immediate regression?\n\nHeld-in tests, held-out tests, interface and safety checks\n\nDistribution-level\n\nDoes the gain transfer beyond the update examples?\n\nHidden task families, cross-model or cross-environment tests\n\nTemporal deployment\n\nDoes it remain useful on later data and users?\n\nTemporal holdouts, shadow runs, canaries, longitudinal A/B tests\n\nSystem lifecycle\n\nDoes it compose with later updates and remain maintainable?\n\nDependency-aware regression, cost and artifact-growth tracking, rollback history\n\nThis implies a staged state machine rather than one binary gate:\n\n\\[\n\\text{proposed}\n\\rightarrow\n\\text{safe archive candidate}\n\\rightarrow\n\\text{evaluated}\n\\rightarrow\n\\text{canary}\n\\rightarrow\n\\text{promoted}\n\\rightarrow\n\\text{monitored or rolled back}.\n\\]\nArchive admission and deployment promotion should remain distinct. An unusual or temporarily low-scoring candidate may be retained as a stepping stone if it is runnable, provenance-complete, and safe to store. Deployment requires stronger utility and regression evidence.\n\nReset-free online adaptation makes this problem sharper. After an in-place Harness edit, the environment has advanced, memory has grown, and the next task state may be harder or easier. A later success cannot automatically be attributed to the edit. Where replay is possible, synchronized checkpoints and shadow Harnesses can support counterfactual comparison. Where it is not, long-horizon monitoring, uncertainty estimates, and fast rollback become part of the algorithm rather than optional deployment infrastructure.\n\nThe unit of evaluation should ultimately be a performance trajectory, not a one-time score. A durable improvement raises future utility after accounting for cost, regressions, and maintenance burden across a meaningful time window.\n\n5.3 Coordinating Parametric Learning and External Editable State\n\nExternal state is attractive because it is fast to edit, inspect, transfer, and roll back. Model parameters are compact at inference time and can internalize patterns that should generalize broadly. Treating either as the only learning substrate creates a failure mode.\n\nThe concern that external state will inevitably grow is slightly too strong: a disciplined lifecycle can merge, compress, prune, garbage-collect, and retire Skills or Harness components. But without those operations, append-only external learning does tend toward duplicated procedures, longer context, conflicting rules, rising dependency cost, and a system that neither humans nor Agents can safely maintain.\n\nParameter learning is not an automatic cure. Consolidating weak or biased external experience into weights can make errors harder to inspect and roll back, introduce catastrophic forgetting, and change the trajectory distribution on which the current Harness was optimized. The problem is therefore one of \nallocation and timescale\n, not a choice between weights and files.\n\nA promising architecture treats external state as fast, reversible memory and model parameters as slow, carefully consolidated memory:\n\n```\nflowchart LR\n    E[\"New experience\"] --> F[\"Fast external update<br/>prompt, Skill, memory, Harness\"]\n    F --> V[\"Cross-task and temporal validation\"]\n    V -->|\"unstable or narrow\"| P[\"Keep external, revise, merge, or prune\"]\n    P --> F\n    V -->|\"stable and broadly reusable\"| D[\"Distill verified trajectories and contracts\"]\n    D --> T[\"Slow parameter update\"]\n    T --> R[\"Regression, safety, and capability evaluation\"]\n    R -->|\"reject\"| F\n    R -->|\"accept\"| M[\"Deploy new model version\"]\n    M --> C[\"Re-evaluate and simplify external state\"]\n    C --> F\n```\n\nThis loop raises four concrete research questions:\n\nWhat should be internalized?\n Repeated, cross-task, stable procedures are stronger candidates than environment-specific rules or rapidly changing facts.\n\nWhen is evidence mature enough?\n Consolidation should require broader and longer-lived validation than an ordinary external edit.\n\nHow is attribution preserved?\n Training examples should retain links to the external artifacts, trajectories, tests, and decisions from which they were distilled.\n\nHow is the Harness re-based after a model update?\n A stronger or behaviorally different model may make old prompts redundant, invalidate routing assumptions, or interact badly with accumulated Skills.\n\nA clean co-learning experiment should compare \\(\\theta\\)-only, \\(H\\)-only, and joint \\((\\theta,H)\\) updates from the same environment checkpoints under matched interaction and compute budgets. Teacher relabeling should be ablated on and off. Evaluation should report not only task utility but also external-state size, regression rate, adaptation cost, and how much of the joint gain remains after either the model or Harness is reset to its previous version.\n\nContinual Harness offers an early boundary example because it combines online Harness refinement with a separate model-update loop. It does not settle the allocation problem: its co-learning path depends on a process reward model and an external frontier teacher, while attribution between model and Harness gains remains difficult.\n\nThe desired endpoint is not an ever-growing external repository or a model that periodically absorbs everything. It is a \nmemory hierarchy with consolidation\n: external state absorbs experience quickly and transparently; repeated validated structure is compressed into more stable representations, including parameters when appropriate; redundant external state is simplified; and every consolidation step is independently regression-tested.\n\nTogether, the three challenges define a stricter standard for self-improvement: the system must know what evidence points in the right direction, demonstrate that a change survives beyond its local gate, and maintain a healthy division of labor between fast editable state and slow parametric learning.\n\n6. References\n\n[1] Anthropic. 2025. \nEquipping Agents for the Real World with Agent Skills\n.\n\n[2] Genewein, T., et al. 2026. \nFrom AGI to ASI\n.\n\n[3] Agrawal, L., et al. 2025. \nGEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning\n.\n\n[4] Zhang, J., et al. 2024. \nAFlow: Automating Agentic Workflow Generation\n.\n\n[5] Hu, S., et al. 2024. \nAutomated Design of Agentic Systems\n.\n\n[6] Robeyns, M., et al. 2025. \nA Self-Improving Coding Agent\n.\n\n[7] Zhang, J., et al. 2025. \nDarwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents\n. \nCode\n.\n\n[8] Zhang, J., et al. 2026. \nHyperagents\n. \nCode\n.\n\n[9] Weng, J. 2026. \nLearning Beyond Gradients\n. \nCode\n.\n\n[10] Novikov, A., et al. 2025. \nAlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery\n.\n\n[11] Alzubi, S., et al. 2026. \nEvoSkill: Automated Skill Discovery for Multi-Agent Systems\n.\n\n[12] Ni, J., et al. 2026. \nTrace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills\n.\n\n[13] Yang, Y., et al. 2026. \nSkillOpt: Executive Strategy for Self-Evolving Agent Skills\n.\n\n[14] Lin, H., et al. 2026. \nMUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation\n.\n\n[15] Zhang, H., et al. 2026. \nCoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification\n.\n\n[16] Ma, Z., et al. 2026. \nSkillClaw: Let Skills Evolve Collectively with Agentic Evolver\n. \nCode\n.\n\n[17] Yan, Z., et al. 2026. \nOpenSkill: Open-World Self-Evolution for LLM Agents\n. \nCode\n.\n\n[18] Lou, X., et al. 2026. \nAutoHarness: Improving LLM Agents by Automatically Synthesizing a Code Harness\n.\n\n[19] Lee, Y., et al. 2026. \nMeta-Harness: End-to-End Optimization of Model Harnesses\n.\n\n[20] Lin, J., et al. 2026. \nAgentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses\n.\n\n[21] Zhang, H., et al. 2026. \nSelf-Harness: Harnesses That Improve Themselves\n.\n\n[22] Pan, W., et al. 2026. \nEvolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference\n. \nCode\n.\n\n[23] Karten, S., et al. 2026. \nContinual Harness: Online Adaptation for Self-Improving Foundation Agents\n. \nCode\n.\n\ncomments powered by \nDisqus", "url": "https://wpnews.pro/news/self-improving-agent-systems-a-unified-view", "canonical_source": "https://yigengjiang.github.io/posts/self_improving_agent_systems/", "published_at": "2026-07-13 02:13:12+00:00", "updated_at": "2026-07-13 02:35:11.861245+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "machine-learning", "ai-research"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/self-improving-agent-systems-a-unified-view", "markdown": "https://wpnews.pro/news/self-improving-agent-systems-a-unified-view.md", "text": "https://wpnews.pro/news/self-improving-agent-systems-a-unified-view.txt", "jsonld": "https://wpnews.pro/news/self-improving-agent-systems-a-unified-view.jsonld"}}