Self-Improving Agent Systems: A Unified View of Algorithms, Skills, and Harnesses
yigeng & GPT 5.6
2026-07-12
{ Machine Learning }
[self-improvement,
Agent]
Self-improvement is often discussed as if it meant a model rewriting its own weights. That is one possible endpoint, but it is not where most practical progress is happening today. Deployed agents are compound systems: a foundation model is surrounded by prompts, tools, workflows, memory, Skills, sub-agents, evaluators, and runtime policies. Changing any persistent part of that system can change what the agent is able to do next.
This article develops one framework for reading a fast-growing and easily fragmented literature. The central claim is simple:
Self-improvement is an evidence-driven, persistent state transition whose effect on future behavior can be evaluated.
The primary difference between prompt evolution, workflow evolution, Skill evolution, Harness evolution, and self-referential agent evolution is therefore not how ambitious their names sound. It is what state they allow to change . A second, independent question is who constructs the successor . Evidence, search topology, update timing, and governance form additional axes.
The result is a deliberately bounded account of self-improvement. Most systems covered here keep the foundation model frozen and improve model-external state. Joint model–Harness adaptation appears only as a boundary case. The paper-to-paper arrows below should be read as conceptual relationships unless an explicit methodological inheritance is stated.
- Preliminaries
1.1 Scope, Thesis, and Unit of Analysis
The unit of analysis in this article is not an isolated language model. It is a deployed agent system together with the machinery that proposes, evaluates, archives, and promotes changes.
For predominantly frozen-model settings, a useful engineering decomposition is
[ A_t=(\theta_t,H_t), ] where (\theta_t) denotes model parameters and (H_t) denotes persistent model-external state. In most of the systems discussed here, (\theta_{t+1}=\theta_t) while (H_{t+1}\neq H_t). The model is unchanged, yet the behavior of the deployed system may improve because its instructions, control flow, tools, reusable capabilities, or runtime have changed.
This framing excludes two broad areas from the main discussion. First, parameter-only methods such as ordinary fine-tuning are outside scope unless they are explicitly coupled to Harness adaptation. Second, one-off reflection, best-of-(N) answer selection, and temporary in-context adaptation are not persistent self-improvement by themselves. They may generate useful evidence, but they do not automatically create a better future system.
It is helpful to describe the whole improvement system as
[ \mathcal{E}_t=(A_t,U_t,L_t;V,B), ] and then separate its functional roles:
Symbol
Role
Typical realizations
(A_t=(\theta_t,H_t))
Active agent system
Model plus prompts, workflow, tools, Skills, memory, and runtime
(U_t)
Updater
Prompt optimizer, coding agent, Refiner, MetaAgent
(L_t)
Lineage or archive
Best-so-far state, Pareto pool, search tree, population, open archive
(V)
Evaluation and promotion
Tests, validation tasks, preference gates, rollout metrics
(B)
Governance boundary
Permission enforcement, credential boundaries, sandboxing, budgets, provenance, audit, rollback
This is a functional decomposition, not a claim that the sets are always disjoint. Hyperagents, for example, embeds part of the update mechanism inside its editable program, so (U_{\text{embedded}}\subseteq H_t), while parent selection, evaluation, and other outer-loop functions remain external. The distinction still matters: a system can expose a broad editable surface while keeping its evaluator and authorization enforcement protected. If the candidate can rewrite both its behavior and the standard used to judge that behavior, a higher score no longer establishes improvement.
1.2 Agent Systems as Persistent Editable State
A deployed agent is better viewed as a stateful program than as a single model call. The model produces actions, but the surrounding system decides what context reaches it, which tools exist, what results persist, how errors are recovered, and when another model call is made.
flowchart LR
X["Task or environment"] --> H["Harness H_t"]
T["Frozen or slowly changing model θ_t"] --> H
H --> Y["Actions, artifacts, and trajectory τ_t"]
Y --> E["Evidence: outcomes, traces, failures, cost"]
E --> U["Updater U_t"]
U --> C["Candidate persistent state H'"]
C --> V["Protected evaluation V"]
V -->|"promote"| H
V -->|"admit as safe candidate"| L["Lineage / archive L_t"]
V -->|"invalid or unsafe"| Q["Rejected-candidate buffer"]
B["Governance B"] -.->|constrains edits and deployment| U
B -.->|protects| V
This diagram already reveals two different meanings of “self.” At the system level, the deployed system improves when some persistent part of it changes. At the stricter self-referential level, the current agent, a parent version, or an editable MetaAgent participates in constructing its successor. The first does not imply the second.
It also reveals why edit scope cannot be treated as a single measure of autonomy. A fixed external optimizer may search over an entire agent codebase, while a self-referential updater may be allowed to change only one prompt. Breadth of the target and identity of the updater are independent.
1.3 What Is a Skill?
Anthropic defines Agent Skills operationally as organized folders of instructions, scripts, and resources that an agent can discover and load dynamically. At minimum, a Skill is a directory centered on a SKILL.md file containing metadata and instructions. More complex Skills may include scripts, references, templates, or assets. Their key systems idea is progressive disclosure : lightweight metadata supports discovery, the main instructions are loaded when relevant, and additional files are accessed only when needed ( Anthropic, 2025 ).
For this article, a Skill is a reusable, persistent, and portable capability artifact with five conceptual parts:
Part
Question
Trigger and metadata
When should the agent discover or invoke it?
Procedure
What workflow, heuristics, or domain rules should it follow?
Resources
Which scripts, references, assets, or dependencies support execution?
Validation contract
How can correct or useful behavior be checked?
Version and provenance
Where did it come from, what changed, and how can it be rolled back?
A Skill is not merely a prompt because it is intended to be discoverable, reusable, and lifecycle-managed. It is not identical to a tool because it may teach the agent when and how to compose multiple tools. It is not the whole Harness because it normally packages a task-scoped capability rather than controlling the global execution runtime.
The word “skill” is used much more broadly elsewhere in the literature. To keep the comparison coherent, this article does not use it to mean an implicit memory policy, a repository operating system, or a multi-agent orchestration policy.
1.4 What Is a Harness?
A Harness is the model-external scaffold that determines what the model sees, what actions it can take, how state is maintained, and how execution is orchestrated. Depending on the system, its editable surfaces may include:
system prompts and context construction;
tools, tool schemas, middleware, and action filters;
Skills and Skill retrieval;
memory, files, logs, and artifact stores;
sub-agent definitions and delegation;
workflow, retries, stopping rules, and error recovery;
permission-request policies, capability routing, and runtime policies.
The shorthand “Agent (\approx) Model + Harness” is useful engineering intuition, not a literal identity. A Harness is defined here by its system role, not by its file format. It may be represented as text, configuration, a graph, a directory, executable code, or a mixture of these.
That definition also requires a boundary. A Harness may decide when to request a capability or which tool route to use, but credential enforcement, final authorization rules, hidden tests, immutable scoring paths, sandboxing, audit logs, and ultimate rollback authority should not be editable by the candidate at the same optimization level. They belong to the governance envelope (B), even if implemented in code next to the Harness.
1.5 Self-Improvement, Evolution, and Self-Reference
The terminology in this area is overloaded, so I will use three terms with different scopes.
Term
Meaning in this article
Self-improvement
The broad research program of using feedback to create persistent system changes intended to improve future behavior; a demonstrated improvement additionally requires independent outcome evidence
Evolutionary search
The subset of this program that explicitly uses variation, selection, lineages, populations, Pareto sets, search trees, or open archives
Self-reference
The target agent, a parent version, or an editable MetaAgent participates in constructing its successor
“Skill Evolution” and “Harness Evolution” are retained as broad field labels in the chapter titles because the literature uses them that way; not every included method implements evolutionary search in the stricter algorithmic sense. These definitions allow component-scoped methods to be described precisely. Prompt and Skill evolution expose comparatively narrow edit surfaces; Harness evolution may range from one executable constraint to the full model-external scaffold. Narrow scope does not imply weak impact. Likewise, a broad program search is not necessarily self-referential if the optimizer that produces the target remains fixed.
There is therefore no defensible one-dimensional ladder from “low” to “high” self-evolution. Edit scope, updater identity, evidence quality, search topology, update timing, parameter coupling, and outer-loop mutability are separate dimensions. The rest of the article uses those dimensions as a roadmap.
- Self-Improving Agent Systems
2.1 Why Self-Improvement Matters
Modern foundation models are rarely deployed alone. Their behavior depends on prompts, workflows, tools, memory, context management, sub-agents, and the runtime that connects them. Improving an agent therefore does not necessarily require changing model weights. For closed-weight systems in particular, persistent model-external state is often the only practical learning substrate available to application builders.
This matters economically and scientifically. A transient insight is paid for again in every context window. A persistent improvement can be reused across tasks, users, and deployments. External artifacts are also comparatively easy to inspect, diff, test, transfer, and roll back. In the near term, this makes model-external self-improvement less like science fiction and more like automated systems engineering.
The long-term motivation should still be stated carefully. Google DeepMind’s report From AGI to ASI identifies recursive improvement as one of four possible pathways from AGI to ASI, alongside further scaling, paradigm shifts, and large-scale multi-agent collectives. The report also emphasizes frictions and open questions along each pathway. It is therefore evidence that recursive improvement is a serious research hypothesis—not evidence that today’s agent loops are sufficient, monotonic, or destined to produce ASI.
The papers in this chapter study bounded versions of a more concrete question:
What must be editable, observable, and independently verifiable for useful changes to accumulate?
Most freeze the foundation model and modify prompts, workflows, or agent code. Some use a fixed optimizer; others let the current agent or one of its historical versions construct a successor. Automated optimization is not automatically self-reference, and self-reference is not automatically recursive intelligence amplification.
2.2 An Operational Definition and the Minimal Loop
Two statements should be kept separate:
Self-improvement process. Evidence from executions, evaluations, or the environment produces a persistent candidate change to editable state with the aim of improving future behavior.
Validated improvement claim. Independent outcome evaluation shows that the changed system improves future behavior relative to an appropriate baseline.
The first describes an update process; the second establishes that the process succeeded. Before the second condition is met, “self-modification” or “adaptation” is the more precise description of an individual change.
The framework has four requirements:
Persistence. The change survives beyond the immediate answer or trajectory.
Evidence. Execution, evaluation, or environmental feedback informs the change.
An effective update operator. Some mechanism converts evidence into a bounded candidate delta.
Outcome evaluation. Future behavior is compared with a relevant baseline before improvement is claimed.
The minimal loop is:
flowchart LR
Z["Persistent editable state Z_t"] --> X["Execute tasks"]
X --> E["Collect traces, outcomes, cost, and failures"]
E --> D["Diagnose reusable patterns"]
D --> P["Propose a bounded update"]
P --> M{"Activation mode"}
M -->|"candidate or shadow"| V["Independent evaluation"]
M -->|"in place"| N["Next live interaction"]
V -->|"reject or roll back"| Z
V -->|"archive"| L["Candidate archive / lineage"]
V -->|"promote"| Z2["Active state Z_(t+1)"]
N --> E
L -->|"future parent"| D
Z2 --> X
This loop separates several operations that are often conflated:
Candidate generation is not acceptance. An edit shows that a system can change, not that it improved.
Archive admission is not deployment promotion. A low-scoring branch may be valuable as a stepping stone without being safe enough to serve users.
Diagnostic feedback is not final authority. Adaptive critics can guide search, but a protected evaluation path must anchor deployment claims.
Immediate online edits are possible. A held-out gate for every update is the stronger deployment design, not a descriptive requirement satisfied by every paper. When an edit enters a live trajectory directly, it is an adaptation until later outcome evidence supports an improvement claim; recovery mechanisms become more important.
A one-off reflection that improves the current answer may be useful adaptation, but it is not persistent self-improvement under this definition. Best-of-(N) sampling chooses an answer, not a successor system. Similarly, generating one prompt, Skill, or program is construction unless subsequent evidence can revise it.
2.3 A Two-Axis Roadmap with Orthogonal Overlays
The most stable primary descriptor is the editable set , not a linear ladder of artifact names. Let
[ H_t={ \text{prompts, workflows, tools, Skills, memory, context, runtime} } ] denote the model-external scaffold. A method declares an editable subset
[ \Omega_{\text{method}}\subseteq\operatorname{components}(\theta,H,U,L), ] while evaluation (V) and governance (B) normally remain protected. The artifact representation—text, graph, directory, configuration, or executable code—is orthogonal to the semantic update object.
Within the Agent-design branch, prompt (\rightarrow) workflow (\rightarrow) code-defined target Agent (\rightarrow) full Agent scaffold remains a useful analytical edit-surface narrative. It is not the taxonomy for the whole article. Skills are portable components that may sit inside a Harness; a broad Agent program and a broad Harness can overlap in implementation. Chapters 3 and 4 therefore re-index the same framework around declared Skill and Harness boundaries rather than continuing a size hierarchy.
The second axis is the updater–target relationship :
[ \text{fixed external optimizer} \rightarrow \text{lineage member edits a successor} \rightarrow \text{editable MetaAgent}. ] Four further overlays should remain orthogonal:
Evidence and evaluation: labels, tests, execution traces, opaque oracles, self-preference, deployment outcomes, or open-world verification.
Search topology: greedy replacement, best-history search, Pareto pools, MCTS, populations, or open archives.
Update timing: between experiments, between deployments, or in place during one continuous trajectory.
Parameter coupling: frozen model, alternating model–Harness updates, or joint adaptation of ((\theta,H)).
The following table locates the systems in this chapter. It is a mechanism-level map, not a claim that each paper inherited from the preceding one.
System
Primary persistent update
Updater–target relationship
Search and evidence
Timing
GEPA
Prompts in a compound AI system
Fixed prompt optimizer
Trace reflection; Pareto pool
Between evaluations
AFlow
Workflow nodes, operators, edges, and prompts
Fixed workflow optimizer
MCTS; execution score
Between evaluations
ADAS
Code-defined target agent
Fixed MetaAgent
Archive; validation score; runtime repair
Between evaluations
SICA
Complete coding-agent framework
Historical Agent generates a successor
Best-history selection; score, cost, time
Episodic
DGM
Coding-agent codebase
Archived parent produces a descendant
Open archive; empirical benchmark validation
Episodic
Hyperagents
TaskAgent and MetaAgent in one program
Editable MetaAgent modifies itself and TaskAgent
DGM-style archive; multi-domain evaluation
Episodic
Learning Beyond Gradients
Heuristic software system
Coding agent maintains external program state
Reward, logs, replay, tests
Iterative
AlphaEvolve
External algorithm or program
Fixed evolutionary coding pipeline
Program database; automated evaluators
Episodic
Update object constrains the search space and the available validation tools, but does not determine them. Code can be compiled and unit-tested, yet code evolution may still use weak benchmarks. A prompt may be hard to interpret causally, yet a protected held-out evaluation can provide a strong promotion gate.
2.4 Evolving Agent Artifacts with a Fixed Optimizer
The first family treats an agent artifact as an optimizable object while keeping the optimizer outside the target lineage.
GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning operates at the prompt layer. It samples trajectories from a compound AI system—including reasoning, tool calls, tool outputs, and evaluator feedback—then asks a reflection model to diagnose failures in natural language and propose prompt updates. Complementary candidates are retained through Pareto selection. The central insight is that a trace contains richer credit-assignment information than a final scalar reward. In this taxonomy, GEPA is persistent, feedback-driven component evolution, but its prompt optimizer is not itself being evolved.
AFlow: Automating Agentic Workflow Generation widens the editable surface from instructions to control flow. It represents an agentic workflow as LLM-invoking nodes connected by code-defined edges, then uses Monte Carlo Tree Search to modify prompts, operators, and execution structure. Each search node is a runnable workflow. The move from GEPA to AFlow is therefore a conceptual expansion from text to executable orchestration, not a documented implementation lineage.
Automated Design of Agentic Systems , or ADAS, widens the space again by representing target agents as code. Meta Agent Search reads an archive of earlier designs, proposes and implements a new agent, repairs runtime errors, and evaluates the result. Code can express prompts, tools, workflows, verification, and multi-agent interaction, so ADAS should not be reduced to workflow search.
The common boundary is decisive: the optimized artifact changes, but the optimizer that designs it remains fixed. ADAS may search a broader target than a self-editing prompt method while still being less self-referential.
2.5 From Agent Design to Self-Referential Evolution
The transition to self-reference is mainly a change in updater identity. The question is no longer only whether an optimizer can discover a better agent, but whether an agent from the evolving lineage can participate in producing its descendant.
A Self-Improving Coding Agent , abbreviated SICA, treats a complete coding-agent framework as editable software. Historical versions are evaluated, and a high-utility Agent analyzes the archive and implements the next candidate. The editable surface can include tools, sub-agent use, prompts, context management, and control logic. Its utility includes cost and execution time in addition to benchmark performance, discouraging “improvements” that merely spend more.
The conceptual move from ADAS to SICA is not “workflow becomes program”; both already use code. It is fixed MetaAgent designs a target (\rightarrow) an Agent from the lineage constructs its successor . In its experimental setting, SICA shows that a multi-component coding-agent framework can become its own software-engineering object.
The Darwin Gödel Machine , or DGM, adds open-ended search over multiple lineages. It samples parents from an archive of coding agents, modifies their code, empirically evaluates descendants, and retains a growing tree of candidates. A temporarily weak branch can remain available as a stepping stone instead of forcing every generation to descend from the current best.
It is useful to compare SICA with DGM as best-history search versus a branching open archive. That relationship is analytical; it should not be read as a verified implementation inheritance. DGM’s own ablations support the value of both self-modification and open-ended exploration within its experimental setting.
The Hyperagents paper motivates coding as a favorable first domain for self-reference because task performance and self-modification draw on the same broad competence: software engineering. A coding Agent that becomes better at understanding repositories may also become better at modifying its own scaffold. This is a structural motivation, not a generally proven law, and the alignment does not hold automatically elsewhere. A paper-review Agent can become better at review without becoming better at editing the system that implements review.
Hyperagents explicitly extends DGM to address this limitation. It combines a TaskAgent, which solves domain tasks, with a MetaAgent, which reads evaluation evidence and modifies both the TaskAgent and itself. Both live in one editable program. The lineage can therefore accumulate better task behavior and better mechanisms for diagnosing evaluations, maintaining memory, tracking performance, and generating future changes.
The paper instantiates this idea as DGM-Hyperagents, making DGM (\rightarrow) Hyperagents the clearest methodological continuation in this chapter. Its contribution is deeper than adding non-coding domains: the procedure responsible for producing descendants becomes part of the editable state, and cross-domain experiments test whether that improvement procedure can transfer.
Even here, “recursive self-improvement” should be used carefully. Evaluators, task distributions, resource limits, sandboxing, and parts of parent selection remain external. Hyperagents make more of the updater editable; they do not eliminate the trust root.
2.6 Beyond the Agent Boundary
The same mutation–evaluation loop can improve programs that are not the Agent’s own scaffold. These systems form an adjacent branch. They are not the next step in the DGM (\rightarrow) Hyperagents methodological continuation, and SICA is used only as a conceptual comparison to DGM.
Learning Beyond Gradients argues that a coding agent can make a heuristic software system learn by reading failures, editing policies and state detectors, adding tests, inspecting logs or replays, and retaining useful memory. The persistent state is not a neural network or merely an Agent class; it is a maintained software system whose future behavior changes through evidence-driven revision. Its evidence boundary should be kept explicit: this is an author project page and repository , not a peer-reviewed experimental paper.
AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery provides an evaluator-centric version of program evolution. An evolutionary coding pipeline samples from a program database, asks LLMs to generate code changes, runs automated evaluators, and returns scores and feedback to the database. Its applications include algorithm discovery and infrastructure optimization.
AlphaEvolve is adjacent to Agent self-improvement rather than a strict instance. The optimized program is normally an external solution, not the coding agent’s own execution or update machinery. Its relevance lies in transferable mechanisms: diff-based mutation, population management, multi-objective evaluation, and evaluator-first problem formulation.
A useful boundary test is:
If the evolved program persistently controls how the Agent will perceive, act, remember, or improve in future runs, it belongs to Agent or Harness evolution. If it is an external task solution, it is adjacent program evolution.
2.7 Synthesis and Transition to Component-Scoped Improvement
The Agent literature can be read as two interacting expansions.
flowchart LR
P["Prompt · GEPA"] -.->|conceptual edit-surface expansion| W["Workflow · AFlow"]
W -.->|conceptual edit-surface expansion| C["Code-defined target Agent · ADAS"]
C -.->|conceptual updater shift| S["Self-editing framework · SICA"]
S -.->|conceptual search-topology contrast| D["Open archive · DGM"]
D -->|"explicit methodological extension"| H["TaskAgent + editable MetaAgent · Hyperagents"]
X["Fixed external optimizer"] -.->|updater class| P
X -.->|updater class| W
X -.->|updater class| C
Y["Lineage member edits successor"] -.->|updater class| S
Y -.->|updater class| D
Z["Successor-generation mechanism is editable"] -.->|updater class| H
The conceptual relationships and their evidence status are:
Relationship used here
Evidence status
GEPA (\rightarrow) AFlow (\rightarrow) ADAS as widening edit scope
Analytical ordering, not claimed paper inheritance
ADAS (\rightarrow) SICA as fixed designer (\rightarrow) lineage self-edit
Analytical comparison
SICA and DGM: best-history versus branching archive
Conceptual contrast, not asserted implementation inheritance
DGM (\rightarrow) Hyperagents
Explicit methodological extension
Learning Beyond Gradients and AlphaEvolve
Adjacent expansions beyond Agent-scaffold evolution
This framing explains why Skill and Harness evolution deserve their own chapters. They re-index the framework around declared component boundaries and expose more specific engineering questions. A Skill is a portable capability artifact with a lifecycle. A Harness is the broader runtime that discovers, composes, executes, and governs those artifacts.
Conversely, several systems called Agent self-improvement—especially SICA, DGM, and Hyperagents—can also be described as broad Harness evolution when the model is frozen. The Agent label emphasizes who modifies whose successor . The Harness label emphasizes which model-external execution surfaces are editable . Keeping those questions separate is the key to unifying the literature without flattening it.
- Skill Evolution: Improving Externalized Capabilities
3.1 What Evolves: Skills as Persistent Artifacts
This chapter uses Skill in the narrow, artifact-centric sense established above. The important point is not the Markdown format itself, but that a capability is externalized into a persistent, reusable package instead of being reconstructed in every context window. Anthropic also explicitly anticipates agents eventually creating, editing, and evaluating Skills themselves ( Anthropic, 2025 ).
We can represent a versioned Skill as
[ S_t=(m_t,i_t,c_t,r_t,q_t,d_t,p_t), ] where (m_t) contains metadata and trigger conditions, (i_t) root instructions, (c_t) executable scripts, (r_t) references or assets, (q_t) tests and validation contracts, (d_t) dependencies, and (p_t) provenance and version history. Not every Skill needs every field, but making them explicit clarifies what an update may change and what must remain auditable.
Skill evolution is then a persistent transition:
[ \widetilde S_{t+1}=U(S_t,E_t), \qquad S_{t+1}= \begin{cases} \widetilde S_{t+1}, & \text{if }V(\widetilde S_{t+1},S_t)=1,\ S_t, & \text{otherwise}. \end{cases} ] Here (E_t) is evidence from failures, successful trajectories, external resources, tests, or deployment outcomes. (U) proposes a candidate, while (V) decides whether it should persist.
This makes a Skill parameter-like but not parametric . It changes future behavior and can accumulate across tasks, yet remains readable, portable, versionable, and rollbackable. The central question is not merely how to write a good instruction file. It is how an agent can discover, construct, validate, maintain, and deploy reusable external capability.
3.2 Where Skills Come From: Failures, Trajectories, and Open-World Evidence
Skill acquisition begins with abstraction. A trace contains task-specific actions, errors, and observations; a reusable Skill must extract what remains useful on future tasks. The following systems are best organized by the evidence available for that abstraction, not by publication order.
EvoSkill starts from labeled execution failures. It diagnoses a missing capability, proposes a new Skill or an edit to an existing one, and materializes the proposal as a structured Skill folder. Its key move is to convert “the Agent failed this task” into “the Agent lacks this reusable procedure.” Candidate programs are retained on a Pareto frontier when their Skills improve held-out performance.
Trace2Skill addresses a different bottleneck: one trajectory rarely contains enough evidence to define a robust procedure. It analyzes a diverse pool of successful and failed executions in parallel, extracts trajectory-local lessons, and consolidates recurring failures and workarounds into a coherent Skill directory. The result is not raw episodic memory retrieved at inference time, but a declarative artifact intended to transfer across tasks and models. Discovery identifies what is missing ; distillation determines how repeated experience should be compressed .
Within this article’s minimal loop, Trace2Skill primarily contributes the acquisition and distillation stage. The paper need not be read as instantiating a full recurrent deployment loop in which every produced artifact is repeatedly revised by later feedback.
CoEvoSkills moves to settings in which a complex multi-file Skill must be built while the true evaluator exposes only opaque pass/fail outcomes. A Skill generator iteratively revises the artifact, while an information-isolated surrogate verifier develops executable checks and provides denser diagnostics. If the surrogate passes but the external oracle fails, the verifier—not only the Skill—is updated. Acquisition becomes inseparable from learning how to test.
OpenSkill removes still more of the assumed learning loop. During construction, it does not rely on curated Skills, successful target-task trajectories, target answers, or a ready-made verifier. It retrieves knowledge and verification anchors from documentation, repositories, and the web, synthesizes candidate Skills, and builds virtual tasks for refinement. Target-task ground truth is reserved for final evaluation.
Figure 1. OpenSkill uses open-world sources to build both a Skill and a leakage-free virtual practice environment. Source: Yan et al., 2026 .
The conceptual progression is a change in evidence conditions:
[ \text{labeled failures} \rightarrow \text{trajectory pools} \rightarrow \text{opaque outcomes} \rightarrow \text{open-world evidence and self-built practice}. ] This is not a simple strongest-to-weakest supervision ranking. An opaque oracle and an open-world verification anchor provide different information. What matters is whether the available evidence can be converted into a stable procedure without smuggling benchmark answers into the update loop.
3.3 The Skill Lifecycle: Optimize, Maintain, and Deploy
Creating a candidate Skill is only the beginning. A deployed capability must be tested, registered, retrieved at the right time, revised when it fails, and retired when it becomes redundant or harmful.
flowchart LR
E["Failures, trajectories, and resources"] --> P["Propose or revise Skill"]
P --> V["Execute tests and evaluation tasks"]
V --> G{"Promotion gate"}
G -->|"reject"| R["Repair, buffer, or archive"]
R --> P
G -->|"accept"| M["Merge, version, and register"]
M --> X["Retrieve and execute"]
X --> D["Collect deployment evidence"]
D --> P
SkillOpt isolates the optimization stage. It treats a Skill document as trainable external state, converts scored rollouts into bounded add, delete, or replace operations, and accepts an edit only when it strictly improves a held-out selection score. A rejected-edit buffer and explicit edit budgets make the process closer to controlled optimization than unconstrained rewriting. The final artifact is loaded normally, so optimization adds no extra model call at deployment.
MUSE-Autoskill broadens the problem from optimizing one artifact to managing a long-lived capability. It creates Skills on demand, tests and registers them, retrieves them for later tasks, attaches experience to individual Skills, and supports refinement, merging, and pruning. A Skill library improves only if useful packages can be found and maintained without unbounded duplication.
SkillClaw extends this lifecycle into multi-user deployment. It aggregates interactions across users and time, identifies recurring successes and failures, proposes refinements or new capabilities, and maintains the results in a shared repository. An improvement discovered by one user can propagate system-wide. The same property raises the risk: a bad local edit becomes a supply-chain change once synchronized to everyone.
The conceptual expansion is
[ \text{single-artifact optimization} \rightarrow \text{lifecycle management} \rightarrow \text{shared deployment evolution}. ] These works address successive lifecycle bottlenecks; the sequence is not a claim of direct citation inheritance.
3.4 Verification: When Does Change Become Improvement?
The governing principle is:
An edit creates a candidate; only evidence-backed acceptance establishes improvement.
Proposal evidence and promotion evidence should be separated whenever possible. The former may be rich and adaptive—reflection, diagnostics, retrieved documentation, or generated tests. The latter should be independent enough to prevent the updater from approving its own mistakes.
System
Evidence used to propose changes
Persistent output
Verification or promotion signal
EvoSkill
Labeled failures
Skill folders and candidate frontier
Held-out frontier performance
Trace2Skill
Successful and failed trajectories
Consolidated Skill directory
Structural guardrails and held-out transfer evaluation; no strict gate for every patch
SkillOpt
Scored rollout batches
Optimized Skill document
Strict improvement on a selection split
MUSE-Autoskill
Tasks, runtime errors, Skill-level experience
Tested Skill packages and library state
Unit tests and runtime feedback
CoEvoSkills
Outputs and surrogate diagnostics
Multi-file Skill bundle
Surrogate tests plus opaque external oracle
SkillClaw
Cross-user deployment trajectories
Shared Skill repository
Delayed real-environment validation before promotion
OpenSkill
Open resources and verification anchors
Task-oriented Skill set
Self-built virtual tests; hidden ground truth only for final evaluation
The table exposes a verification ladder:
[ \text{held-out score} \rightarrow \text{programmatic tests} \rightarrow \text{surrogate diagnostics + opaque oracle} \rightarrow \text{delayed deployment outcomes} \rightarrow \text{self-built virtual verification}. ] These mechanisms should not be collapsed into one “supervision strength” number. They differ in coverage, independence, cost, and error sensitivity.
Two evidence boundaries are especially important. CoEvoSkills is not fully unsupervised: its surrogate provides actionable feedback, but a real opaque oracle still anchors correctness. OpenSkill’s supervision-free claim applies to construction, not to final benchmark evaluation. As verification becomes more autonomous, proxy drift, incomplete coverage, false acceptance, and self-confirmation become the dominant failure modes.
3.5 Synthesis and Bridge to Harnesses
Skill evolution is attractive because it offers a fast, inspectable adaptation layer. A Skill can be diffed, tested, transferred to another model, canary-deployed, and rolled back without changing model weights. But a high-quality artifact can still fail if the surrounding runtime retrieves it at the wrong time, allocates insufficient context, routes tools incorrectly, mishandles dependencies, or violates its execution contract.
The boundary between Skill and Harness is determined by system role rather than file extension. A script packaged inside a task-scoped Skill remains part of that Skill. Global policies for retrieval, context construction, tool routing, scheduling, permission requests, memory, recovery, and sub-agent orchestration belong to the Harness—even if stored as Markdown. Credential enforcement and maximum permission ceilings remain in the governance boundary.
Continual Harness makes this boundary concrete. During one reset-free trajectory, its Refiner can edit the system prompt, sub-agents, Skills, and memory in place. Individual navigation Skills may be repaired and reused, and the paper separately analyzes Skill quality with a direct pathfinding oracle. Yet the overall update object is the composite runtime state (H_t=(p_t,G_t,K_t,M_t)), not the Skill library alone. It is Harness evolution with Skill evolution as one component.
This gives us the transition: Skill evolution changes what reusable capability is available; Harness evolution changes how the Agent operates with that capability.
- Harness Evolution: Improving the Agent Scaffold
A model does not act directly on the world. It receives context through a runtime, calls tools through an interface, stores selected state, delegates work, retries failures, and decides when to stop. Harness evolution asks whether that execution system can improve while the underlying model remains fixed—or, at the boundary, while model and Harness adapt together.
Evidence convention. Paper-linked mechanism descriptions below are paper-reported facts. The coordinate assignments and conceptual arrows are this article’s mechanism-level synthesis; they do not imply a citation, chronological, or implementation lineage.
4.1 A Coordinate System for Harness Improvement
With a fixed model (M), task (x), and Harness (H), execution produces a trajectory
[ \tau \sim p_M(H,x). ] Changing (H) changes what the model observes, which actions it can take, what persists, and how failures are handled. “The Harness changed” is still too coarse to characterize an improvement method. To align with the global notation, we describe a fixed-model Harness update with six coordinates:
Coordinate
Question
Typical choices
(\Omega_H\subseteq H)
Which Harness surface is editable?
Action verifier, prompt, tool, Skill, memory, stateful program, full scaffold
(U)
Who proposes it?
External optimizer, target model, specialized Refiner
(E)
What evidence drives it?
Environment errors, rewards, traces, failure clusters, self-preference
(T)
When does it become active?
Between evaluations, between deployments, or inside one continuous run
(V)
How is it evaluated or accepted?
Interface checks, held-out tests, Pareto selection, preference gate, online outcome
(B)
What remains immutable?
Credential and permission ceilings, sandbox, hidden evaluator, budgets, audit, rollback
This is a Harness-focused projection of the global framework, not the complete taxonomy. Search topology (L)—greedy, population, Pareto, or open archive—and parameter coupling (P_\theta)—frozen, alternating, or joint—remain overlays.
flowchart LR
X["Task or environment"] --> R["Rollout"]
H["Active Harness H"] --> R
R --> E["Evidence E"]
E --> U["Updater U"]
U --> C["Candidate H'"]
C --> A{"Activation mode T"}
A -->|"staged"| VP["Pre-activation validation V"]
VP -->|"promote"| H
VP -->|"safe but exploratory"| L["Archive L"]
VP -->|"reject"| Q["Rejected-candidate buffer"]
A -->|"in place"| H
H --> O["Observed future outcomes"]
O --> VA["Post-activation evaluation V"]
VA -->|"retain"| H
VA -->|"roll back"| K["Checkpointed prior Harness"]
K --> H
B["Immutable boundary B"] -.->|constrains| U
B -.->|protects| VP
B -.->|protects| VA
The last two coordinates are particularly important. An updater can generate candidates without having final deployment authority. Staged methods evaluate before activation; online methods may activate first and evaluate later. A robust design keeps at least one evaluation and governance layer outside editable state; otherwise, the system may “improve” by weakening its test or suppressing failures.
4.2 What Is Updated: From Executable Code to Agent Scaffolding
At the narrowest end, AutoHarness synthesizes executable code that prevents invalid interaction with an environment. The paper studies action filters that generate legal candidates, action verifiers that reject illegal model outputs, and code policies that can choose actions without invoking the model at decision time. A language model mutates candidate programs, while search uses legality and reward feedback to decide what to refine.
The important point is not simply “the artifact is code.” AutoHarness externalizes deterministic rule tracking—a failure mode language models handle unreliably—into an executable mechanism. Its editable subset (\Omega_H) is narrow, its environmental evidence (E) is comparatively hard, and legality can be tested directly. The trade-off is that the learned mechanism is usually environment-specific.
Meta-Harness expands (\Omega_H) from a local action function to a complete stateful program. Its Harness controls what to store, retrieve, and present to the model, allowing coupled changes to context construction, memory updates, tool use, and orchestration.
Its distinctive design is the evidence interface exposed to the proposer. Instead of compressing each experiment into a short summary, Meta-Harness stores every candidate’s source, score, and raw execution traces in a filesystem. An agentic proposer inspects this history with ordinary file tools, creates a new program, and submits it to interface validation and task evaluation. A population supports Pareto selection when objectives such as accuracy and context cost conflict.
Figure 2. Meta-Harness exposes the source, traces, and scores of all prior candidates through a filesystem rather than compressing them into a single prompt. Source: Lee et al., 2026 .
End-to-end search can discover interactions that independent prompt edits would miss, but it is harder to attribute improvement when many behaviors are coupled inside one program.
Agentic Harness Engineering , or AHE, addresses this problem with three forms of observability. Component observability represents prompts, tools, middleware, Skills, sub-agent configuration, and memory as file-level edit units. Experience observability distills very long trajectories into layered evidence. Decision observability requires every edit to predict its intended fixes and possible regressions; the next evaluation compares those predictions with observed task-level changes and can roll back at file granularity.
The resulting conceptual expansion is
[ \text{local executable constraint} \rightarrow \text{full stateful program} \rightarrow \text{componentized and observable scaffold}. ] This is not a direct paper lineage. It highlights a trade-off: end-to-end program search explores coupled strategies, while componentized engineering improves diagnosis, ownership, falsifiability, and rollback.
4.3 Who Performs the Update: Self-Referential Harness Improvement
The next distinction concerns (U), not the representation of (H). Many optimizers rely on a separate, often stronger coding Agent. Self-Harness instead asks whether the target model can identify its own model-specific weaknesses and propose changes to the Harness it will subsequently use.
Its loop has three stages: Weakness Mining collects execution traces and clusters verifier-grounded failure patterns; Harness Proposal invokes the same target model in a proposer role to generate bounded modifications; Proposal Validation tests candidates on both held-in and held-out splits. An edit is promoted only if it does not regress on either split and improves at least one.
Figure 3. Self-Harness changes updater identity: the target model participates in reshaping its own operating Harness. Source: Zhang et al., 2026 .
This is self-referential at the updater coordinate, but it is not unconstrained self-approval. The evaluator, hidden split, acceptance rule, permission ceiling, and deployment authority remain outside editable state. Self-Harness changes (U) while protecting (V) and (B).
The trade-off is structural. The target model is well positioned to reveal model-specific failure modes a generic optimizer may miss. Its diagnoses are also bounded by the same competence and biases that produced those failures. The held-out gate is not incidental machinery; it is what converts self-proposed change into demonstrated improvement.
4.4 What Evidence Drives the Update: Learning from Unlabeled Histories
Real deployments accumulate trajectories, but users rarely provide a ground-truth score for every coding, research, or operational task. Retrospective Harness Optimization , or RHO, asks whether a Harness can improve from historical trajectories without validation labels during optimization.
RHO selects a difficult and diverse coreset of past tasks, re-solves each task several times, and analyzes grouped rollouts. Self-validation identifies questionable tool calls, assumptions, and stopping decisions; self-consistency exposes consequential disagreement. The diagnostics guide several candidate updates, which are compared through pairwise self-preference. If no candidate has positive mean preference over the baseline, the Harness remains unchanged.
Figure 4. RHO converts unlabeled deployment histories into grouped rollouts, diagnoses disagreement and errors, and selects a candidate through self-preference. Source: Pan et al., 2026 .
This changes the supervision regime, not the definition of a Harness. “No ground-truth labels during optimization” also does not remove the need for independent final evaluation. The paper reports its scientific results on held-out benchmark tests with external grading; its label-free claim applies to the optimization loop, not to final evaluation. Self-preference is a proxy for task utility. A model may prefer confidence, verbosity, or internal consistency even when an external objective would rank the behavior lower.
4.5 When Does Improvement Happen? Reset-Free Online Adaptation
The methods above largely separate execution and improvement into rounds. Continual Harness: Online Adaptation for Self-Improving Foundation Agents adds the temporal axis: updates occur inside one ongoing trajectory without resetting the environment.
Its running state is
[ H_t=(p_t,G_t,K_t,M_t), ] where (p_t) is the system prompt, (G_t) the sub-agent collection, (K_t) the Skill library, and (M_t) persistent memory. A fixed meta-tool API bounds CRUD operations over these components. After a warm-up, a Refiner periodically reads a recent trajectory window and diagnoses navigation loops, failed tool calls, stalled objectives, missed exploration, and reusable successful action sequences. The edited Harness becomes active on the next interaction step.
sequenceDiagram
participant E as Persistent environment
participant A as Agent using H_t
participant R as Same-model Refiner
participant S as Component stores
A->>E: Act and observe
E-->>A: New state
A->>S: Append trajectory and memory
loop After warm-up, every F steps
R->>S: Read recent trajectory window
R->>R: Diagnose loops, failures, stalls, and reusable success
R->>S: Edit prompt, sub-agents, Skills, or memory
S-->>A: Activate H_(t+1) immediately
end
A->>E: Continue without reset
The Agent and Refiner roles use the same foundation model, though with different contexts and invocation times. Continual Harness therefore overlaps with full-scaffold evolution in (\Omega_H), same-model self-improvement in (U), and trajectory-driven learning in (E). Its distinctive contribution is (T): failure, repair, and reuse occur in the same persistent run.
That temporal regime complicates validation. The online loop applies edits in place rather than running a held-out acceptance test for every candidate. This statement is an inference from the reported method, not a separate claim by the authors. Future behavior mixes the effects of the changed Harness, advancing environment state, accumulated memory, and changing task difficulty. The paper reports successful in-loop Skill repair, but its results also suggest a method- and domain-dependent capability floor: a weaker model can underperform the minimalist baseline because it cannot reliably exploit or improve the growing scaffold.
The paper includes a second loop at the boundary of pure Harness evolution. Rollout windows from an open-source model are scored by a process reward model, low-reward behavior is relabeled by a frontier teacher, and soft supervised fine-tuning updates the model before the next reset-free training iteration. The environment persists while both (\theta) and (H) change at different timescales.
This is best described as model–Harness co-learning. The primary loop updates (H) through online refinement; the outer loop updates ((\theta,H)) and depends on an external teacher. The official implementation exposes the Refiner and persistent component stores.
4.6 Synthesis: Harness Evolution versus Agent Self-Reference
These papers do not form a single ladder; they change different coordinates.
System
Editable (\Omega_H)
Updater (U)
Proposal evidence (E)
Activation (T)
Evaluation (V)
AutoHarness
Action verifier, filter, or code policy
External synthesis/search loop
Legality and reward feedback
Between search trials
Environment legality and task reward
Meta-Harness
Complete stateful program
Agentic proposer
Prior source, scores, and raw traces
Between evaluations
Interface checks, task score, Pareto selection
AHE
File-level scaffold components
Evolving coding Agent
Layered traces and change manifests
Round-based
Predicted-versus-observed effects and rollback
Self-Harness
Bounded model-specific deltas
Target model in proposer role
Verifier-grounded failure clusters
Staged candidate
Held-in and held-out regression gate
RHO
Persistent tools, prompts, and Skills
Same backbone in several roles
Unlabeled history and grouped rollouts
Retrospective, between deployments
Self-preference internally; held-out external benchmark for the scientific claim
Continual Harness
Prompt, sub-agents, Skills, and memory
Same-model Refiner
Recent trajectory window
Mid-trajectory, in place
No per-edit held-out gate is reported; later trajectory outcomes
Across all six systems, (B) should protect permission ceilings, credentials, evaluator integrity, resource limits, audit, and rollback authority. Their search topology (L) and parameter-coupling mode (P_\theta) remain additional overlays.
Two conclusions follow.
First, Harness evolution and self-referential Agent evolution answer different questions. Harness evolution asks which model-external execution surface changes. Self-reference asks whether the target system participates in constructing its successor. A method can belong to both.
Second, broadening the editable surface is only one direction of progress:
[ \text{narrow code} \rightarrow \text{full scaffold} \rightarrow \text{observable components}. ] Updater identity, evidence, timing, validation, and governance remain orthogonal. Weakening external supervision or activating edits faster may increase autonomy, but it also increases the burden on evaluation, attribution, checkpointing, rollback, and immutable trust boundaries.
Joint model–Harness learning pushes this burden further. It must distinguish gains caused by parameter updates, Harness updates, teacher relabeling, and a changing trajectory distribution. The central question is not whether an Agent can rewrite its scaffold. It is whether useful changes can accumulate while (V) remains informative and (B) remains outside the system’s ability to redefine success.
- Future Challenges
The reported studies demonstrate, within their benchmark and deployment settings, that an Agent can turn experience into persistent changes to prompts, workflows, Skills, code, and Harness state. Three bottlenecks determine whether those loops become reliable learning systems rather than increasingly elaborate self-editors:
Is the system optimizing the right objective when ground truth is absent?
Does an update that passes a local test remain useful over time?
How should fast external adaptation interact with slower parameter learning?
These questions are sequential. A durable update is not valuable if the objective is wrong, and a good objective is not enough if improvements decay or the learning state becomes impossible to maintain.
5.1 Improving Toward What? Evaluation Without Ground Truth
When ground truth exists, the basic protocol is familiar: use one split to generate or select updates, keep another split hidden for evaluation, and prevent the updater from editing the scoring path. The difficult case is open-world deployment, where outcomes may be delayed, partially observed, multi-objective, or never reduced to a clean reward.
“No reward” does not necessarily mean “no evidence.” A real system may still observe hard constraints, task completion artifacts, user corrections, downstream consequences, disagreement among independent evaluators, or whether a decision survives later review. The problem is that these signals answer different questions and may all be biased.
Evidence source
What it can establish
Main failure mode
Hard constraints and executable checks
Validity, safety invariants, interface compliance
Necessary behavior may still be low quality
Delayed environment or user outcomes
Whether behavior eventually produced value
Sparse feedback, confounding, and long delay
Human or user preference
Quality dimensions that are hard to formalize
Inconsistency, cost, and preference manipulation
Multiple independent critics or models
Disagreement and uncertainty
Correlated model bias and shared blind spots
Self-generated tests and self-preference
Dense diagnostic signal for search
Circular self-approval and Goodharting
Counterfactual or causal comparison
Whether the update, rather than context, caused the gain
Expensive or impossible in a changing environment
The key design rule is to separate proposal evidence from promotion authority . Self-reflection, self-preference, generated tests, and surrogate verifiers can guide candidate search. They should not become the sole reason a high-impact update is deployed. Promotion should combine the strongest available external anchors and retain an uncertainty-aware option to abstain, defer, or request human review.
In many environments there is also no single “correct direction.” Task utility, cost, latency, safety, user preference, maintainability, and exploration may conflict. Collapsing them into one reward hides value judgments and invites reward hacking. A better formulation maintains a vector objective or Pareto frontier and makes the protected constraints explicit:
[ J(H)=\bigl( \text{task utility}, -\text{cost}, -\text{risk}, \text{maintainability} \bigr). ] The central research question is therefore not merely how to synthesize a reward. It is how to construct a falsifiable evaluation contract : what evidence would show that the proposed direction is wrong, which signals are allowed to adapt, and which authority remains outside the self-improvement loop?
5.2 From Local Validation to Durable Improvement
Suppose the first problem is solved and a candidate passes its validation gate. That still proves only a local statement: the update helped on the sampled tasks, under the current model, Harness, environment, and budget. It does not show that the change will remain useful after the task distribution shifts, other components evolve, users behave differently, or the artifact library grows.
Local success can fail to become durable improvement in at least four ways:
evaluation overfitting: the edit memorizes recurring benchmark structure or exploits a verifier blind spot;
distribution shift: future tasks exercise capabilities absent from the validation set;
component interference: a later Skill, model, or routing change invalidates the original gain;
maintenance debt: local patches accumulate until context cost, coupling, or regression risk outweighs their benefit.
Durability must therefore be evaluated at multiple horizons.
Horizon
Question
Appropriate evidence
Candidate-level
Does the delta fix its target failure without an immediate regression?
Held-in tests, held-out tests, interface and safety checks
Distribution-level
Does the gain transfer beyond the update examples?
Hidden task families, cross-model or cross-environment tests
Temporal deployment
Does it remain useful on later data and users?
Temporal holdouts, shadow runs, canaries, longitudinal A/B tests
System lifecycle
Does it compose with later updates and remain maintainable?
Dependency-aware regression, cost and artifact-growth tracking, rollback history
This implies a staged state machine rather than one binary gate:
[ \text{proposed} \rightarrow \text{safe archive candidate} \rightarrow \text{evaluated} \rightarrow \text{canary} \rightarrow \text{promoted} \rightarrow \text{monitored or rolled back}. ] Archive admission and deployment promotion should remain distinct. An unusual or temporarily low-scoring candidate may be retained as a stepping stone if it is runnable, provenance-complete, and safe to store. Deployment requires stronger utility and regression evidence.
Reset-free online adaptation makes this problem sharper. After an in-place Harness edit, the environment has advanced, memory has grown, and the next task state may be harder or easier. A later success cannot automatically be attributed to the edit. Where replay is possible, synchronized checkpoints and shadow Harnesses can support counterfactual comparison. Where it is not, long-horizon monitoring, uncertainty estimates, and fast rollback become part of the algorithm rather than optional deployment infrastructure.
The unit of evaluation should ultimately be a performance trajectory, not a one-time score. A durable improvement raises future utility after accounting for cost, regressions, and maintenance burden across a meaningful time window.
5.3 Coordinating Parametric Learning and External Editable State
External state is attractive because it is fast to edit, inspect, transfer, and roll back. Model parameters are compact at inference time and can internalize patterns that should generalize broadly. Treating either as the only learning substrate creates a failure mode.
The concern that external state will inevitably grow is slightly too strong: a disciplined lifecycle can merge, compress, prune, garbage-collect, and retire Skills or Harness components. But without those operations, append-only external learning does tend toward duplicated procedures, longer context, conflicting rules, rising dependency cost, and a system that neither humans nor Agents can safely maintain.
Parameter learning is not an automatic cure. Consolidating weak or biased external experience into weights can make errors harder to inspect and roll back, introduce catastrophic forgetting, and change the trajectory distribution on which the current Harness was optimized. The problem is therefore one of allocation and timescale , not a choice between weights and files.
A promising architecture treats external state as fast, reversible memory and model parameters as slow, carefully consolidated memory:
flowchart LR
E["New experience"] --> F["Fast external update<br/>prompt, Skill, memory, Harness"]
F --> V["Cross-task and temporal validation"]
V -->|"unstable or narrow"| P["Keep external, revise, merge, or prune"]
P --> F
V -->|"stable and broadly reusable"| D["Distill verified trajectories and contracts"]
D --> T["Slow parameter update"]
T --> R["Regression, safety, and capability evaluation"]
R -->|"reject"| F
R -->|"accept"| M["Deploy new model version"]
M --> C["Re-evaluate and simplify external state"]
C --> F
This loop raises four concrete research questions:
What should be internalized? Repeated, cross-task, stable procedures are stronger candidates than environment-specific rules or rapidly changing facts.
When is evidence mature enough? Consolidation should require broader and longer-lived validation than an ordinary external edit.
How is attribution preserved? Training examples should retain links to the external artifacts, trajectories, tests, and decisions from which they were distilled.
How is the Harness re-based after a model update? A stronger or behaviorally different model may make old prompts redundant, invalidate routing assumptions, or interact badly with accumulated Skills.
A clean co-learning experiment should compare (\theta)-only, (H)-only, and joint ((\theta,H)) updates from the same environment checkpoints under matched interaction and compute budgets. Teacher relabeling should be ablated on and off. Evaluation should report not only task utility but also external-state size, regression rate, adaptation cost, and how much of the joint gain remains after either the model or Harness is reset to its previous version.
Continual Harness offers an early boundary example because it combines online Harness refinement with a separate model-update loop. It does not settle the allocation problem: its co-learning path depends on a process reward model and an external frontier teacher, while attribution between model and Harness gains remains difficult.
The desired endpoint is not an ever-growing external repository or a model that periodically absorbs everything. It is a memory hierarchy with consolidation : external state absorbs experience quickly and transparently; repeated validated structure is compressed into more stable representations, including parameters when appropriate; redundant external state is simplified; and every consolidation step is independently regression-tested.
Together, the three challenges define a stricter standard for self-improvement: the system must know what evidence points in the right direction, demonstrate that a change survives beyond its local gate, and maintain a healthy division of labor between fast editable state and slow parametric learning.
- References
[1] Anthropic. 2025. Equipping Agents for the Real World with Agent Skills .
[2] Genewein, T., et al. 2026. From AGI to ASI .
[3] Agrawal, L., et al. 2025. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning .
[4] Zhang, J., et al. 2024. AFlow: Automating Agentic Workflow Generation .
[5] Hu, S., et al. 2024. Automated Design of Agentic Systems .
[6] Robeyns, M., et al. 2025. A Self-Improving Coding Agent .
[7] Zhang, J., et al. 2025. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents . Code .
[8] Zhang, J., et al. 2026. Hyperagents . Code .
[9] Weng, J. 2026. Learning Beyond Gradients . Code .
[10] Novikov, A., et al. 2025. AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery .
[11] Alzubi, S., et al. 2026. EvoSkill: Automated Skill Discovery for Multi-Agent Systems .
[12] Ni, J., et al. 2026. Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills .
[13] Yang, Y., et al. 2026. SkillOpt: Executive Strategy for Self-Evolving Agent Skills .
[14] Lin, H., et al. 2026. MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation .
[15] Zhang, H., et al. 2026. CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification .
[16] Ma, Z., et al. 2026. SkillClaw: Let Skills Evolve Collectively with Agentic Evolver . Code .
[17] Yan, Z., et al. 2026. OpenSkill: Open-World Self-Evolution for LLM Agents . Code .
[18] Lou, X., et al. 2026. AutoHarness: Improving LLM Agents by Automatically Synthesizing a Code Harness .
[19] Lee, Y., et al. 2026. Meta-Harness: End-to-End Optimization of Model Harnesses .
[20] Lin, J., et al. 2026. Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses .
[21] Zhang, H., et al. 2026. Self-Harness: Harnesses That Improve Themselves .
[22] Pan, W., et al. 2026. Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference . Code .
[23] Karten, S., et al. 2026. Continual Harness: Online Adaptation for Self-Improving Foundation Agents . Code .
comments powered by Disqus