Shuaiqi Liu*, Zhengkai Lin*, Yuxiang Zhang*, Yuanyi Ren*, Yue Wu, Yongbin Li, Zheng Wang, Zhihang Fu*, Jieping Ye
Alibaba Group
English | 中文版
A living survey and structured literature map for self-improving agent systems: systems that transform experience and evaluation feedback into persistent updates to their own components, moving toward recursive self-improving agents.
This repository accompanies the survey “The Path to Recursive Self-Improving Agents: Foundation, Framework, and Future Directions.”
The survey studies agent systems that can autonomously convert execution traces, training signals, validation results, and other feedback into durable changes to their own components. It formalizes the agent system as:
where (\mathcal{M}) is the foundation model, (\mathcal{H}) is the agent harness, (\mathcal{D}) is the agent data system, (\mathcal{T}) is the agent trainer, and (\mathit{Imp}) is the improvement mechanism. A self-improving system follows the update rule:
Recursive self-improvement (RSI) appears when (\mathit{Imp}t) is itself part of the modifiable system state and can become (\mathit{Imp}{t+1}), allowing the system to improve its own future improvement process.
The paper makes four main contributions:
Formal foundation: It defines agent-system self-improvement and recursive self-improvement, then introduces a five-level capability grading standard from manual improvement to general recursive self-improvement.Unified framework: It models the foundation model, agent harness, agent data system, agent trainer, and improvement mechanism as a coupled evolving system.Taxonomy of existing work: It organizes research into agent harness self-improvement, agent data system self-improvement, agent trainer self-improvement, and cross-component co-improvement.Roadmap for future research: It identifies open problems around long-horizon evaluation, modifiable infrastructure, bounded-to-general RSI, safety, and human-agent co-improvement.
The unified framework emphasizes two questions:
What can be improved? The autonomous improvement scope may include themodel,** harness**,** data system**,** trainer**, and eventually the** improvement mechanismitself. How does improvement proceed?The improvement mechanism**process includes diagnosing bottlenecks, proposing candidate modifications, evaluating and selecting candidates, and integrating accepted changes into persistent system state.
The survey distinguishes self-improving systems by autonomy in the improvement loop, whether the improvement mechanism is mutable, and whether the capability generalizes across domains.
| Level | Category Name | Autonomous Proposal | Autonomous Implementation | Autonomous Validation | Improve the Improvement Mechanism | Domain Generality |
|---|---|---|---|---|---|---|
| L1 | ||||||
| Manual Improvement | ||||||
| L2 | ||||||
| Assisted Improvement | ✓ | |||||
| L3 | ||||||
| Programmatic Self-Improvement | ✓ | ✓ | ✓ | |||
| L4 | ||||||
| Bounded Recursive Self-Improvement | ✓ | ✓ | ✓ | ✓ | ||
| L5 | ||||||
| General Recursive Self-Improvement | ✓ | ✓ | ✓ | ✓ | ✓ |
L1 — Manual Improvement: This type of agent system has no autonomous improvement capability. It is deployed and executed in a fixed form, and every change requires a manual development and deployment process.L2 — Assisted Improvement: This type of system can propose candidate modifications or provide diagnostic evidence, but humans remain responsible for validating and applying substantive changes. The improvement mechanismImp
is human-maintained, so L2 is a precursor to self-improvement rather than a full instance of it.L3 — Programmatic Self-Improvement: This type of system can autonomously propose, apply, and validate modifications to operational components such as the foundation model, agent harness, data system, or trainer. However,Imp
remains fixed or externally maintained.L4 — Bounded Recursive Self-Improvement: This type of system can not only propose, validate, and apply candidate modifications, but also rewrite its own improvement mechanism, making the improvement process self-referential. The process supports sustained long-term progress within a bounded domain.L5 — General Recursive Self-Improvement: L5 retains the autonomy, self-reference, and long-term progress of L4, while transferring improvement capability across broad and evolving task domains rather than remaining limited to a fixed benchmark or narrow operational setting.
The harness is the execution layer that determines what the model observes, what actions it can invoke, and how observations and actions are organized into task workflows. Harness self-improvement means converting traces, evaluation feedback, and failures into persistent changes to memory, skills, prompts, workflows, tools, or executable scaffolds.
This category of work primarily modifies experience memory, retrieval rules, and memory architecture, turning execution traces into reusable, persistent state that conditions later behavior.
| Paper | Link | Year | Level |
|---|---|---|---|
| Reflexion | |||
linklinklinklinklinklinkThis category of work primarily modifies skill libraries, tool libraries, or the shared skill-tool ecosystem that agents retrieve, compose, and reuse.
| Paper | Link | Year | Level |
|---|---|---|---|
| Tool Makers | |||
linklinklinklinklinklinklinklinklinklinklinklinklinklinklinkThis category of work primarily modifies prompt programs, context playbooks, guideline documents, or decision-rule libraries; a small subset also makes the prompt optimizer itself mutable.
| Paper | Link | Year | Level |
|---|---|---|---|
| Dynamic Cheatsheet | |||
linklinklinklinklinklinklinklinklinklinklinkThis category of work primarily modifies trajectory orchestration, agent composition, or workflow/communication topology, so the harness spends compute where uncertainty is high and reuses useful intermediate results.
| Paper | Link | Year | Level |
|---|---|---|---|
| AgentGA | |||
linklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinkThis category of work primarily modifies harness policy, runtime scaffold, agent source code, or the improvement mechanism itself, making the modified agent part of the mechanism that produces further changes.
| Paper | Link | Year | Level |
|---|---|---|---|
| Life-Harness | |||
linklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinkThe agent data system manages the lifecycle of training and evaluation data. It includes data production and data utilization, connecting inference-time interaction records with persistent learning in the trainer.
This category of work primarily modifies environment configurations, simulators, or the tool/skill libraries used to construct environments, keeping training environments aligned with the agent's evolving capability boundary.
| Paper | Link | Year | Level |
|---|---|---|---|
| EnvGen | |||
linklinklinklinklinkThis category of work primarily modifies task instructions, constraints, or evaluation criteria, using diagnostic feedback, fixed quality-control signals, or competitive feedback to keep tasks relevant and appropriately difficult.
| Paper | Link | Year | Level |
|---|---|---|---|
| WebRL | |||
linklinklinklinkThis category of work primarily modifies solution trajectories, self-play data, or preference data that record agent experience for a given task.
| Paper | Link | Year | Level |
|---|---|---|---|
| SPIN | |||
linklinklinklinkThis category of work primarily modifies judge models, validators, or quality filters that determine which synthesized data is retained.
| Paper | Link | Year | Level |
|---|---|---|---|
| Self-Improving VLM Judges | |||
This category of work primarily modifies the curriculum scheduler, task allocation, or difficulty adaptation that determines how produced data are selected and ordered for training.
| Paper | Link | Year | Level |
|---|---|---|---|
| AMC-TSI | |||
linklinklinkThis category of work primarily couples two or more data-production or data-utilization modules through feedback loops, so multiple modules improve collaboratively rather than in isolation.
| Paper | Link | Year | Level |
|---|---|---|---|
| Agent0-VL | |||
linklinklinklinkThe trainer converts agent experience into persistent model updates. Trainer self-improvement revises the persistent state that determines supervision, optimization, infrastructure, or the mechanism that improves the trainer itself.
This loop modifies persistent trainer state, such as a reward model, verifier, rubric memory, process reward model, batch-size controller, or infrastructure configuration, within an active training lineage while the improvement mechanism remains fixed.
| Paper | Link | Year | Level |
|---|---|---|---|
| ACE (Adversarial Code Evolution) | |||
linklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinkThis loop modifies a training script, reward code, fine-tuning recipe, optimizer code, or training pipeline across bounded experiments, under a fixed improvement mechanism that proposes and evaluates trainer candidates.
| Paper | Link | Year | Level |
|---|---|---|---|
| AIDE | |||
linklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinklinkThis loop modifies the persistent improvement mechanism itself, e.g., a search runner, diagnostic harness, improvement model, or improvement harness, across successive trainer-improvement cycles.
| Paper | Link | Year | Level |
|---|---|---|---|
| AutoScientists | |||
linklinklinklinklinkSingle-component self-improvement is often insufficient because system-level bottlenecks shift as components evolve. Co-improvement studies feedback loops across multiple components.
Improved skills, tools, workflows, or scaffolds make training signals more informative, while training feedback reveals which harness modules should be revised next.
| Paper | Link | Year | Level |
|---|---|---|---|
| SkillRL | |||
linklinklinkHarness modules change the data the agent system can produce or access, while data-side signals drive the creation and refinement of those harness modules.
| Paper | Link | Year | Level |
|---|---|---|---|
| CODESKILL | |||
linklinklinkBetter training produces better data, which in turn improves training, forming a feedback loop between an environment/world model and the policy.
| Paper | Link | Year | Level |
|---|---|---|---|
| WebEvolver | |||
linkThe survey highlights five open problems for moving from current self-improving agent systems toward reliable and general RSI Agents:
Long-horizon real-world evaluation: Benchmarks should test whether repeated self-modification improves reliability, robustness, usefulness, capability retention, and the improvement process itself under evolving workloads, goals, budgets, tools, and environments.Observable, scalable, and modifiable training and inference infrastructure: RSI needs unified, agent-friendly representations of execution traces, training records, validation results, version histories, and resource usage so agents can observe, reproduce, compare, and safely revise the improvement process.From bounded recursive self-improvement to general recursive self-improvement: The key challenge is to make improvement mechanisms transferable across domains while avoiding the inappropriate transfer of task-specific strategies that fail in new settings.Safety and controllability under recursive self-modification: Systems must keep capability-oriented components modifiable while protecting governance components through stricter verification, continuous monitoring, rollback versions, safety alerts, incident logs, and human review.Human-expert and agent co-improvement: High-impact applications need agents that know when to involve experts, choose the right interaction form, provide interpretable evidence, and convert expert feedback into reusable improvement resources.
This repository is intended to become a community-maintained living survey.
When adding or revising a paper entry, please include:
Title and linkVenue and yearComponent: Harness Self-Improvement, Data System Self-Improvement, Trainer Self-Improvement, or Multi-Component Co-Improvement** Subcategory**: e.g., Memory, Task Synthesis, Inner-Loop Trainer Adaptation** L-level**: L1–L5** Domain****Rationale**: cite concrete evidence for the component and L-level judgment
If there is a dispute over the grading result of an existing paper, please submit a commit and provide the above information, the proposed level, and relevant evidence.
If you find this survey or repository useful, please cite this paper.
@article{202608.0051,
doi = {10.20944/preprints202608.0051.v1},
url = {https://www.preprints.org/manuscript/202608.0051},
year = 2026,
month = {August},
publisher = {Preprints},
author = {Shuaiqi Liu and Zhengkai Lin and Yuxiang Zhang and Yuanyi Ren and Yue Wu and Yongbin Li and Zheng Wang and Zhihang Fu and Jieping Ye},
title = {The Path to Recursive Self-Improving Agents: Foundation, Framework, and Future Directions},
journal = {Preprints}
}