ALTK-Evolve: On-the-Job Learning for AI Agents

A developer released ALTK-Evolve, a long-term episodic memory subsystem that lets AI agents distill reusable guidelines from past interaction traces and inject only relevant ones at action time. Evaluated on AppWorld with a ReAct agent, the memory layer raised aggregate Scenario Goal Completion from 50.0% to 58.9%, with the largest gain on hard tasks (+14.2 points, 19.1% to 33.3%). The system is available as a Claude Code plugin that stores extracted entities on the filesystem and uses hooks for automatic retrieval.

Imagine a brilliant line cook who has memorized every cookbook but forgets your kitchen every morning. They don’t remember your oven runs hot, or that regulars like extra salt; they’ll follow a recipe card yet freeze when you’re out of lemons. That’s most AI agents: excellent at following prompts, poor at accumulating wisdom about your environment. Feeding yesterday’s logs back into the prompt just makes them re‑read history; it doesn’t help them generalize from it. A junior needs different recipes for “vinaigrette” and “duck à l’orange.” A chef learns “acid balances fat” and applies it everywhere. Likewise, reliable agents should distill principles from experience and apply them to new tasks, not just near duplicates of old ones. This long‑term memory subsystem does exactly that: it converts interaction traces into candidate guidelines, filters for quality, and injects only relevant guidance at the moment of action. Agents need principles, not transcripts. A recent MIT study https://www.forbes.com/sites/jasonsnyder/2025/08/26/mit-finds-95-of-genai-pilots-fail-because-companies-avoid-friction/ found that 95% of pilots fail because agents don't adapt and learn on the job. ALTK-Evolve addresses this learning gap using long term episodic memory to help agents reason better. Evolve is a memory system for AI agents, that can help agents improve over time, learning from and using guidelines generated from previous executions. Operationally, the system runs as a continuous loop: 1 Downward flow observation & extraction : Capture full agent trajectories user utterances, thoughts, tool calls, results in an Interaction Layer e.g., Langfuse or another OpenTelemetry‑based observability tool . Pluggable extractors mine traces for structural patterns and persist them as candidate entities. 2 Upward flow refinement & retrieval : A background consolidate‑and‑score job merges duplicates, prunes weak rules, and boosts proven strategies, evolving a high‑quality library of entities such as guidelines, policies, and SOPs. Retrieval pulls only the relevant items via the Interaction Layer and injects them back into context at the Application Layer. This approach works for a few key reasons: We evaluated the framework on AppWorld https://appworld.dev , where agents complete realistic multi‑step tasks via APIs, averaging 9.5 APIs across 1.8 apps, with hard cases requiring more complex control flow. A ReAct agent received the task instruction plus the top 5 retrieved guidelines generated on a prior run train/dev and tested on an unseen partition test-normal . We report Scenario Goal Completion SGC , a strict consistency metric requiring success across variants. | Difficulty | Baseline SGC | + Memory | Δ | |---|---|---|---| | Easy | 79.0% | 84.2% | +5.2 | | Medium | 56.2% | 62.5% | +6.3 | | Hard | 19.1% | 33.3% | +14.2 | | Aggregate | 50.0% | 58.9% | +8.9 | Here are some key conclusions from the evaluations: Find more details about the experiments in the paper at https://arxiv.org/abs/2603.10600 https://arxiv.org/abs/2603.10600 . You have a choice in how to integrate ALTK‑Evolve into your agent. Install the plugin into Claude Code: claude plugin marketplace add AgentToolkit/altk-evolve claude plugin install evolve-lite@evolve-marketplace That’s it The plugin extracts entities from trajectories and stores them as files on your filesystem. It uses Claude Code’s hooks for automatic retrieval. Prefer to watch instead of read? See the short Evolve-Lite Claude Code walkthrough video : Demo https://youtu.be/XIlYA79pYp4 Check out the walkthroughs here https://github.com/AgentToolkit/altk-evolve/blob/main/docs/integrations/claude/evolve-lite.md for examples of how to learn with Claude Code in Lite mode. Lite mode is easy to test‑drive but has limitations. For example, it doesn’t glean insights from across agent sessions or perform consolidation and garbage collection of entities. The low‑code and pro‑code versions below address these limitations. There are also one-step integrations with Codex https://agenttoolkit.github.io/altk-evolve/examples/hello world/codex/ and IBM Bob https://agenttoolkit.github.io/altk-evolve/examples/hello world/bob/ . Try them out Add a single altk evolve.auto import and flip a flag to emit traces to an Arize Phoenix UI. Then sync traces to generate improvement guidelines without changing your current stack. It works with popular LLM clients and agent frameworks e.g., OpenAI, LiteLLM, and Hugging Face agents , so you keep your current stack and simply gain visibility. To see just how easily this fits into existing projects, explore our hands‑on examples https://github.com/AgentToolkit/altk-evolve/tree/main/examples/low code showcasing different framework integrations. For full details on configuration and capabilities, read our low‑code tracing documentation https://github.com/AgentToolkit/altk-evolve/blob/main/docs/guides/low-code-tracing.md . We integrated ALTK‑Evolve directly into CUGA https://github.com/cuga-project/cuga-agent via MCP to create a tight, low‑overhead learning loop. Before each run, the get guidelines https://github.com/cuga-project/cuga-agent/blob/kaizen-lite-mode/src/cuga/backend/cuga graph/nodes/cuga lite/cuga lite graph.py L738-L755 MCP tool is called to surface task‑specific steering and reduce trial‑and‑error. After the run, CUGA sends back structured execution traces via save trajectory https://github.com/cuga-project/cuga-agent/blob/9773d66aba33895d42e335af780dbd3ff202fec9/src/cuga/backend/cuga graph/nodes/cuga lite/cuga lite node.py L393-L408 , so Evolve can learn from what actually happened and improve future guidance. The result is an integration that gets better over time while staying transparent, composable, and easy to adopt. Prefer a visual tour? Watch the CUGA integration walkthrough : video https://youtu.be/CMG7ysVt2 8 Your agent shouldn’t wake up as an intern every morning. This approach helps it learn on the job. If you're using Claude Code, Codex, and IBM Bob, try it out in minutes and see how it improves your agent. Tip: Star the repo, it helps others discover the project and directly guides what we build next.