The Graph That Learns: Building Self-Improving Agent Loops

A developer has built a prototype of a self-improving AI agent that combines agent loops with graph-based memory to accumulate knowledge from completed tasks. The system records task outcomes, file relationships, and reviewer information in a graph, enabling the agent to start future similar tasks with prior context instead of from scratch. This approach aims to give agents institutional memory, making them more efficient at solving recurring problems.

AI agents are getting very good at doing things. They can read a ticket, modify code, open a pull request, query an API, send an email, and keep going until a goal is reached. But there is a problem hiding underneath all of this: Most agents don't actually get smarter from doing the work. They execute a loop, finish the task, and forget what happened. The next time the same problem appears, they start over. That feels wrong. A useful agent shouldn't just complete tasks. It should accumulate knowledge about how to complete those tasks better. This is where two ideas become extremely powerful when combined: Agent loops provide action. Graphs provide memory. And when the graph is updated by the agent's own experience, you get something much more interesting: A self-improving system. In this post, we're going to build a small version of that idea. Consider an engineering agent with this goal: Fix the failing checkout test. The agent might: Read issue ↓ Inspect repository ↓ Find failing test ↓ Inspect implementation ↓ Modify code ↓ Run tests ↓ Fix failure ↓ Open pull request Great. But tomorrow another checkout test fails. The agent starts from scratch. It doesn't remember: The agent has intelligence. But it has no institutional memory . That's a huge difference. At its simplest, an agent is not magic. It's a loop: Goal ↓ Observe ↓ Choose action ↓ Execute ↓ Check result ↓ Repeat We can write that conceptually as: js while goalComplete { const observation = observe ; const action = decide observation, ; const result = execute action, ; check result ; } This is the core of an agent. The model provides reasoning. Tools provide capabilities. The loop provides persistence toward a goal. But there's another component we need. Learning. Imagine the agent completes a task. Instead of simply returning: Task complete. it records what happened: Task: Fix checkout timeout. Observation: Checkout requests were being retried twice. Action: Changed retry behavior in checkout.ts. Result: Tests passed. Related files: payments.ts retry.ts Reviewer: maya Outcome: Merged. Now imagine that the next checkout problem occurs. The agent can see this previous experience. Instead of starting from zero: What is checkout.ts? it can start with: Previous changes to checkout.ts indicate that retry.ts and payments.ts are frequently involved. A previous fix modified retry behavior. Let's inspect those relationships first. That's a dramatically better agent. But there is an important question: How should we store the memory? You could dump everything into a document. Something like: Previous task: Checkout timeout. Files: checkout.ts payments.ts retry.ts People: Maya Bobby Solution: Changed retry behavior. This works. Until you have 10,000 tasks. Then you have a giant pile of text. The interesting information isn't just the facts. It's the relationships . For example: Task │ ├── affected → checkout.ts │ │ │ └── related → retry.ts │ ├── fixed-by → commit 823 │ ├── reviewed-by → Maya │ └── resulted-in → merged PR Now we have a graph. And graphs give us something extremely useful: Traversal. We can ask: What happened to this file? Who worked on it? Who reviewed those changes? What other files changed with it? Which previous tasks touched this area? Which approaches worked? Which approaches failed? The agent doesn't need to remember everything. It needs to know where to look . Let's build a small prototype. Our agent will have one goal: Investigate a failing test. It will have four tools: type Tool = | "search code" | "read file" | "run tests" | "inspect history"; And it will maintain a graph containing: Task File Test Change Person Outcome Relationships will include: AFFECTS READ MODIFIED FIXED FAILED REVIEWED RELATED TO The important part is that agent activity becomes graph data . Create a simple graph implementation: type NodeType = | "task" | "file" | "test" | "change" | "person" | "outcome"; type Relationship = | "AFFECTS" | "READ" | "MODIFIED" | "FIXED" | "FAILED" | "REVIEWED" | "RELATED TO"; type Node = { id: string; type: NodeType; label: string; metadata?: Record<string, unknown ; }; type Edge = { from: string; to: string; type: Relationship; metadata?: Record<string, unknown ; }; Then: class Graph { private nodes = new Map<string, Node ; private edges: Edge = ; addNode node: Node { this.nodes.set node.id, node ; } addEdge edge: Edge { this.edges.push edge ; } getNode id: string { return this.nodes.get id ; } neighbors id: string, relationship?: Relationship, { return this.edges .filter edge = { if edge.from == id { return false; } if relationship && edge.type == relationship { return false; } return true; } .map edge = { edge, node: this.nodes.get edge.to , } .filter result = result.node == undefined, ; } } That's enough for our prototype. We don't need Neo4j. We don't need a distributed graph database. We don't even need persistence yet. We're trying to understand the architecture. Now let's represent an experience. type AgentExperience = { task: string; observations: string ; actions: string ; result: "success" | "failure"; files: string ; }; Suppose our agent successfully fixes a checkout problem. We can record: js const experience: AgentExperience = { task: "Fix checkout timeout", observations: "Requests were retried twice", "payments.ts was involved", , actions: "Inspected checkout.ts", "Inspected retry.ts", "Changed retry behavior", "Ran checkout tests", , result: "success", files: "checkout.ts", "retry.ts", "payments.ts", , }; Now turn that experience into graph nodes. js const taskId = "task:checkout-timeout"; graph.addNode { id: taskId, type: "task", label: experience.task, } ; for const file of experience.files { const fileId = file:${file} ; graph.addNode { id: fileId, type: "file", label: file, } ; graph.addEdge { from: taskId, to: fileId, type: "AFFECTS", } ; } We have transformed an experience into structured memory. Now let's create the actual loop. type AgentState = { goal: string; observations: string ; actions: string ; complete: boolean; }; async function runAgent goal: string, graph: Graph, { const state: AgentState = { goal, observations: , actions: , complete: false, }; while state.complete { const context = buildContext state, graph ; const decision = await decideNextAction state, context, ; state.actions.push decision.action, ; const result = await executeTool decision.action, decision.input, ; state.observations.push result, ; state.complete = await isComplete state, ; } return state; } This is a normal agent loop. But notice this: js const context = buildContext state, graph ; The graph is now part of the agent's observation system. The agent doesn't just observe the repository. It observes its accumulated experience . That's the beginning of a self-improving agent. Now comes the interesting part. When the task finishes, we update the graph. js function recordExperience graph: Graph, state: AgentState, { const taskId = task:${crypto.randomUUID } ; graph.addNode { id: taskId, type: "task", label: state.goal, } ; for const action of state.actions { const actionId = action:${crypto.randomUUID } ; graph.addNode { id: actionId, type: "change", label: action, } ; graph.addEdge { from: taskId, to: actionId, type: "MODIFIED", } ; } const outcomeId = outcome:${crypto.randomUUID } ; graph.addNode { id: outcomeId, type: "outcome", label: state.complete ? "Success" : "Failure", } ; graph.addEdge { from: taskId, to: outcomeId, type: state.complete ? "FIXED" : "FAILED", } ; } Now the graph has changed because of the agent's experience. That's the key. The system isn't just reading a knowledge base. The agent is writing back into it. Now let's say another task appears: Fix checkout requests timing out. Before deciding what to do, the agent queries the graph. js function findRelevantExperience graph: Graph, query: string, { const tasks = graph.neighbors "task:checkout-timeout", ; return tasks; } In a real system, we'd use semantic search, graph traversal, or both. The important concept is: New task ↓ Find similar experiences ↓ Traverse relationships ↓ Build context ↓ Choose action Now the agent can start with: Previous experience: checkout.ts ↓ retry.ts ↓ successful fix Previous action: Changed retry behavior. Previous outcome: Tests passed. Potential next action: Inspect retry.ts first. The second agent doesn't need to rediscover everything. It inherits the first agent's experience. We can now expand our original agent loop. Instead of: Goal ↓ Observe ↓ Act ↓ Check ↓ Repeat we get: ┌───────────────────────┐ │ │ ▼ │ Goal │ │ │ ▼ │ Query the Graph │ │ │ ▼ │ Observe │ │ │ ▼ │ Decide │ │ │ ▼ │ Execute │ │ │ ▼ │ Check │ │ │ ┌──────┴──────┐ │ │ │ │ Failure Success │ │ │ │ └──────┬──────┘ │ │ │ ▼ │ Update Graph ─────────────────┘ This creates an important feedback cycle: Experience ↓ Graph ↓ Context ↓ Better decision ↓ New experience ↓ Graph That is what I mean by a self-improving graph . The graph becomes a record of what the system has learned by doing. There is one thing we absolutely cannot do. We cannot assume every experience is correct. Imagine an agent tries this: Action: Increase retry count from 2 → 10 Result: Test passed. The agent records: Increasing retries fixes checkout problems. But maybe the test passed because the flaky test happened to pass. Now we've poisoned the graph. The next agent sees: Historical knowledge: Increasing retries works. And repeats the mistake. This is where verification becomes critical. Instead of recording: Agent says it worked. we should record: Agent changed code. ↓ Tests passed. ↓ Integration tests passed. ↓ Pull request merged. ↓ No incident occurred. Confidence should increase as independent evidence accumulates. For example: type Outcome = { status: "success" | "failure"; confidence: number; evidence: string ; }; Then: js const outcome: Outcome = { status: "success", confidence: 0.92, evidence: "unit tests passed", "integration tests passed", "pull request merged", , }; The graph should therefore store not just knowledge. It should store: knowledge + evidence + confidence. That distinction becomes extremely important at scale. Another subtle problem: An agent might make 50 observations while solving a task. Should all 50 become permanent memory? Probably not. We need to distinguish between: Something the agent saw. checkout.ts imports retry.ts Something that happened. PR 842 modified checkout.ts Something that happened after an action. Tests passed. A relationship that is useful beyond the original task. checkout.ts frequently changes with retry.ts A generalized strategy. When checkout tests fail with timeout errors, inspect retry behavior first. These are different levels of information. A good learning system should progressively promote information: Observation ↓ Evidence ↓ Repeated pattern ↓ Validated relationship ↓ Reusable knowledge That's much safer than throwing every agent thought into a vector database and calling it memory. RAG usually looks like: Question ↓ Search documents ↓ Retrieve chunks ↓ Send chunks to model ↓ Generate answer That's useful. But it primarily answers: What information is relevant? A graph can answer a different question: How are these things related? Consider: checkout.ts A document search might retrieve: PR 842 PR 811 PR 743 A graph can expose: checkout.ts │ ├── modified by → PR 842 │ │ │ ├── authored by → Maya │ └── reviewed by → Bobby │ ├── modified by → PR 811 │ │ │ └── related → retry.ts │ └── modified by → PR 743 │ └── caused → checkout incident Now the agent can reason over the structure. RAG gives you relevant documents. A graph can give you contextual relationships . The two are not competitors. They are extremely complementary. This might be even more valuable than learning from success. Suppose an agent tries: Approach A It fails. Then: Approach B It succeeds. We should record both. Task │ ├── attempted → Approach A │ │ │ └── FAILED │ └── attempted → Approach B │ └── SUCCEEDED Now future agents have something extremely valuable: negative knowledge. They don't just know what worked. They know what was already tried. That prevents agents from repeatedly walking into the same hole. Eventually, the architecture starts looking like this: ┌────────────────────────────────────┐ │ Agent │ │ │ │ Goal → Plan → Act → Verify │ │ ↑ │ │ │ │ ▼ │ │ Context ← Experience │ └───────────────┬────────────────────┘ │ ▼ ┌─────────────────┐ │ Engineering │ │ Graph │ ├─────────────────┤ │ Code │ │ People │ │ PRs │ │ Tests │ │ Incidents │ │ Decisions │ │ Outcomes │ │ Evidence │ └─────────────────┘ The agent operates on the software. The graph observes what happens. The graph accumulates relationships. The agent queries those relationships. And the cycle continues. This is where things get really interesting. We often think about an AI system improving through: Better model ↓ More training ↓ Better weights But an agent can also improve through: Better experience ↓ Better graph ↓ Better context ↓ Better decisions No model retraining required. The underlying model can remain exactly the same. What changes is the environment around the model . This is an important distinction. A powerful model with poor context can perform badly. A slightly less capable model with excellent context, tools, memory, and verification can perform surprisingly well. Now take this idea outside of one agent. Imagine a company where every engineering activity contributes to the graph: GitHub ↓ Pull Requests ↓ Code Changes ↓ Reviews ↓ Deployments ↓ Incidents ↓ Fixes ↓ Agent Experiences You eventually get something much larger than a dependency graph. You get a model of how the organization actually operates. You can ask: Who understands this system? Why was this architecture chosen? What usually breaks when this service changes? Which files are high-risk? Which engineers have solved similar problems? What approaches have already failed? What changed after the last incident? What should an agent inspect before modifying this service? These questions aren't answered by source code alone. They're answered by relationships across engineering history . The prototype we built is intentionally simple. A production system could add much more. Relationships change over time. Person A understood service X in 2024. Person B became the primary contributor in 2026. The graph needs to understand time. Not every relationship is equally trustworthy. Observed directly ↓ High confidence Inferred from repeated behavior ↓ Medium confidence Model-generated hypothesis ↓ Low confidence The graph can track: Agent ↓ Task ↓ Actions ↓ Outcome ↓ Time ↓ Cost ↓ Human review Now you can measure which strategies actually work. A human might tell the agent: Don't modify this service. The dependency is intentional. That becomes knowledge. The graph gets better. The next agent benefits. Now imagine several specialized agents: Code Agent Security Agent Testing Agent Incident Agent Documentation Agent All contributing to the same graph. One agent discovers something. Another agent can use it. That's when the graph becomes shared memory across an agent workforce. I think there is a fundamental shift happening in how we build software agents. The first generation of agents was mostly about: Can the model perform the task? The next generation is about: Can the system learn from performing the task? Those are very different problems. A single agent completing one task is useful. An agent that improves the context available to the next agent is much more powerful. And an organization where thousands of engineering actions continuously improve a shared knowledge graph starts to look like something new. Not just an AI assistant. Not just a chatbot. Not just RAG. A learning system . The core architecture is surprisingly simple: Agent ↓ Acts ↓ Produces evidence ↓ Updates graph ↓ Graph provides better context ↓ Agent makes better decisions ↓ Acts again That's the loop. And the loop is the product. The biggest mistake we can make with AI agents is treating every task as an isolated conversation. Software isn't isolated. People aren't isolated. Decisions aren't isolated. Failures aren't isolated. Everything is connected. A pull request connects to files. Files connect to services. Services connect to incidents. Incidents connect to fixes. Fixes connect to engineers. Engineers connect to decisions. Decisions connect to outcomes. And those relationships contain something incredibly valuable: experience. The job of a self-improving engineering system is to capture that experience, verify it, connect it, and make it available when the next decision needs to be made. The model provides reasoning. The tools provide action. The graph provides memory. The feedback loop provides improvement. Put those four things together and you get something far more interesting than an agent that can execute a task. You get an agent that can learn from doing the work. This is the idea behind Helix : an engineering intelligence layer that builds a living graph of your software, engineering activity, decisions, and evidence so AI can understand what happened before it decides what to do next. If you're building AI agents for serious engineering work, see what Helix is building → https://www.helix-engineering.dev/