{"slug": "karpathys-llm-wiki-as-agent-memory", "title": "Karpathy’s LLM Wiki as Agent Memory", "summary": "Andrej Karpathy's LLM Wiki design is being adopted as an external memory architecture for AI agents, mapping raw source documents into a structured markdown knowledge base. The system uses three layers—immutable raw sources, a maintained wiki of interlinked files, and a schema file—to support semantic, episodic, procedural, and other memory types across agent sessions. Developers are implementing the design to give agents persistent storage for facts, events, and workflows without modifying original source materials.", "body_md": "Lately, I’ve been down a rabbit hole exploring [agent memory](https://angiejones.tech/agent-memory/).\n\nWhat makes it so fascinating is all of the various types of memory we need to consider when building agents.\n\nConversational |\nStores the messages exchanged between the user and assistant so the agent can refer back to prior turns in the conversation. |\nSemantic |\nStores durable facts and meanings that should outlive the exact conversation where they were learned. |\nEpisodic |\nStores events: what happened, when it happened, what actions were taken, and what outcomes occurred. |\nProcedural |\nStores knowledge about how to do things, including workflows, rules, steps, and reusable processes. |\nEntity |\nStores facts about specific people, accounts, projects, systems, objects, or other named entities. |\nWorking |\nStores temporary information used while reasoning through the current task; usually short lived and not meant to persist forever. |\nSummary |\nStores compressed versions of longer conversations, documents, threads, or contexts so the agent can retain the important points without replaying everything. |\n\nAt work, I’m building agents to handle various operational tasks and have found [Karpathy’s LLM Wiki design](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) to be an excellent solution for implementing most types of memory for my agents.\n\nThe LLM wiki is a simple yet powerful collection of interlinked markdown files that are updated as the LLM gains more knowledge.\n\n## LLM Wiki Architecture\n\nThe wiki itself is one of three layers that make up the system.\n\n`raw sources → wiki → schema`\n\nYou provide the original sources, the LLM maintains a wiki derived from those sources, and a schema file tells the LLM how to consistently maintain the wiki.\n\n### Raw Sources\n\nThe first layer is the raw sources, which is the input the agent consumes (e.g. documents, articles, notes, etc). Raw sources are considered the immutable source of truth. The agent can read them, summarize them, and use them as evidence, but it should not modify them.\n\n### Wiki\n\nThe second layer is the wiki itself which is the structured markdown knowledge base the LLM maintains. This is where the agent starts connecting the dots across everything it has seen and gradually creates a knowledge base it can keep coming back to.\n\nIn addition to the knowledge pages, the wiki also includes two special files: `index.md`\n\nwhich helps the agent (and human) navigate the knowledge base, and `log.md`\n\nwhich preserves a chronological record of ingests, queries, lint passes, and other activity.\n\n### Schema\n\nThe schema is the instruction layer that tells the agent how to maintain the wiki. This is usually an `AGENTS.md`\n\nfile explaining conventions, workflows, and structure the agent should follow to keep the wiki intact.\n\n## How the Wiki Becomes Memory\n\nWhile Karpathy’s LLM wiki is not branded as a formal agent memory architecture, it maps surprisingly well to how agent memory works in practice.\n\nIt treats a maintained wiki, supported by raw sources and an instructional schema, as an external memory layer by giving the agent somewhere persistent to store, refine, and retrieve knowledge across sessions.\n\nThe raw sources are the evidence layer. They are not really memory in the agentic sense but they are the original records of information. They can contain facts, events, conversations, procedures, and entity details, but they are unprocessed. Think of them as the things memories are built from.\n\nTogether, the wiki and its schema support several memory functions:\n\n**Semantic memory:** stores durable facts and synthesized knowledge**Entity memory:** contains pages about people, projects, systems, etc**Episodic memory:** captures logs and records of what happened**Summary memory:** stores compressed versions of longer material**Procedural memory:** encodes how the agent should maintain the wiki\n\nThe schema defines the agent’s memory maintenance or “dreaming” routines, with linting the wiki being one example.\n\n## Example of LLM Wiki as Memory\n\nHere’s an example of what this looks like in practice.\n\nI am the program chair for several global conferences this year, which means I have to stay on top of trends, speakers, and sessions – thousands of them. I built an agent that reads raw event materials then maintains a wiki with all of this data.\n\n```\nevent-memory/\n├── AGENTS.md                    # Schema: how the agent maintains the wiki\n├── raw/                         # Immutable source materials \n│   ├── events/                 # Raw event-specific materials\n│   └── sources/                # Raw external/public sources\n└── wiki/                        # LLM-maintained knowledge base\n    ├── index.md                 # Content catalog of wiki pages and summaries\n    ├── log.md                   # Chronological record of ingest and updates\n    ├── events/                  # Summaries: dates, tracks, themes\n    ├── sessions/                # Session pages: title, abstract summary\n    │   ├── building-reliable-agents.md\n    │   └── evaluating-agent-workflows.md\n    ├── speakers/                # Speaker pages: bio, affiliation, sessions\n    │   ├── speaker-a.md\n    │   └── speaker-b.md\n    ├── sources/                 # Summaries from articles, reports, discussions\n    ├── trends/                  # Evolving patterns over time\n    └── topics/                  # Topic summaries\n```\n\nTogether, the raw sources, schema, and wiki form the agent’s long term memory layer. This helps the agent understand how trends are evolving, what topics should be covered at a given conference, and which sessions address current trends.\n\n### Entity Memory\n\nThe wiki provides entity memory through its pages for speakers, sessions, events, topics, and other named things the agent needs to reason about.\n\n```\nwiki/speakers/angie-jones.md\nwiki/sessions/building-mcp-servers.md\nwiki/events/mcp-dev-summit-2026-mumbai.md\nwiki/topics/authorization.md\n```\n\nEach page gives the agent a place to keep track of what it knows about that entity and how it relates to everything else in the system.\n\nFor example, the agent may know:\n\n- which speakers are presenting at an event\n- which sessions belong to a topic area\n- which topics are appearing across multiple events\n- what company or project a speaker represents\n- which sessions a speaker has delivered\n- how speakers, sessions, topics, and events connect to broader trends\n\nOver time, those relationships become just as valuable as the individual facts themselves. Instead of seeing a collection of disconnected pages, the agent can understand how people, events, sessions, and ideas fit together.\n\nThat is a clear demonstration of entity memory.\n\n### Semantic Memory\n\nTrend pages are perhaps the clearest example of semantic memory because they capture patterns and meanings extracted from many sources over time.\n\nThe agent looks across what’s happening in the ecosystem then stores the meanings across many events.\n\nThe trend pages tell the agent things like:\n\n- MCP builders are increasingly concerned with controlling what tools can do, who/what can invoke them, how actions are approved, and how execution is audited across trust boundaries.\n- Tool-loading and context-budget pressure remain active practical pain points.\n- Agentic AI trust increasingly depends on operational controls and observability, not just orchestration or autonomy.\n\nSemantic memory also appears throughout the rest of the wiki. Topic pages, speaker pages, session pages, and source summaries all preserve information that can be reused across future events and future agent sessions.\n\n### Episodic Memory\n\nEpisodic memory is the “what happened, when, and what changed” layer of the system. In my example, it shows up mainly in log.md, dated source refresh pages, and trend history records.\n\nThis allows me to interrogate the agent with questions such as when a topic first appeared or when a trend started phasing out.\n\n### Summary Memory\n\nSummary memory appears throughout the wiki as the agent continuously compresses larger sources into more useful knowledge artifacts.\n\nFor example, to gather trends, the agent reads live raw material from all over the web. It then compresses it into consumable summaries. Instead of preserving every post and comment from a discussion thread, it writes a summarized source page like:\n\n`wiki/sources/mcp/mcp-remote-runtime-auth-registry-sprawl-refresh-2026-06-01.md`\n\nThat page is a compressed representation of a larger evidence set. It’s effectively turning all that noise into knowledge the agent can build on.\n\nThe same pattern shows up across the wiki in several places. Session abstracts become summarized session pages, speaker biographies become short speaker pages, and collections of related sessions become topic summaries.\n\nSummary memory is key for performance because in future sessions, agents don’t need to re-read every raw source every time. They can start from the summarized source page, then inspect raw links only when needed.\n\nAs the wiki grows, retrieval becomes increasingly important. Small wikis can often be navigated through the index alone, but larger ones may benefit from search tools, embeddings, vector search, reranking, or other retrieval mechanisms. In this model, however, those technologies support the memory system rather than define it. The memory is the maintained body of knowledge itself. Search simply helps the agent navigate it.\n\n### Procedural Memory\n\nLike any memory system, the wiki requires maintenance. The schema layer (`AGENTS.md`\n\n) functions much like procedural memory by encoding how knowledge is merged, updated, corrected, and occasionally retired as new evidence arrives.\n\nIt has rules for:\n\n- avoiding duplicate pages\n- decaying stale trends\n- distinguishing one-off event observations from reusable cross-event knowledge\n- updating existing canonical pages instead of creating clutter\n- logging meaningful operations\n- keeping the index current\n\nAll of this is handled by the agent itself, not the human. This beautifully implements Karpathy’s “LLM as wiki bookkeeper” idea.\n\n## The Memory Was There All Along\n\nA wiki is such a simple concept, while memory is quite a complex one. Yet at the end of the day, both are really forms of knowledge management.\n\nThe real power of the LLM Wiki pattern is that it turns scattered information into a living memory system, even if it isn’t marketed as one.\n\nThe structure itself reveals the memory model:\n\n```\nAGENTS.md          # procedural memory\nwiki/\n├── events/        # episodic + summary memory\n├── sessions/      # entity + summary memory\n├── speakers/      # entity memory\n├── sources/       # summary + episodic memory\n├── trends/        # semantic + episodic memory\n├── topics/        # semantic memory\n└── log.md         # episodic memory\n```\n\nThat said, this pattern has not replaced every kind of memory for me. I’ve used LLM wikis with several agents, and I have not seen them implement conversational or working memory within the wiki. Those are usually short-lived memory types that my agents only need in-session, so the agent has not needed to preserve them as long-term memory.\n\nFor persistent memory, though, the wiki pattern has been the most practical approach I’ve used.", "url": "https://wpnews.pro/news/karpathys-llm-wiki-as-agent-memory", "canonical_source": "https://aaif.io/blog/karpathys-llm-wiki-as-agent-memory/", "published_at": "2026-06-08 17:05:51+00:00", "updated_at": "2026-06-11 18:48:40.501204+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "artificial-intelligence", "machine-learning", "ai-tools"], "entities": ["Karpathy", "Angie Jones"], "alternates": {"html": "https://wpnews.pro/news/karpathys-llm-wiki-as-agent-memory", "markdown": "https://wpnews.pro/news/karpathys-llm-wiki-as-agent-memory.md", "text": "https://wpnews.pro/news/karpathys-llm-wiki-as-agent-memory.txt", "jsonld": "https://wpnews.pro/news/karpathys-llm-wiki-as-agent-memory.jsonld"}}