I've covered several RAG methods before, and this post is about methods that are a bit different from traditional RAG systems, but still involve retrieval. They are good examples showing how retrieval methods for AI has evolved.
In Apr 2026, Andrej Karpathy introduced an interesting approach to knowledge management called LLM Wiki. You can read original writing of Karpathy here: Karpathy's LLM Wiki
Traditional RAG systems often retrieve relevant information from databases using vector similarity, as I've covered in other posts in this series. But the LLM may need to repeatedly retrieve and synthesize the same information for different queries.
Karpathy proposed a different approach: What if an LLM continuously maintained a structured knowledge base instead of processing raw documents from scratch for every query?
The idea is to build a persistent, interconnected collection of Markdown documents that an LLM automatically creates, updates, and organizes.
In LLM Wiki, the LLM processes incoming documents and integrates their information into an existing knowledge base. It summarizes new documents, extracts important concepts and entities, and updates the existing knowledge. And then, it creates relationships between related topics.
Importantly, when it notices contradictions or outdated information, it can update or correct its wiki pages (during both ingesting new sources and answering the questions.)
The key concept is that the system makes its knowledge base evolve.
Karpathy has proposed the LLM Wiki architecture consisting of three layers.
| Layer | Description |
|---|---|
| Raw Sources | Original documents, papers, articles, and other materials that remain unchanged |
| Wiki | LLM-generated Markdown pages containing summaries, concepts, entities, and cross-references |
| Schema | Instructions defining how the LLM should organize, update, and maintain the wiki |
The schema can be implemented by AGENTS.md, CLAUDE.md or instruction prompts if you want to use an LLM API on your own.
A simple directory structure might look like this:
knowledge-base/
├── raw/
│ ├── paper_a.pdf
│ └── article_b.md
├── wiki/
│ ├── index.md
│ ├── concepts/
│ │ ├── kv-cache.md
│ │ └── prefix-caching.md
│ └── sources/
│ ├── paper_a.md
│ └── article_b.md
├── AGENTS.md or CLAUDE.md
└── log.md
The LLM maintains the wiki, while the raw sources remain available for validation or additional context.
Karpathy describes three primary operations for maintaining the knowledge base.
1. Ingest
When a new document arrives, the LLM reads it, summarizes it, extracts important information, and integrates it into existing wiki pages.
It will connect the new concepts or entities with existing ones, update existing knowledge, or correct the information when it finds some discrepancies.
2. Query
When a user asks a question, the LLM navigates the wiki, identifies relevant pages, and generates an answer based on the retrieved knowledge.
Importantly, it can correct or strengthen the wiki pages if it finds useful insights or critical errors and discrepancies. So generated answers can be incorporated into the wiki. For example, the wiki doesn't explicitly show relationship between A and B, but the LLM finds out the relation while it navigates the wiki pages. Then LLM can write about the relationship information in the wiki. This is one of the key differences from conventional RAG pipelines.
3. Lint
As the wiki grows, inconsistencies or outdated knowledge can emerge.
Therefore, the LLM periodically reviews the wiki to identify contradictory information, outdated claims, missing relationships between pages, important concepts without dedicated pages, knowledge gaps requiring additional research, etc.
And as better models become available, they may improve the quality of this maintenance.
You can construct the wiki through AI assistants like Codex or Claude Code. After inputting the instruction in AGENTS.md or CLAUDE.md, ask them to build the wiki based on your raw sources. Or you can just use the LLM through API.
Actually, building the wiki is not that difficult. But a tricky part is how you navigate the wiki pages. When you use this only for yourself, you can just ask Claude code or Codex to navigate on their own. But if you are building production-ready products, you would need to build a navigation process too. You can use exact-match counts and rank the pages according to the numbers. However, as your knowledge gets large and complicated, you might need more sophisticated navigating methods. For example, you can incorporate other retrieval methods into your wiki system.
I've built the wiki from scripts of Demon Slayer episodes.
This is one of the wiki pages. You can see related pages and the findings from my questions.
If you want to utilize the LLM Wiki, I recommend using Obsidian. It is a very useful app for knowledge management and note-taking with markdown format.
Obsidian is a good fit for the LLM wiki because of this graph-view feature. You can see how your wiki pages are related to each other in the graph. With this app, you can easily manage and see your knowledge base.
Although the approach is promising, several challenges remain.
Information Loss: Summarization may omit details depending on your models' performance.
Maintenance Overhead: You have to keep updating and validating your knowledge base. This requires additional costs.
Scalability: Navigating a large wiki may eventually require more sophisticated indexing and retrieval mechanisms.
For reliable applications, it is important to preserve original documents, maintain source references, and validate important updates.
These days, frontier models can handle very long context, even over 1 million tokens. But still they can suffer from performance deterioration due to excessively long context. This is called context rot.
In December 2025, researchers from MIT introduced Recursive Language Model (RLM). (They revised the paper in May 2026.)
Rather than feeding an entire document into an LLM at once, RLM allows the model to programmatically explore the context, divide it into smaller pieces, and invoke additional LLM calls to process selected information.
In fact, the central idea of RLM is very simple: Treat the input context as an external object, and let LLM explore it.
The methodology is only about three pages in the paper. In the proposed implementation, the input context is stored as a variable in a Python REPL environment. Then the root model generates and executes Python code to examine the data. It can search for specific keywords or patterns, extract relevant sections, divide a large context into smaller chunks, and invoke sub-agents to analyze the chunks.
That is, the system becomes like a programmer searching files with Python. It can even let sub-agents do some parts of the job. In a typical data analysis tool, documents are divided according to predefined rules and processes. But RLM allows the LLM to determine its own processing strategy. For example, if it fails to retrieve the relevant information, it can try another strategy and recursively rerun the retrieval process.
I've made a demo of RLM with the existing database, and let's see how it retrieves information.
I've created root agent using gpt-6.1-terra model and sub-agents with my local model qwen3.8:27b.
Caveat: The agents will write code and run python REPL on your computer. But we don't know what code they will run. Mostly, they would be fine, but we can't be sure they always run safe code. So I'd recommend making them run on containers. I've run it in my docker container for this example.
I've made the RLM system ingest whole Demon Slayer scripts from season 1 to 4. I tried to tokenize them through the OpenAI tokenizer page, and it's about 423k tokens.
I asked this question to RLM:
[root] node_start / gpt-6.1-terra
task:
How have Demon Slayer Corps fought against the upper moon demons?
Then let's see how the RLM explores the database.
[root] repl_ready
[root] step
[root] model_usage / gpt-6.1-terra
[root] python [cell-0001]
code:
print([(d['id'],d['title'],len(d['text'])) for d in documents])
[root] observation [cell-0001]
output:
[('s01e01', 'Demon Slayer S.1 E.01', 23608), ('s01e02', 'Demon Slayer S.1 E.02', 22640), ('s01e03', 'Demon Slayer S.1 E.03', 21502), ('s01e04', 'Demon Slayer S.1 E.04', 19949), ('s01e05', 'Demon Slayer S.1 E.05', 19126), ('s01e06', 'Demon Slayer S.1 E.06', 18489), ('s01e07', 'Demon Slayer S.1 E.07', 18126), ('s01e08', 'Demon Slayer S.1 E.08', 24764), ('s01e09', 'Demon Slayer S.1 E.09', 24077), ('s01e10', 'Demon Slayer S.1 E.10', 21203), ...
it first retrieves the document titles.
[root] step
[root] model_usage / gpt-6.1-terra
[root] python [cell-0002]
code:
for id in ['s02e06','s02e07','s02e15','s02e16','s02e17','s02e18','s03e04','s03e05','s03e07','s03e08','s03e09','s03e10','s03e11']:
d=next(d for d in documents if d['id']==id); print('\n',id, 'paragraphs',len(d['paragraphs'])); print(str(d['paragraphs'][:2])[:500])
[root] observation [cell-0002]
output:
s02e06 paragraphs 481
[{'number': 1, 'text': 'Demon Slayer S.2 E.06'}, {'number': 3, 'text': 'Title: Akaza'}]
s02e07 paragraphs 371
[{'number': 1, 'text': 'Demon Slayer S.2 E.07'}, {'number': 3, 'text': 'Title: Set Your Heart Ablaze'}]
Then it looks into some episodes.
[root] step
[root] model_usage / gpt-6.1-terra
[root] python [cell-0003]
code:
for q in ['simultaneously','both.*head','Upper Rank Three','Upper Rank Six','Upper Rank Five','Upper Rank Four','hundred years','true body','burn.*blood']:
print('\n###',q)
for r in search(q,window=2,limit=7): print(r['text'] if isinstance(r,dict) else r)
[root] observation [cell-0003]
output:
### simultaneously
### both.*head
[s01e11:p0197] Crow
[s01e11:p0198] Start running! Start running! Tanjirou, Zenitsu, run!
[s01e11:p0200] Both of you head over to your next destination!
It has made a function and tried to search using some keywords.
[root] step
[root] model_usage / gpt-6.1-terra
[root] python [cell-0005]
code:
for id in ['s02e06','s02e07','s02e15','s02e16','s02e17','s02e18','s03e05','s03e08','s03e09','s03e10','s03e11']:
d=next(x for x in documents if x['id']==id); print('\n###',id)
for p in d['paragraphs']:
if re.search(r'(behead|decapitat|seven(th)? form|flame breathing|sun will|sunlight|poison|head[s]? (off|at|together)|neck[s]? (at|simultaneous|off)|Upper Rank|little (one|demon)|main body|real body|red blade|blood demon art)',p['text'], re.I): print(p['number'],p['text'][:220])
It uses regex to search.
[agent-1] node_start / qwen3.8:27b-mlx
task:
Compare Corps fights with upper-rank demons: tactics, participants and outcomes. Cite labels, distinguish allies from Corps.
context:
[s02e06:p0260] (Thoughts) Upper… Three? What’s he doing here right now?
[s02e06:p0261] [Action: Akaza launches himself forward and prepares to hit Tanjirou.]
[s02e06:p0262] Rengoku
[s02e06:p0263] (Thoughts) Flame Breathing, Second Form!
And then, it starts assigning a task to a sub-agent. The root agent gives some context to the sub-agent and make it search on its own.
[agent-1] step
[agent-1] final_turn
[agent-1] model_usage / qwen3.8:27b-mlx
[agent-1] node_end
answer:
The available subcontext covers only a few upper-rank demon fights, so a full cross-corpus comparison is not possible; here is what the evidence supports, distinguishing Corps (Demon Slayer Corps) fighters from non-Corps allies.
1) Upper Moon 3 Akaza vs. Rengoku (Corps) + Tanjiro/Inosuke (allies, not Corps): Rengoku uses Flame Breathing Second Form 'Rising Scorching Sun' to slice Akaza's arm, but Akaza regenerates instantly; Rengoku identifies him as an Upper Rank by his overwhelming aura [s02e06:p0260-p0273]. Akaza prioritizes killing the wounded Tanjiro to remove interference [s02e06:p0278]. Rengoku blocks a lethal strike to Akaza's solar plexus [s02e07:p0176-p0180]; Akaza, facing imminent daybreak, decides he must kill and flee [s02e07:p0184-p0187]. Tanjiro tries to decapitate the demon [s02e07:p0204]; Akaza rips off his arms and stomps, pushing Tanjiro and Inosuke back [s02e07:p0239-p0241]. Outcome: the fight is unresolved in this excerpt, with daybreak forcing Akaza to retreat.
2) Upper Moon 6 Gyokko vs. Tokitou (Corps) + Gyuutarou (ally): Gyokko uses 'Demon Blood Art… Killer Fish Scales' to attack Tokitou [s03e09:p0188-p0191]; Tokitou is weakened and overwhelmed by smaller
... [truncated 1,358 characters; full text in saved trace]
The sub-agent completes its job and returns the result.
[root] step
[root] model_usage / gpt-6.1-terra
[root] node_end
The Corps have fought Upper Rank demons with Hashira-led swordsmanship, teamwork and tactics tailored to each demon’s abilities. The results have varied:
- **Akaza, Upper Rank Three:** Flame Hashira Rengoku fought him with Flame Breathing and tried to hold him until sunrise. Tanjiro and Inosuke joined the attempt, but Akaza escaped; Rengoku died from his wounds. [s02e06:p0263-p0273] [s02e07:p0160-p0254] [s02e07:p0325-p0329]
- **Gyutaro and Daki, Upper Rank Six:** Sound Hashira Tengen Uzui, Tanjiro, Zenitsu and Inosuke coordinated to behead the siblings at the same time. Uzui used his Musical Score technique to read and deflect Gyutaro’s attacks; Tanjiro beheaded Gyutaro while Zenitsu and Inosuke beheaded Daki. Nezuko, a demon ally, then helped neutralize the fighters’ poison. [s02e15:p0245-p0258] [s02e17:p0364-p0389] [s02e18:p0023] [s02e18:p0140-p0147] [s02e18:p0221-p0250]
- **Gyokko, Upper Rank Five:** Mist Hashira Muichiro Tokito used Mist Breathing’s Obscuring Clouds to confuse Gyokko and decapitate him. [s03e09:p0198-p0241] [s03e09:p0271]
- **Hantengu, Upper Rank Four:** Love Hashira Mitsuri Kanroji held off his powerful fighting form while Tanjiro, Genya and Nezuko pursued the hidden main body. Tanjiro tracked it by scent and eventually beheaded it, destroying the demon. [s03e10:p0120-p0122] [s03e10:p0439-p0482] [s03e11:p0562-p0578] [s03e11:p0657-p0680]
These battles show why simple beheading was not always enough: the slayers had to discover *which* body to cut, or coordinate attacks on more than one target. [s02e15:p0252-p0258] [s03e10:p0120-p0122]
After iterating the processes a few times, finally, it combines all of the outputs and returns a final answer. This looks perfectly acceptable. It found relevant information across the scripts having over 423k tokens.
task:
When Tanjiro visited the headquarter of the corp, there were 9 hashiras. But why did only 7 hashiras go into the castle at the end?
I also asked this one requiring a lot of chunks of context. First, it should find out who entered the castle and who didn't, and then it has to know a reason why each missing one couldn't enter.
answer:
Two of the original nine Hashira were no longer active by the time of the Infinity Castle sequence: [s01e23:L1093-L1097]
- **Kyojuro Rengoku, the Flame Hashira, died** fighting Akaza after the Mugen Train battle. [s02e07:L0167-L0173] [s02e07:L0235-L0237] [s02e07:L0863-L0867]
- **Tengen Uzui, the Sound Hashira, retired** after the Entertainment District battle. He lost his left hand and left eye and said he could no longer fight. He later helped train Demon Slayers, but the available script doesn’t establish where he was when the castle opened. [s02e18:L1289-L1299] [s04e01:L1871-L1893]
The other seven were still active. They didn’t choose to enter the castle: Nakime opened it beneath them during their confrontation with Muzan. [s04e08:L0865-L0903] [s04e08:L0905-L0931]
I had thought it could have failed, but it managed to answer very well.
It iterated the root agent and sub-agent processes several times, but we can see the RLM can retrieve relevant information from large dataset by itself. I didn't command anything. I just input the database and let it search.
It would be difficult to retrieve all relevant information from large dataset only with the traditional RAG. A basic vector-based RAG system typically retrieves chunks that are semantically similar to your query. But RLM actively writes code, runs it, and orchestrates sub-agents on its own. Of course, it can introduce additional overhead from code execution and recursive LLM calls, but in some cases, it would be worthwhile.
I think LLM wiki can work well with RLM because they look complementary. When we fail to get information from existing wiki pages, we have to look for raw sources. We can use traditional RAG, but for better search, I believe it would be better to utilize RLM in that case.
If the raw sources are too huge, we can first apply the RAG for filtering and allow it to retrieve a lot of chunks. Then RLM will navigate the retrieved chunks.
Like this, RAG, RLM and LLM Wiki are not competing alternatives. Rather, they are complementary approaches. There are plenty of posts claiming "RLM kills RAG!" or "LLM Wiki makes RAG obsolete!". However, most of these claims are exaggerated clickbait. None of these approaches has replaced the others. What matters most is designing a knowledge-base architecture that leverages each approach where it makes the most sense for your system.
RAG, graphRAG, RLM, LLM Wiki: they're all, in one way or another, approaches to helping LLMs access and use relevant information from external sources. There might be more knowledge-base or retrieval related approaches beyond these. Fundamentally, the need for these approaches stems from a limitation of today's LLMs: they don't automatically update their model parameters based on each new interaction in real time. Even when we feed new information into an LLM, it doesn't actually update its parameters to incorporate that knowledge. Instead, it simply processes the information within its context window, giving the impression that it has learned something new.
That's why we need external knowledge bases to support long-term memory. But storing information is only half the battle. We also need retrieval systems that can find the right information based on the context and the user's question. Actually, this is very tricky.
It is easy to build a knowledge-base demo or personal project for myself, but when it comes to production-grade systems, it's an entirely different story. Do we need a separate knowledge base for each user? That could get expensive. Should we allow users to share some of their knowledge? If so, how do we handle privacy, security, and legal concerns surrounding personal information? Where should we store the data? Should we periodically prune outdated information or reset the knowledge base? And can we keep all of this within our budget? There are so many things to consider.
Lastly, I want to give a round of applause to the researchers and developers behind these methods. They've tackled challenges related to knowledge management and memory in AI, each with their own approach.
As I write this, I can't help wondering whether we'll someday have AI systems that can remember what they've seen and been told in real time, just like humans do, without relying on a separate knowledge base. I hope that day comes soon—even if it makes this entire post obsolete! 😂