Original idea, credit: gist.github.com/karpathy/llm-wiki. This follows that gist's structure section for section, but written as a how-to for someone who wants to actually set one up, based on running one of these for about ten weeks.
Most ways of putting documents in front of an LLM work like this: you upload a pile of files, the LLM searches them when you ask a question, and it generates an answer from whatever it finds. That's fine for one-off questions, but nothing is retained between sessions. Ask something that depends on five documents together, and the LLM re-does that assembly work from scratch every single time. Nothing compounds.
The alternative: instead of just searching raw documents at query time, have the LLM build and maintain an actual wiki — a folder of markdown pages, cross-linked, that sits between you and your source material. When new material comes in, the LLM doesn't just file it away for later search. It reads it, pulls out what matters, and updates the wiki: revises the relevant topic pages, adds cross-references, flags anything that contradicts what was already there. The wiki is a standing asset that gets richer over time, not a search index you re-query from zero.
You don't write the wiki. You curate what goes in, direct what the LLM should pay attention to, and ask the questions. The LLM does the actual writing, filing, and cross-referencing — the parts that are tedious enough that a human stops doing them consistently after a few weeks.
Three folders, three jobs.
Raw sources — everything you're feeding in: clipped articles, meeting notes, your own drafts, PDFs. This folder is read-only from the LLM's side. It never rewrites its own sources. This is what makes the wiki trustworthy — you can always go back to what was actually said.
The wiki — markdown pages the LLM owns entirely: page-per-source summaries, page-per-topic syntheses, comparisons, an overview page. It creates these, updates them as new material touches them, and keeps them cross-linked. You read this layer; you don't maintain it.
The instruction file — one file (CLAUDE.md
for Claude Code, `AGENTS.md`
for Codex-style tools) that tells the LLM the folder layout, what metadata every page carries, and what to do for each kind of request. This is the actual configuration of the whole system — get this right and the rest follows.
One addition worth planning for before you start: if your material splits into more than one theme (in my case, work-strategy material versus general reading), decide the routing rule for that split up front, and put it in the instruction file as a table, not a paragraph. I didn't do this early enough and had weeks of pages filed under the wrong framing before I caught it.
Three core things the LLM does, plus one that isn't in the original idea but that I found necessary once the wiki got automated.
Ingest. You point the LLM at a new source and say "ingest this." It reads the source, writes a summary page, updates any topic pages the source touches, updates the index, and appends an entry to the log. Define this as a strict contract in the instruction file — not "summarize and file appropriately," but an explicit list of exactly what gets touched and what gets produced. Vague instructions get inconsistent results across sessions; a fixed contract doesn't.
Query. You ask a question. The LLM reads the index to find relevant pages, reads those pages, and answers with citations back to the wiki. If the answer itself is worth keeping — a comparison, an analysis, something you worked out together — file it back into the wiki as a new page instead of letting it disappear into chat history. That's how your own exploration compounds alongside the material you feed in.
Lint. Periodically, have the LLM audit the wiki itself: pages nobody links to, claims that newer material has superseded, topics mentioned repeatedly but with no page of their own, broken cross-references. Have it flag these, not silently fix them — a wiki that quietly edits itself on every pass is harder to trust than one that tells you what it noticed and waits for a decision.
Challenge (the addition). Every substantive page gets a section where the LLM argues against its own conclusions — the counterargument, the weakest assumption, what the strongest opposing case would say. Without this, a wiki that only ever synthesizes agrees with itself very fluently and never gets stress-tested.
Ingest, query, lint — all three assume you're the one initiating. You feed something in, or you ask something. There's a fourth mode worth having, and it's the one I'd actually miss most if I had to cut it: the wiki periodically decides what's changed and pushes a synthesis at you, unprompted, without you having to think to ask.
Mine does this daily. After the day's ingest, before anything else, it writes a short synthesis of what came in and why it matters — not a re-dump of the sources, an actual argument: what moved, what it means, what to do about it. Then a separate step converts that into audio and it's waiting on my phone before I leave in the morning.
The reason this earns a place next to the other three operations rather than living under "automation": query only returns value if you remember to ask something worth asking, and after the novelty of a new tool wears off, most people stop asking. A pushed digest doesn't have that failure mode — it shows up whether or not you thought to look, and because it's genuinely synthesized rather than just filed, it's worth attending to instead of skipping. Converting it to audio wasn't decoration — a page of text competes with everything else fighting for your eyes; a ten-minute episode has an empty slot already carved out (a commute, a walk) that nothing else is contesting.
Two things had to be true before this actually got used instead of becoming one more digest I learned to ignore. First, it has to scale to the day's real input — a quiet day gets a two-minute status check, not a padded-out episode dressed up to look like something happened; a fixed length forced onto every day is exactly what turns a digest into noise you tune out. Second, whatever gets fed to the audio step has to already read as something meant to be heard, not glanced at — bullet points and headers narrate badly, so the step right before generation is always a rewrite into plain prose.
The general shape probably outlives the specific implementation: a wiki rich enough to synthesize is rich enough to decide, on its own schedule, what's worth telling you — and to push that in whatever format you'll actually consume. Audio suited a commute. A different channel might suit yours better, but the principle — value extraction doesn't have to wait for a question — is the part worth keeping.
Two files carry the weight of making a growing wiki actually navigable.
index.md — one line per page, organized by category, updated on every ingest. When answering a query, the LLM reads this first to find candidate pages, then goes and reads those specific pages rather than the whole wiki. This alone scales to a few hundred pages without needing a real search engine.
log.md — append-only, one entry per action, in a fixed format (## [YYYY-MM-DD] operation | Title
works well — it's greppable: grep "^## \[" log.md | tail -10
gives you the last ten things that happened). This is what lets you and the LLM reconstruct recent history without re-reading everything, and it's the thing that makes a scheduled or automated version of this debuggable when something quietly stops working.
Not in the original idea, and the harder of the two paths — worth doing only if you actually want the wiki growing on days you don't open it.
If you go this route: capture material through some always-on channel (I use a Telegram bot that routes messages into the right raw folder by tag), run ingest on a schedule, and generate a periodic summary. Expect it to fail silently before it's reliable — mine ran into a scheduled job that didn't fire for four straight days because the only sign of failure was a log line nobody was watching. Add a separate, minimal daily check-in ("did the job run today, yes or no") before trusting any of this unattended. And don't let the automation carry its own paraphrased copy of the rules in a script — point every scheduled prompt back at the instruction file, or the copy will drift out of sync the first time you update the real rules and you won't notice until the output looks subtly off. Anything the automation discovers on its own — not material you explicitly fed in — should land in a review queue for you to approve, never straight into the wiki.
- Decide your page-type schema (source pages vs. topic pages vs. analysis pages, at minimum) before you've written fifty pages under one undifferentiated format. Re-sorting after the fact is real work.
- If your wiki tool resolves links by page title but your files are named some other way (dates, slugs), check that cross-links actually resolve — don't assume they do because the markdown has the right syntax.
- Keep the instruction file short enough to fully re-read at the start of every session. If it grows past a couple hundred lines, split it: a short "always loaded" card with the rules you need every time, and a longer reference file the LLM only opens when it's doing that specific operation.
- Don't let quiet stretches get the same treatment as active ones — if nothing new came in, say that in one line, don't generate a full synthesis that treats silence as if it confirms something.
- If the wiki ever surfaces action items, put an expiration on them: resolved, blocked, or explicitly dropped within a fixed window, or they pile up unreviewed.
Underneath these three systems is a single idea, and it's the reason "automation" undersells what's actually going on: the wiki isn't just executing a reading list I wrote once. It's building its own model of what I care about — from what I've already fed it and cited — and using that model to keep widening what it pays attention to, without me maintaining the list by hand. That's the difference between a filing system and something closer to a second brain: not that it remembers what I tell it (any wiki does that), but that it notices patterns in what I've shown interest in and goes looking for more of it on its own.
A research agent that goes and finds material on its own. The people and topics I care about opt in explicitly — a flag on their own page, plus whatever feeds they publish to. A few times a week it checks each one for anything new, judges every item against the same one-line bar as manual ingest ("does this change how I think"), and drops anything that clears it into a review queue — never straight into the wiki. Default is drop; the bar is written so that "interesting" isn't enough, it has to name the page it would change and how. The rubric lives in its own small file, separate from the main instruction file, so I can tighten or loosen what counts as worth keeping without touching the rest of the system. It's also the part most likely to fail quietly — a feed goes stale, a fetch step can't write where it thinks it can — so it logs a per-run summary, not just its output, and I treat "ran but found nothing" and "didn't run" as different states that need to be visibly different.
A second process that finds who to add to the first one. It scans the wiki pages the LLM just wrote for people it cited but isn't tracking yet, then asks itself whether each one clears a bar — a real individual, producing original work, with a discoverable feed, actually cited (not a passing mention) — before adding them to the watch list on its own. One line in a page's metadata, reversible by deleting it. This is the piece that actually closes the loop: citing someone in a summary is itself a signal of interest, and this step turns that signal into tracking without me having to notice I'd care and go add it manually. The interest model isn't a list I wrote — it's compounding out of the wiki's own history. Cite someone, get them tracked, see their next piece surface a week later, cite that too.
A dashboard that's a window, not a control panel. One generated page, rebuilt fresh each time it's opened, showing the state of everything else at a glance: when material last came in, what's sitting in the review queue and how long it's been there, whether the scheduled jobs actually ran, what's coming due next. It writes nothing back — pure read-only status, nothing to break. I built this after the pipeline once went quiet for several days with no visible sign anything was wrong except an absence I hadn't noticed. Now there's one page to check instead of piecing it together from several log files.
None of this would matter without the review-queue rule from the automation section above: everything the first two systems find is a candidate, not a fait accompli. An interest model that's wrong in a way you can't see is worse than no interest model — the moment it starts writing to the wiki directly instead of proposing, you've lost the ability to notice when it's drifted toward the wrong things.
The slow part of keeping a knowledge base current was never the reading — it was the bookkeeping. Updating every cross-reference, noticing when something new contradicts something old, keeping dozens of pages internally consistent. That's exactly the kind of work a person stops doing carefully after a few weeks, and exactly the kind of work an LLM doesn't get tired of. The failures I actually hit running mine were never the wiki losing coherence — they were plumbing: a missing environment variable, a scheduled job silently not firing, a link format that didn't resolve. Each is fixable once, permanently, with a note in a troubleshooting log. None of them recurred once fixed.
This describes a pattern, not a fixed spec — the exact folders, page formats, and how much (if any) you automate should fit your own material and how much you want to stay hands-on. Start small: one raw folder, one wiki folder, one instruction file with a single page type and one operation ("ingest this"). Feed it a handful of real sources, see what the pages look like once two sources touch the same topic, and expand the schema from what you actually needed — not from a plan made before you had any pages at all.