{"slug": "a-four-type-framework-for-llm-wiki-by-karpathy", "title": "A Four-Type Framework for LLM Wiki by karpathy", "summary": "A developer building a personal knowledge base using Karpathy's LLM Wiki framework discovered that storing only declarative knowledge (facts and definitions) is insufficient for expert-level reasoning. By analyzing teaching dialogues and expert decision-making, they identified four distinct knowledge types—declarative, procedural, experiential, and interaction—and added a 'mine' operation to extract reasoning paths and decision sequences from raw material. The work highlights that future AI improvements depend on capturing these deeper knowledge types, not just larger factual databases.", "body_md": "Karpathy's LLM Wiki is brilliant. You dump raw material in, an LLM extracts concepts and links them together, and you get a personal knowledge base that actually works.\n\nI built one. 100+ pages. It's great.\n\nBut I hit a wall that made me rethink everything.\n\nI asked my AI to act as a programming tutor. It could recite every concept perfectly.\n\nStudent: *\"I don't understand Promises.\"*\n\nAI: *\"A Promise is an object representing the eventual completion or failure of an asynchronous operation...\"*\n\nWrong answer. The right answer was: *\"Do you understand callbacks first? What about synchronous execution? What have you tried so far?\"*\n\nThe AI had knowledge. It had zero judgment.\n\nAnd then I realized why: **every single page in my wiki was the same type of knowledge.**\n\nLLM Wiki 1.0 stores declarative knowledge — facts, definitions, summaries. Things that answer \"What is this?\"\n\nBut think about what makes a human expert different from a textbook:\n\nA great programming mentor doesn't just know what Promises are. They know **why you teach callback → Promise → async/await in that exact order** — and never the reverse. That's not a fact. It's a reasoning path.\n\nA master astrologer doesn't just know what each star represents. They know **why you check 命宮 first, then 三方四正, when to prioritize 格局, when a palace is a consequence rather than a cause.** That's not a fact either. It's a decision sequence.\n\nAnd here's the kicker: **even knowing the reasoning path isn't enough.**\n\nWe annotated Anderson's (1972) Socratic tutoring dialogues — full 41-turn and 30-turn conversations, labeling every decision point. Knowing the 23 Socratic rules (the reasoning path) is one thing. Reading a complete dialogue — watching the expert set a trap, wait 15 seconds in silence, break their own rules when the student gets frustrated — is something else entirely.\n\nKnowing the recipe ≠ having watched the chef cook.\n\nAnd there's still one more type.\n\nStudent says: *\"I have no motivation lately.\"*\n\nA knowledge-based response: *\"Here are the top 5 causes of low motivation...\"*\n\nAn expert response: *\"When was the first time you noticed this?\"*\n\nThe expert isn't answering. They're diagnosing. They know that \"no motivation\" is a surface symptom — the real problem could be burnout, unclear goals, a specific failure, or something else. Until you know which, any advice is a guess.\n\n**That's four distinct types of knowledge:**\n\nLLM Wiki 1.0 only stores type 1.\n\nWashU researchers analyzed 98 real CS TA sessions — 17 hours, 8,203 utterances.\n\nSocratic questioning (guided reasoning, diagnostic probes): **0.6%.**\n\nTAs directly giving the answer: **75%.**\n\nThese TAs knew the method. They were trained. Under time pressure, they defaulted to giving answers anyway.\n\nKnowing the rules ≠ being able to execute them.\n\nThat gap — between knowing and executing — is exactly where procedural, experiential, and interaction knowledge live. If you don't store those types, you can't train them. If you can't train them, you can't execute under pressure.\n\nKarpathy's framework has one operation: **ingest** — extract facts from raw material.\n\nThat produces declarative knowledge beautifully. But you can't get reasoning paths, worked examples, or guidance strategies by looking for facts. You have to look for **decisions** — what did the expert choose, when, and what followed?\n\nWe added a second operation: **mine.**\n\nSame raw material. Completely different extraction target.\n\nOver two weeks, we mined five teaching case studies:\n\n**Procedural frameworks extracted:**\n\n**Experiential cases annotated (decision-point level, not summaries):**\n\n**Interaction pattern (emerging):**\n\nThe four-type distinction applies wherever expertise exists:\n\nIn every domain, experts have all four types. Knowledge bases only capture the first.\n\nThe next generation of AI won't be defined by larger knowledge bases.\n\nIt will be defined by better reasoning, better teaching, and better judgment.\n\nThose don't come from more declarative knowledge. They come from organizing knowledge differently.\n\n**Judgment isn't a knowledge problem. It's a knowledge-type problem.**\n\n*Built on @karpathy's LLM Wiki foundation. The idea of \"mine\" as a second operation is what's new here — ingest extracts facts, mine extracts decisions. If you're building an AI tutor, a knowledge system, or anything that needs judgment, the four-type checklist might save you months.*", "url": "https://wpnews.pro/news/a-four-type-framework-for-llm-wiki-by-karpathy", "canonical_source": "https://dev.to/rongrong/a-four-type-framework-for-llm-wiki-by-karpathy-5f1n", "published_at": "2026-06-28 12:32:33+00:00", "updated_at": "2026-06-28 13:04:47.243161+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-agents", "artificial-intelligence", "machine-learning"], "entities": ["Karpathy", "WashU", "Anderson", "LLM Wiki"], "alternates": {"html": "https://wpnews.pro/news/a-four-type-framework-for-llm-wiki-by-karpathy", "markdown": "https://wpnews.pro/news/a-four-type-framework-for-llm-wiki-by-karpathy.md", "text": "https://wpnews.pro/news/a-four-type-framework-for-llm-wiki-by-karpathy.txt", "jsonld": "https://wpnews.pro/news/a-four-type-framework-for-llm-wiki-by-karpathy.jsonld"}}