LLM Wiki v3, an AI-first implementation spec for building a persistent memory operating system for agents.

Builds on Andrej Karpathy's LLM Wiki and Rohit Ghumare's LLM Wiki v2. This document is self-contained. You do not need to read v1 or v2 first.

This document is for a frontier model (GPT-5.5, Claude Opus, etc.) to implement the full system. The first build defines folder structure, schema, verifier, claims layer, ingest rules, and graph conventions. After the foundation is built, smaller models can maintain it. Substantive foundational changes still need the strongest available model.

/goal Build a complete LLM Wiki v3 memory system from this specification. Use a frontier model for the initial implementation. Do not make a demo, sketch, or partial scaffold. Create the full folder structure, domain schema, source digest format, claims layer, index, hot file, log, verifier, graph projection, source delta scanner, privacy checks, autonomy hard gate, silent watchdog, and ingest protocol. Seed the wiki from the user's chosen raw source folder if one is provided. Run verification before reporting completion. If any step cannot be completed, report the blocker and leave the system marked incomplete.

The wiki must let an AI agent:

• Start every session by reading current operating state.
• Ingest sources without losing provenance.
• Extract atomic claims that can be audited later.
• Promote claims into concepts, entities, procedures, syntheses, decisions, comparisons.
• Connect sources, claims, and pages with wikilinks.
• Regenerate a graph projection from Markdown files.
• Run deterministic verification.
• Refuse completion when sources are orphaned, claims unlinked, links broken, or privacy checks fail.
• Give the human a clear report of what changed and what still needs judgment.

Failure gates: folders + placeholder files only = fail. Source digests without claims or links = fail. Claims of verification without verifier output = fail.

Atomic claims. Every assertion is a separate traceable unit with provenance, confidence, and status. Wrong claims get updated without rewriting pages. Retracted sources flag every dependent claim.Deterministic verification. A script returns specific counts. Zero issues = green. Anything else = work not done.Graph projection. Entire wiki projects into a JSON graph (optional Kuzu DB). Agents query relationships, trace provenance, find contradictions, walk impact chains without reading every file.MCP access. Local MCP server exposes the graph as query tools for any MCP-compatible agent.Source delta scanning. SHA256 hashes on raw sources detect new, modified, deleted files automatically.Session crystallization. After every meaningful session, save a concise record, extract durable claims, update related pages.Operating memory. Tracks current context, active work, next actions via a hot file read on every session start.

The wiki is not a vector database. It is not a notes folder. It is the agent's operating memory — a structured, verified, claim-linked system that compounds knowledge across sessions instead of re-retrieving fragments.

Folder of original materials. Agent reads but never modifies. Corrections live in the wiki. SHA256 hash detects changes. raw/

is often a symlink to a Documents folder — the wiki points to files, tracks hashes, doesn't duplicate.

Every assertion as a separate claim with: unique ID (claim-YYYY-MM-DD-NNN

), plain-language statement, confidence (high/medium/low), provenance (source(s)), status (active/contested/superseded/stale), supersession links. Claims live in claims/active.md

, claims/contested.md

, claims/superseded.md

, claims/low-confidence.md

. Claims must NOT be buried inside concept pages.

Concepts, entities, comparisons, queries, syntheses, decisions, procedures. Reference claims for provenance. Chain is always walkable: page → claim → source → raw file.

SCHEMA, index, hot file, log, taxonomy, graph projection, dashboards, review queue, lint reports. Agent reads SCHEMA + hot first every session. Health is a deterministic script.

raw/

— immutable source materialsources/

— source digests written by the agentclaims/

— atomic assertions- Durable wiki pages (concepts, entities, syntheses, comparisons, procedures, queries, decisions)

• Operating layer (schema, index, hot, log, taxonomy, verifier reports, graph projection, dashboards, review queues)

llm-wiki/
  SCHEMA.md
  index.md
  hot.md
  log.md
  raw/
  sessions/
  sources/
  claims/
    active.md
    contested.md
    superseded.md
    low-confidence.md
  entities/
  concepts/
  comparisons/
  queries/
  syntheses/
  decisions/
  procedures/
  templates/
  scripts/
  _meta/
    taxonomy.md
    topic-map.md
    graph.md
    review-queue.md
    open-questions.md
    source-quality.md
    raw-binary-index.md
    source-delta.md
    source-inbox-deltas.md
    source-inbox-deltas.state.json
    lint-reports/
    dashboards/
    graph-projection/
  _archive/

Portability rule: foundational upgrades to the wiki operating model belong in this public specification, not only in one local vault. Public documentation must be corpus-neutral: describe patterns and tooling, not private source names, counts, or contents.

SCHEMA.md

= constitution. hot.md

= working state. index.md

= root map. log.md

= audit trail. claims/

= auditable assertion layer. _meta/

= operating system.

Every file is plain Markdown. No database. No platform dependency. No lock-in.

Index rule: root index.md

is the global map. Scoped indexes are allowed for a project folder, domain hub, or workflow-specific entry page only when they serve a different navigation job. They are harmful when they repeat the root index, exist only to satisfy a verifier, or create a second source of truth. Every index must declare its scope and job. Machine inventories and coverage reports belong in _meta/

with descriptive names.

Domain-specific schema including: domain definition, page types, file naming rules, frontmatter rules, tag taxonomy, claim rules, confidence rules, source quality rules, ingest completion rules, privacy rules, verification requirements.

Priority when instructions conflict: user's current instruction first → security/data integrity → verified state → schema.

Under 50 lines. Contains: current focus, active source batches, open blockers, next best actions, recent verification state, human decisions needed.

Human and agent map of the wiki. Pages listed by type with one-line summaries. Minimum sections: Entities, Concepts, Comparisons, Syntheses, Procedures, Queries, Decisions, Sources. Updated whenever global pages are created, renamed, deleted, or materially changed. Scoped indexes updated when their folder/project/domain/workflow changes.

Append-only audit trail. Every meaningful operation gets a dated entry: ingest, update, query, verify, graph, delete, retire, privacy review, source delta scan. Format: "YYYY-MM-DD action | what changed, what files changed, verification run." Be specific.

Every durable wiki page must have frontmatter.

Concept/entity/synthesis/comparison/procedure/query/decision pages:

---
title: Page Title
created: YYYY-MM-DD
updated: YYYY-MM-DD
type: concept | entity | comparison | query | synthesis | decision | procedure
category: optional-domain-category
tags: []
sources: []
claims: []
confidence: low | medium | high
maturity: seed | developing | stable | canonical | stale
status: active | contested | superseded | stale | archived
supersedes: []
superseded_by: []
related: []
---

tags

from _meta/taxonomy.md

. sources

references source digests. claims

references claim IDs in claims/active.md

.

Source digests:

---
title: Source Title
source_url: absolute path or URL
author: author if known
ingested: YYYY-MM-DD
sha256: body hash
source_type: paper | article | book | transcript | documentation | personal-note | webpage | gist | session | other
source_quality: low | medium | high
authoritativeness: primary | secondary | commentary | unknown
immutable: true
---

sha256

computed over body, not frontmatter. On re-ingest, recompute and compare. Skip if unchanged, flag drift if changed.

## claim-YYYY-MM-DD-NNN, short claim title

- Claim: Plain language atomic claim.
- Type: factual | conceptual | procedural | decision | preference | inference
- Status: active | contested | superseded | stale | archived
- Confidence: low | medium | high
- Sources: [[sources/source-slug]]
- Supports: [[concepts/page-slug]], [[entities/page-slug]]
- Contradicts: none
- Supersedes: none
- Superseded by: none
- Last confirmed: YYYY-MM-DD
- Notes: Limits, caveats, or why this matters.

Claims must be atomic — one assertion per claim. Do not turn summaries into claims.

high

: multiple independent sources support the claim, or one authoritative primary source directly establishes it.medium

: one credible source supports it, or several weak sources point the same way.low

: inferred, weak, old, contested, unclear, or based on uncertain extraction.

Do not mark high just because prose sounds confident. Single-source claims usually start as medium.

• Read SCHEMA.md

. - Read hot.md

. - Read index.md

. - Check last 20-30 entries of log.md

. - Navigate to relevant pages via index or graph — do not guess or skip.

• Drop source into raw/

(immutable, never edited after placement). - Write source digest into sources/

. - Extract atomic claims into claims/

. - Update or create concept/entity pages that reference those claims.

• Update graph projection, index.md

,log.md

. - Run verifier — do not report done if it fails.

• Read SCHEMA and hot file.

• Search index and graph for relevant pages.

• Read pages and their claims.

• Synthesize answer with citations back to sources.

• Mark confidence explicitly.

• File as durable query or synthesis page if worth keeping.

• Save concise record under sessions/

. - Extract durable claims into claims/

. - Update related concepts, entities, procedures, syntheses.

• Update index.md

,hot.md

,log.md

.

• Run verifier — fix broken wikilinks, missing frontmatter, orphan pages.
• Run graph projection.
• Run system health check.
• Run source delta scan.
• Flag stale claims for review.
• Merge duplicate concepts.
• Archive superseded pages.

Light ingest: raw source + source digest + log entry. Only when user explicitly asks. Status: staged, not complete.** Standard ingest**: light + update entities/concepts/index/log.** Deep ingest (default)**: standard + atomic claims, graph updates, contradiction check, review queue, synthesis.

If the user says "ingest this," "add these files," or "process this corpus," perform deep ingest unless the user explicitly says light. A source digest alone is staged evidence, not a completed ingest.

• Source digest exists with provenance and hash.
• Atomic claims exist, or source explicitly marked non-claim-bearing.
• Relevant concept, entity, synthesis, procedure, or decision pages link to the source.
• Claims link back to pages they support. index.md

,hot.md

,log.md

, and relevant_meta/

files updated.- Graph projection regenerated, orphan detection passes.

• Verifier passes.

Do not report ingestion complete before all seven gates pass.

For large corpora, separate extraction from promotion, but status must remain incomplete until promotion and verification finish. Use language: "Extraction complete, promotion pending." / "Source digests staged, full ingest not complete." / "Orphan detection still pending." Never hide staged status.

Parallel extraction creates one dangerous failure mode: each worker produces locally valid claims that become invalid when merged. Worker outputs are temporary staging files, not durable wiki pages.

Correct parallel merge:

• Assign each worker a non-overlapping claim ID range before extraction, or renumber every staged claim during merge.
• Strip worker frontmatter, batch headings, and local indexes before writing to claims/active.md

. - Normalize every claim field to canonical labels: Claim

,Type

,Status

,Confidence

,Sources

,Supports

,Contradicts

,Supersedes

,Superseded by

,Last confirmed

,Notes

. - Normalize every source and support target into wikilinks ( [[sources/source-slug]]

,[[concepts/page-slug]]

,[[entities/page-slug]]

). - Deduplicate claims by source and assertion, not by claim ID alone.

• Rebuild claims:

frontmatter list andclaims-count

from actual claim headings in body order. - Rebuild concept and entity pages from merged claim graph so sources:

,claims:

, key claim sections, and source coverage sections point to real pages. - Delete temporary batch files after merge and verification.

• Regenerate graph projection after merge, not before.
• Run hard gate. Not clean = ingest not complete.

Do not append batch files directly into claims/active.md

. That creates duplicate claim IDs, embedded batch frontmatter, stale field formats, and false graph health.

For large or changing source folders, use a promotion worker before full ingest. Packet contents:

• Delta summary: added, modified, deleted, uncovered backlog counts.
• Selected source paths, normalized relative to wiki (no leaked absolute paths).
• Per-source readiness labels: text-ready, convertible, transcription-needed, OCR-needed, duplicate, manual-review.
• Copy-paste agent prompt restating full ingest gates.
• Completion gate: packet not complete until hard gate returns clean.

Selection: easiest safe promotion first — extracted text → text-layer docs → OCR-recoverable → media needing transcription → manual-review. Suppress duplicate raw files by content hash. The packet is a work order, not evidence. It becomes useful only after the agent creates/updates digests, extracts claims, wires pages, refreshes operating files, regenerates graph, and passes verification.

Fast path: local extraction first, then promotion. Use text extraction for files with text layers, OCR for scanned/image-only docs, ebook conversion for ebook formats, binary index for files that can't be text-extracted (archives, media, images, spreadsheets). Tool families: markitdown, OCRmyPDF+Tesseract, Calibre, ripgrep. AI should research exact setup for user's platform. Embeddings are optional — do not block ingestion on vector indexing.

• Catalog audio/video in binary index.
• Transcribe locally when relevant.
• Save transcripts in session/staging area, not in immutable raw.
• Create source digests from transcripts only after recording original media path, transcript path, tool family, ASR caveat.
• Extract claims from transcript digest.
• Link claims into existing pages, or create new durable page when media introduces a central model.
• Run hard gate.

Do not treat transcription as completion. A transcript is only extracted text. The source is not integrated until claims, links, pages, indexes, graph edges, and verification are clean.

Media transcript promotion path (validated end to end):

Selection. Identify media files from source delta scan. Prioritize by domain relevance. Group into packets of 5-8 files.Local transcription. Runfaster_whisper

(base.en, CPU int8). Save transcripts with timing segments undersessions/transcripts/<topic>/

. Each transcript gets YAML frontmatter (title

,type: transcript

,tags

,source_url

,status: staged

). Status tracking file tracks per-file progress.Source digest creation. For each transcript, create digest undersources/<slug>-transcript.md

withsource_type: transcript

,source_quality: medium

(ASR caveat),authoritativeness: primary

,raw_video_source

. Body includes prose summary, ASR tool/model notes, extract preview.Claim extraction. 4-6 atomic claims per transcript batch. Each getsclaim-YYYY-MM-DD-NNN

with type, confidence, sources, supports.Page wiring. Link each new digest and its claims into existing concept/entity/comparison pages viasources:

andclaims:

frontmatter arrays and body sections. Create new durable page only when topic is central and does not yet exist.Index and taxonomy. Add digests toindex.md

. Add new tags to_meta/taxonomy.md

. Recompute page counts.Log and hot file. Append dated log entry. Updatehot.md

.Verification. Run hard gate — must be CLEAN before reporting completion.

After digest creation:

• Read source digest.
• Extract atomic claims.
• Search existing pages for related concepts/entities.
• Update existing pages when possible.
• Create new pages only when topic is central or appears across multiple sources.
• Link sources to pages through sources:

frontmatter and body wikilinks. - Link claims to pages they support.

• Add contradictions to claims/contested.md

and_meta/review-queue.md

. - Update index, hot file, log, taxonomy, graph.

• Run verification.

Pages should not become dumps. Source digests preserve detail. Durable pages preserve current understanding.

Create a durable page when: concept appears in multiple sources, concept is central to one important source, entity matters to the domain, procedure will be reused, query answer would be expensive to recreate, decision must be remembered. Do not create pages for passing mentions. Do not create near duplicates — search first. Every durable page must have at least two meaningful links unless it is a deliberate singleton logged in the review queue.

Obsidian-style wikilinks. Examples: [[concepts/agent-memory]]

, [[entities/example-organization]]

, [[sources/source-title-slug]]

, [[claims/active#claim-YYYY-MM-DD-NNN]]

. Links must resolve. Verifier catches broken links.

uses

, depends_on

, improves_on

, extends

, contradicts

, caused

, fixed

, supersedes

, supports

, evidence_for

, example_of

, implemented_by

, owned_by

, part_of

Quick query: answer from wiki, no new files.** Research query**: answer + list missing sources/questions.** Synthesis query**: answer + save durable synthesis.** Audit query**: inspect confidence, contradictions, stale claims, duplicates.

• Never skip orientation — read SCHEMA, hot, index before acting.
• Never overwrite without logging — every change goes in log.md

. - Never claim completion without verification — run verifier.

• Never remove old knowledge silently — use supersession (mark old as superseded, link to new, explain why).
• Never launder LLM inferences as facts — mark confidence honestly.
• Never skip provenance — every claim traces back to a source.
• Never edit raw sources — raw is immutable, corrections live in wiki.
• Never create pages without index entries.
• Always prefer existing pages over new ones — check index first.
• Always update hot.md

when priorities change.

When new information replaces old:

• Preserve older source, claim, or page.
• Mark older status: superseded

orstatus: stale

. - Link old to new with superseded_by

. - Explain why newer claim is preferred.

• Add to _meta/review-queue.md

if human judgment needed.

Claim wrong → supersede the claim. Page obsolete → promote useful content elsewhere, clean backlinks, update index, then delete or quarantine. Do not keep dead pages in active graph. Archive aggressively: status: superseded

or status: stale

pages move to _archive/

within a week.

Create a graph projection from Markdown files. Minimum output:

• JSON graph file in _meta/graph-projection/

. - Nodes for pages, sources, claims, tags, source files.

• Edges for wikilinks, source references, claim support, contradictions, supersession, tags.
• Basic graph stats in _meta/graph.md

.

Optional: local Kuzu database, MCP server exposing graph query tools.

Every wiki change triggers a script that parses all Markdown frontmatter and wikilinks, projects into JSON graph with typed edges. The graph is a derived artifact — Markdown is canonical. If graph is corrupted or stale, re-run projection.

Graph capabilities: neighbor lookup, claim tracing, contradiction detection (edges typed contradicts

/supersedes

), impact analysis (retracted source → walk graph to find all dependent pages/claims).

Not required on day one for tiny wikis, but the projection script should exist. Once wiki exceeds ~50 durable pages, graph projection becomes important.

If agent runtime supports Model Context Protocol, expose local read-only graph tools: search nodes, get node neighbors, find claims for a page, find sources for a topic, graph stats, orphan sources, contested claims. Keep MCP local unless user explicitly asks for remote access.

Mandatory deterministic script. Zero issues = green. Anything else = work not done. Agent cannot claim completion without it. Checks:

• Required files exist
• Required folders exist
• Frontmatter parses
• Required frontmatter fields exist
• Tags exist in taxonomy
• Wikilinks resolve
• Source digests have provenance and hashes
• Duplicate source hashes flagged
• Claims in frontmatter match claim headings in body order
• Claim IDs unique after all batch merges
• Claim sources resolve
• Claim support targets resolve
• High confidence claims have adequate support
• Index includes all durable pages
• Log exists and has recent entries
• Orphan durable pages flagged
• Orphan sources flagged
• Claims that support no page flagged
• Privacy leaks flagged (without scanning generated reports containing prior scan snippets)
• Binary raw files indexed
• Stale/superseded pages flagged
• Graph projection exists and current
• Broken wikilinks
• Orphan pages (no inbound links)
• Malformed frontmatter
• Stale pages not archived
• Low-confidence claims without review notes
• Contradictions between claims
• Duplicate entities/concepts
• Oversized pages (>200 lines)
• Source drift (sha256 mismatch)
• Claims without provenance
• Pages without outbound links

Clean result: ISSUE_COUNTS {}

. If issues remain, report them and do not claim completion.

Verifier is wiki-local. Safe structural fixes applied automatically; ambiguous content changes go to review queue.

Some artifacts exist only to initialize a new wiki (packaged verifier, starter templates, example configs). Every bootstrap artifact must carry an explicit retirement condition. For the verifier: once scripts/llm_wiki_verify.py

exists inside the wiki, the packaged copy is dead — delete it, symlink to wiki-local copy, or replace with stub that delegates. Never leave two copies.

Health checks, cron jobs, watchdogs, and autonomy gates must reference the wiki's own verifier path and verify that path during their own run. Output must include source path so drift is visible.

Wraps the above checks into a single pass that also includes:

• Source delta scan
• Graph projection regeneration
• Full structural verifier
• Privacy/secret-pattern scan
• Orphan source/claim/page detection
• Optional MCP/graph connection check when configured

Outputs dashboard under _meta/dashboards/

and returns:

CLEAN

(exit 0)ACTIONABLE_FIXES_APPLIED

(exit nonzero, follow-up needed)BLOCKED_NEEDS_HUMAN

(exit nonzero, system cannot safely repair itself)SCRIPT_FAILURE

(exit nonzero, runtime failure)

Add a silent scheduled watchdog around the hard gate — silent when clean, emit only actionable failures.

The hard gate is the automation boundary. The hard gate owns the completion claim.

Treat every model edit as an untrusted transform until deterministic checks prove structure, provenance, claims, links, indexes, and graph edges survived.

Controls:

Risk classification. Mark high-risk: schema, claims register, root index, verifier, hard gate, graph projection, source-delta scanner, source digest batches, deletion/migration scripts, public-facing docs.Pre-edit checkpoint. Record structural snapshot: content hash, line count, frontmatter hash, wikilink target set, claim ID set, source reference set, graph-relevant identifiers.Post-edit comparison. Text may change, but claims, source refs, wikilinks, frontmatter semantics, graph edges must only change intentionally.Claim preservation rule. Any pre-existing claim must end in one of three states: still active, explicitly superseded, or deleted with logged reason. Silent claim loss = corruption.Checkpoint cadence. High-risk: verify after every edit. Ordinary: verify every 3-5 model interactions. No long edit chains without deterministic checkpoint.Semantic diff over visual diff. Check source refs, claim IDs, frontmatter, wikilinks, index membership, graph edges, duplicate sections, orphan creation.Round-trip tests where useful. For reversible transforms, edit then check if invariant set can be reconstructed. Expected: preservation of structure, provenance, claims, links, meaning — not exact prose equality.

Completion rule: if a risky edit cannot be verified, it is not complete.

Detect: source digests with zero inbound links, source digests linked only from redundant index pages, concept/entity pages with no inbound links, claims that support no page, pages that cite no source. A source should connect to at least one concept or entity. Acceptable exceptions must be logged.

The wiki watches the evidence boundary, stages new source material, promotes useful material into claims and durable pages, verifies the graph, and refuses to call the system clean when any required layer is broken.

Three jobs:

• Detect change — scan raw inboxes, transcripts, sessions, operating files for new/modified/deleted/uncovered material.
• Promote evidence — convert into digests, claims, pages, graph edges, indexes, review queues, log entries.
• Prove state — run one hard gate that decides whether wiki is clean, blocked, or broken.

If source is only extracted, state is not complete. If transcript has no claims/links/graph edges, state is not complete. If graph has orphans, broken links, missing index entries, stale dashboards, or privacy hits, state is not complete. Correct status is staged/blocked/failed until verification passes.

Scanner watches raw/

and reports: new, modified, deleted files; files needing text extraction/OCR; binary files needing cataloguing; files covered by digests; files staged but not promoted. Writes to _meta/source-delta.md

or _meta/dashboards/

. Computes SHA256 hashes, compares against stored state in _meta/source-inbox-deltas.state.json

.

Rules: use stable wiki-relative paths even when raw/

is a symlink; normalize older absolute-path state; detect duplicates by content hash; prioritize text-ready and convertible files in promotion packets; treat uncovered as promotion queue. Scanner must not modify raw files.

Before ingesting private material, check for: API keys, passwords, tokens, private addresses, medical details, financial records, legal material, third-party sensitive info, private corpus names that shouldn't appear in public docs. Sensitive folders excluded by schema — record in _meta/review-queue.md

or _meta/privacy.md

. Public documentation describes patterns and tooling, never private corpus details.

Exclusions (indexed but never ingested): archives (.zip, .tar.gz, .7z, etc.), video/audio files, sensitive folders, binary files (evidence-by-existence, referenced by path, not extracted unless requested).

• Read SCHEMA.md

. - Read hot.md

. - Read index.md

. - Search relevant pages and source digests.

• Read claims connected to the topic.
• Answer from wiki evidence.
• Cite pages/sources by wikilink.
• File durable answers as query/synthesis pages only when expensive to recreate.
• Update log.md

.

Do not answer from memory when wiki can be checked.

• Run verifier.
• Fix broken links.
• Fix invalid frontmatter.
• Update stale indexes.
• Regenerate graph projection.
• Run orphan detection.
• Run source delta scan.
• Review contested and low-confidence claims.
• Scan for redundant files and duplicate information.
• Classify redundancy as keep/merge/delete/quarantine.
• Merge duplicates into canonical targets before deleting.
• Retire stale pages cleanly.
• Update hot.md

andlog.md

. - Run verifier again.

Pass complete only when verification is clean or remaining issues are explicitly logged with reasons.

Prevent during every write:

• Orient first — read SCHEMA, hot, index, recent log.
• Search before creating — check existing pages, claims, digests, indexes, dashboards.
• Prefer update over duplication.
• Keep one canonical register per job — claims/active.md

is canonical; batch/backup/repair/generated files are temporary. - Use content hashes for source dedupe — same body hash → one canonical digest.

• No duplicate indexes — root index.md

is global; scoped indexes only with real local navigation job. - No placeholder pages — empty/stub/no-source-evidence pages must be completed, merged, or deleted.

• No stale generated reports in active paths — dashboards/packets are regenerated state.
• Consolidate before deletion — move unique facts/claims/links/provenance into canonical target, then remove redundant file.
• Clean references after deletion — backlinks, index entries, graph nodes, stale source pointers, dashboard references.
• Verify after cleanup — hard gate must be clean.

Classification: Keep (canonical sources, active claims, durable concept/entity pages, operating files), Merge (near-duplicate pages, overlapping digests, repeated claims, stale summaries with unique facts), Delete (temporary batch files, obsolete repair files, stale generated packets, empty placeholders, duplicate indexes with no local job), Quarantine (risky retired generated systems, old runtime exports, material needing review).

Completion proof: duplicate clusters reviewed, canonical targets chosen, unique facts preserved, temporary files removed, backlinks cleaned, indexes updated, graph regenerated, verifier clean.

Rules: never delete raw sources automatically, never delete without backlink cleanup, never delete without log entry, never leave broken links, never keep temporary batch/backup/repair/generated packet files in active paths after merge. Prefer supersession for claims, consolidation for duplicates, deletion or quarantine for dead generated pages after useful knowledge is promoted.

Open the wiki folder in Obsidian for graph view, backlinks, wikilink navigation, search, and frontmatter browsing (Dataview plugin). Obsidian is not required — plain Markdown works in any editor.

Required:

• Frontier AI model for initial setup (GPT-5.5, Claude Opus 4.7, or equivalent).
• Local folder for the wiki (Markdown files, no database/server/cloud).
• Python 3 (3.10+) for graph projection, health checks, verifier.
• AI agent that can read local files, run commands, follow instructions (Hermes, Claude Code, Cursor, etc.).

Recommended:

• Obsidian (visual editor with graph view, backlinks, dataview, search).
• ripgrep (fast wiki search).
• Kuzu Python package (local graph DB).
• markitdown (PDF, Word, PowerPoint → Markdown).
• yaml-lint (frontmatter validation).
• sha256sum (source drift detection).
• dataview Obsidian plugin (SQL-like queries over YAML frontmatter).

Cron: Schedule health check and source delta scanner weekly. Dashboards in _meta/dashboards/

.

You do not need to match this setup exactly. The only hard requirement is the folder structure and protocol.

Day one: Raw sources, wiki pages, index, log, SCHEMA (Karpathy v1 baseline).

Week one: Source digests + claims layer.

Month one: Graph projection, health checks, review queue.

Building from scratch: Start with folder structure. Write SCHEMA. Drop sources into raw/

. Follow ingest/query protocols. Add verification when ready. Graph and MCP can wait.

Already have an agent with memory: Add claims layer — extract assertions from existing concept pages into atomic claims with provenance and confidence. Then add source digests, then verification.

Agent uses skill/procedure system: Write a skill with wiki path, verification commands, and rule that verification must pass before completion. Pattern is additive.

Agent uses system prompts: Put 3-5 lines: wiki path, read before claiming, write source digests for new knowledge, run verifier after edits, don't claim done if checks fail.

Want graph access: Register MCP graph server. Optional but powerful at scale.

Want scheduled maintenance: Add weekly cron job for health check and source delta scan.

• Create folder structure. Write domain-specific SCHEMA.md

. Createindex.md

,hot.md

,log.md

. Create taxonomy. Create templates. Create claims files. Create source digest conventions.

• Write verifier. Add frontmatter validation. Add wikilink validation. Add index completeness check. Add claim validation. Add source hash checks. Add orphan detection. Add privacy scan. Run verifier and fix initial issues.

• Write graph projection script. Emit JSON graph. Emit graph stats. Add optional Kuzu support. Add optional MCP tools.

• Create source delta scanner. Create text extraction workflow. Create OCR workflow. Create binary index workflow. Create source digest writer. Create claim extraction protocol. Create concept/entity promotion protocol.

• Ask user for domain and raw source folder. Ingest small representative batch first. Run full ingest. Verify clean. Scale to larger batches.

• Update hot.md

with current state and next best action. Add autonomy hard gate as completion command. Add source delta scanning with wiki-relative paths and duplicate-content suppression. Add promotion packets ordered by easiest safe promotion first. Add silent scheduled watchdog. Document session-start protocol. Confirm verifier output. Report what changed, what remains staged, what is blocked.

Report: wiki path, domain, files created, scripts created, number of source digests, number of claims, number of concepts/entities, graph node and edge count, verifier output, known exclusions, remaining human decisions. No verifier output = not done.

• Folders but no verifier

• Placeholder pages with no source evidence

• Source digests but no claims

• Claims not linked to pages

• Pages updated but not index

• Files updated but not log

• Graph files not reflecting real wikilinks

• Treating embeddings as required for ingest

• Publishing private corpus details in public docs

• Calling staged extraction a complete ingest

• Orphaned source digests disconnected in graph

• Single-source claims marked high confidence without justification

• Duplicate concept pages (index not read first)

• Parallel batch outputs appended directly into claims/active.md

• Temporary batch files left in active claim folders after merge

• Duplicate claim IDs surviving merge

• Claim frontmatter drift from body headings

• Bold labels or bare paths instead of canonical claim fields and wikilinks

• Generated reports counted as privacy evidence after including stale prior scan snippets

• Stale generated pages left in active graph

• Redundant backups, batch files, repair files, generated packets, placeholder notes in active paths

• Duplicate indexes repeating root index without distinct local job

• Deleting duplicate files before preserving unique facts and cleaning backlinks

• Reporting success without showing verification Packaged verifier drift: Two copies of verifier produce different results. Prevent by following bootstrap self-retire rule — delete packaged copy after wiki-local copy exists. Health harness must reference wiki-local path and print that path in output.Delegation-chain corruption: Plausible edits compound without checkpoints, silently dropping claims/sources/links/confidence. Prevent by treating edits as untrusted transforms, snapshotting high-risk artifacts, comparing structural invariants post-edit, running hard gate at bounded intervals.

• Start small — don't pre-design everything. Drop sources in, let model create pages as it encounters real information, let SCHEMA evolve. hot.md

is your steering wheel — keep under 50 lines, update when priorities change.- Don't skip orientation — most duplicates and missed cross-references come from skipping read-before-write.

• Raw sources are immutable — never edit raw/

. Wrong source → create claim saying so. Outdated → supersede with newer source. - Use claims for anything you might disagree with later — cheap, hundreds are fine, make wiki auditable without rewriting pages.

• Archive aggressively — stale/superseded pages → _archive/

within a week. - Run verifier after every meaningful edit, not just maintenance days.

• Set up source delta scanning early — weekly scan of raw/

by SHA256 catches forgotten sources. - Keep taxonomy flat and small — 10-20 tags. Add more later if needed. log.md

is an audit trail, not a diary — every entry: date + action + subject.- Let frontier model write SCHEMA — give it this document, ask for domain-specific SCHEMA, then review.

• Graph projection can wait — index.md

• ripgrep enough until >50 pages. - Review queue is where human judgment lives — check _meta/review-queue.md

weekly. - Privacy check before every ingest — catch credentials, addresses, medical/financial details before they're written.

At session start, read SCHEMA.md, hot.md, index.md, and recent log.md from the LLM Wiki. Before creating anything, search for existing pages, claims, source digests, indexes, and dashboards to prevent duplicates. When new knowledge appears, create source digests, extract atomic claims, update concept/entity pages, update index/hot/log, regenerate graph projection, run orphan detection, scan for redundant files or duplicate information, consolidate or delete redundant material safely, and run the verifier. Do not claim ingest, cleanup, or wiki edits are complete unless the verifier passes or remaining issues are explicitly logged.

Build the system, not a description of the system. Use local files first. Keep raw sources immutable. Preserve provenance. Extract claims. Link everything. Verify deterministically. Keep public docs sterile. Report only what you can prove. If the wiki cannot pass verification, say exactly why and leave the task marked incomplete.

source & further reading

gist.github.com — original article but-it-happened.txt The Hooman Method — my attempt at capturing and formalizing how I use and combine chat and coding agents across my projects. Awesome Copilot Repo Onboarding Agent