How Google's Open Knowledge Format turns YouTube transcripts into a linked, queryable knowledge base your AI agents can search and cite.
What is Open Knowledge Format and why does it matter for YouTube archives? #
Open Knowledge Format (OKF) is a specification from Google for structuring knowledge bases that AI agents can navigate on their own. It grew out of the “LLM Wiki” concept popularized by Andrej Karpathy: a wiki built and organized by a language model instead of by hand. OKF turns that idea into a shared standard, the same way MCP standardized how agents call tools and A2A standardized how agents talk to each other. With OKF, a knowledge base built by one person or team follows a predictable structure that any agent can read, meaning knowledge bases become shareable objects instead of one-off personal projects.
Applied to YouTube, this means an entire channel’s back catalog, potentially hundreds of videos, can be converted into a structured, cross-referenced archive that an AI agent can search, summarize, and cite with timestamps. Instead of scrubbing through hours of footage to find one explanation, you ask a question and get an answer pulled from the exact video and moment where it was covered.
TL;DR #
Open Knowledge Format (OKF) is a Google-defined standard for building agent-readable knowledge bases, extending the LLM Wiki concept popularized by Andrej Karpathy into something shareable and reproducible.YouTube transcripts become the raw material: every video is first converted into a markdown transcript with timestamps, preserving exact source citations.** Canonicalization is the core step**: an LLM scans all transcripts to find recurring concepts and entities (tools, frameworks, workflows) and merges variant phrasings of the same idea into single dedicated files.The result is a linked graph, viewable in tools like Obsidian, where concepts, entities, and source videos connect automatically, so a topic like a coding workflow links back to every video that discusses it.Setup for an end user is a single prompt: point an agent (Claude Code or similar) at the repository and README, and it clones the knowledge base and learns the OKF structure if it doesn’t already know it.Building the base yourself only requires a channel URL paired with a skill that runs the extraction and canonicalization pipeline, though transcript-pulling services vary in cost and reliability.Scale is a real constraint: including every tool or idea ever mentioned would bloat the base past the point of being navigable, so only concepts and entities that recur across multiple videos earn a dedicated file.
Remy doesn't write the code. It manages the agents who do. #
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
How does an OKF-based YouTube knowledge base actually work? #
The structure mirrors how a wiki organizes information, but every layer is plain markdown so both humans and agents can read it without special tooling.
At the top sits an index file. This is the first thing an agent reads, and it lays out the “themes” covered in the knowledge base, the high-level map of what kinds of questions the archive can answer. From there, the agent drills into two main folders: concepts and entities.
Concepts are ideas, frameworks, or patterns discussed across multiple videos, things like a named workflow or a recurring critique of over-engineered tooling. Entities are concrete tools, protocols, or products mentioned repeatedly, such as a specific agent framework or coding tool. Each concept or entity gets its own markdown file that aggregates everything said about it across the entire channel, with links back to the specific videos and timestamps where it was discussed.
Underneath all of that sits a raw folder containing full markdown transcripts of every video, timestamped sentence by sentence. This is the ground truth layer: when an agent cites a source, it can point to the exact moment in the exact video where a claim was made.
Because every file is markdown and every file links to related files, opening the knowledge base in a tool like Obsidian renders an actual graph view. Concepts connect to entities, entities connect to source videos, and clusters emerge that show how ideas relate to one another across a large body of content.
Why does canonicalization matter so much? #
Canonicalization is the step where an LLM reads across all transcripts at once and figures out what actually deserves a dedicated file. This is harder than it sounds because people don’t talk about the same idea the same way every time.
A single workflow might get called by three or four different names across different videos, a shortened acronym in one, a fuller descriptive phrase in another. Keyword matching alone can’t catch this. It takes an LLM reasoning about meaning, not just text, to recognize that these are all references to the same underlying concept and should be merged into one canonical file rather than scattered across duplicate entries.
The same logic applies in reverse: not everything mentioned in a video deserves a file. A tool referenced once in passing doesn’t get a dedicated entity page. Only concepts and entities that show up repeatedly across multiple videos make the cut. This filtering matters for scale. A knowledge base that tries to capture every passing mention becomes too large for an agent to navigate efficiently, which defeats the purpose. The goal is a base dense enough to be useful but curated enough to stay searchable.
What does using the knowledge base actually look like? #
In practice, querying looks like a normal conversation with an AI agent, except the answers come with sourced, timestamped citations instead of generic responses.
Remy is new. The platform isn't. #
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
A user can ask something broad, like a request to walk through an entire multi-step process end to end, and the agent will pull from the index, trace through the relevant concept and entity files, and assemble a step-by-step answer citing the specific videos where each piece was taught. Narrower factual questions, the kind that used to require watching a full video just to get one clear answer, resolve into a short summary with the option to click into the original source for more depth.
This also solves a practical problem for anyone following a channel with a large back catalog: it’s hard to keep up with hundreds of videos, and even harder to remember which one covered a specific detail. A searchable, linked archive turns that catalog into something closer to a reference manual than a stream of individual uploads.
How do you set one of these up for a specific channel? #
Getting access to an existing OKF knowledge base is designed to be a one-step process: a single prompt sent to an agent (running in something like Claude Code, or any agent framework built on top of a similar coding environment) that tells it to learn the OKF structure if it doesn’t already know it, then clone the target repository. Once that’s done, the agent is immediately ready to answer questions against the full archive.
Building a new knowledge base from scratch for a different channel follows a defined pipeline:
Extract transcripts for every video on the channel and convert them into markdown, preserving timestamps.Canonicalize across all transcripts to identify recurring concepts and entities, merging variant phrasings and filtering out one-off mentions.Generate linked files for each concept and entity, with citations back to source videos and timestamps.Build the index that gives an agent the high-level map of themes and where to start looking.
Skills built for this pipeline can take just a channel URL as input and run the whole process. The catch is transcript extraction. YouTube’s own Data API only exposes transcripts for a channel you own, so pulling transcripts from any arbitrary channel requires third-party services or tools. Some are paid and more reliable, others are free but less consistent, and the choice affects both the cost and the quality of the resulting knowledge base.
Is this worth building yourself? #
For anyone tracking a channel with a large archive, or trying to build a searchable reference out of their own video content, the tradeoff is straightforward: real compute and token cost upfront in exchange for a permanently queryable, source-cited archive afterward. Building out a full knowledge base for hundreds of videos requires meaningfully more LLM processing than a simple summary job, since canonicalization means reasoning across the entire corpus rather than one document at a time. The payoff is a structure that scales far better than manually skimming a playlist. Once built, the knowledge base doesn’t need to be rebuilt to answer a new question, and it keeps citations intact, so answers can always be checked against the original video and timestamp. For creators or teams sitting on a large body of recorded content, that combination of searchability and traceability is the main draw.
Frequently Asked Questions #
What is the difference between OKF and a regular vector database RAG setup?
A vector database retrieves chunks of text based on semantic similarity, without inherent structure connecting them. An OKF knowledge base is explicitly organized into an index, canonicalized concept and entity files, and a linked graph, so an agent can navigate relationships between ideas rather than just retrieving isolated matching passages.
Do I need to know the OKF spec to use a knowledge base built with it?
No. An agent can be told to learn the format on the fly if it doesn’t already know it, then load the knowledge base and start answering questions. The learning step happens once, as part of the initial setup prompt.
Can this work for channels other than my own?
Yes, but pulling transcripts for a channel you don’t own requires third-party tools or APIs, since YouTube’s own Data API restricts transcript access to your own channel. Options range from paid, more reliable services to free tools that may be less consistent.
Why doesn’t every tool or topic mentioned get its own file?
Including every passing mention would make the knowledge base too large to navigate efficiently. Only concepts and entities that recur across multiple videos get a dedicated canonical file, which keeps the base dense with signal rather than cluttered with one-off references.
What format are the files stored in?
Everything is plain markdown, including raw timestamped transcripts, concept files, entity files, and the index. This makes the knowledge base readable in any text editor or IDE, and viewable as a linked graph in tools like Obsidian.