Open Knowledge Format (OKF) vs. Vector Databases: Why RAG Needs a New Knowledge Format
Every developer building RAG pipelines eventually runs into the same wall.
You chunk your docs, embed them, throw them into a vector database, and run similarity searches. It works great for simple search bars, but it completely falls apart when an agent needs to reason about a real system. If your AI needs to figure out how a database schema relates to a deprecated API endpoint, standard RAG starts guessing.
The failure isn’t your embedding model; the problem is with chunking.
Splitting text every 500 tokens chops up document structures and throws away explicit links. Instead of following the connections your team actually wrote down, the model is left trying to mathematically predict what connects to what.
In practice, relying on vector chunking alone breaks in three main ways:
Orphaned Dependencies: Let's use an example to explain this; e.g. technical documentation is filled with cross-references. A document might state: “Per the compliance policy in Section 3, user logs must be purged after 30 days.” When you chunk this document, the sentence mentioning Section 3 often ends up in one vector embedding, while Section 3 itself ends up in an entirely different chunk. Unless the user’s query happens to explicitly target Section 3, semantic similarity search will retrieve the rule statement but completely miss the definition it relies on. The dependency becomes an orphan.
Context Fragmentation: Standard RAG pipelines use fixed-token chunking (e.g., splitting every 500 tokens with a 50-token overlap). This mechanical slicing ignores document structure and cuts paragraphs mid-sentence and strips away parent headers. If a chunk contains a code snippet or a database schema, the LLM loses the vital metadata context — such as whether that snippet belongs to a secure internal service or a deprecated v1 public API guide. Without the parent context, the model misinterprets the data.
Probabilistic Failure: For strict engineering schemas, financial formulas, or legal policies, retrieving “semantically similar” text often results in hallucinations. The agent needs the exact, official definition, not a probabilistic guess.
In this article, we will examine how OKF differs from vector databases, why pure semantic search fails for structured system reasoning, and how combining both frameworks could support a reliable, hybrid retrieval architecture for AI agents.
Note: OKF shares its acronym with the unrelated Open Knowledge Foundation, a long-running open-data nonprofit. The two have nothing to do with each other.
What is the Open Knowledge Format (OKF)?
The Open Knowledge Format (OKF) is an open, vendor-neutral file specification published by Google Cloud’s Data Cloud team. It defines a lightweight, file-system-based standard for packaging organisational knowledge into portable, interlinked concept files.
OKF is still early: Google published it as v0.1, explicitly described as “a starting point, not a finished standard,” so some of the conventions discussed here may evolve. Because OKF standardizes context for AI agents, developers frequently confuse it with vector databases.
Format: Plain-text Markdown documents paired with YAML frontmatter.
Key Mechanism: Enforces a minimalist metadata schema (requiring only a type field) and uses standard Markdown links to connect related files.
Purpose: Formalises the “LLM-wiki” pattern — a term coined by AI researcher Andrej Karpathy — into a predictable directory structure so AI agents and human engineers can traverse knowledge deterministically.
On the other hand…
What is a Vector Database?
A Vector Database is a specialised storage and retrieval engine designed to index, store, and query high-dimensional vector embeddings.
Format: It comprises of numerical floating-point arrays generated by embedding models (e.g., [0.012, -0.421, ...]).
Key Mechanism: It uses Approximate Nearest Neighbour (ANN) search algorithms (such as Cosine Similarity or HNSW) to evaluate the mathematical distance between vectors.
Purpose: It solves the “needle-in-a-haystack” problem by enabling fast semantic search across massive volumes of unstructured text, audio, or images without requiring exact keyword matches or predefined schemas.
So how do they differ: Vectors vs Concepts
In simple terms, a vector database is a search infrastructure while an open knowledge format (OKF) is a data schema.
Vector Databases store high-dimensional arrays and run mathematical similarity calculations to find text that shares semantic meaning with a user’s query.
OKF defines how knowledge is written and linked on a file system. It relies on standard Markdown, YAML frontmatter, and Markdown links to build a knowledge graph that agents can traverse deterministically.
A Proposed Hybrid Retrieval Architecture
In practice, both vector databases and OKFs are complementary rather than competing, because Graph traversal requires a starting point, and vector search lacks deterministic navigation.
A vector database will answer where to land in a massive knowledge base, while an OKF will answer where to go next using explicit relationships. So, we can use vector search to find an entry node, then follow OKF’s Markdown links to traverse the graph from there.
Execution Flow
[ User Query ] ──> [ Vector DB ] ──> [ OKF Entry Node ] ──> [ OKF Linked Nodes ] ──> [ LLM Context ]
Semantic Entry: The system queries the vector database using the user’s prompt. The database returns the single highest-matching OKF document (e.g., metrics/mau.md).
Deterministic Expansion: A retriever opens metrics/mau.md and parses its Markdown body for linked concepts (e.g., compliance/privacy.md and tables/user_events.md).
Graph Traversal: The retriever loads those linked concept files up to a defined depth boundary (e.g., max_depth=1) — a limit you'd enforce in your own retriever, since OKF itself doesn't define traversal depth.
Context Injection: The LLM receives complete, self-contained Markdown files with clear boundaries, full headers, and frontmatter categorisation.
This is one implementation strategy among several you could build on top of OKF — the format itself only defines how knowledge is structured and linked, not how a retriever should walk it.
Architectural Decision Matrix
Select your retrieval strategy based on the structure, scale, and precision requirements of your source data.
The Missing Structured Layer
Ultimately, building reliable AI agents requires more than just retrieving a semantically similar paragraph, it now requires providing the agent with an accurate map of how your system actually works.
Vector databases will continue to handle the heavy lifting of unstructured semantic search, but OKF provides a missing structured layer.
And by formalising the “LLM-wiki” pattern into a portable, open-source standard — published on GitHub alongside reference implementations and sample bundles — OKF aims to ensure that the documentation your engineering team writes is the same structure your agents use to reason. It’s early days for the spec, but worth watching as v0.1 matures.
References & Further Reading
Google Cloud Data Cloud Team (2026). Open Knowledge Format (OKF) Specification v0.1. GitHub Repository. github.com/google-cloud/open-knowledge-format(Note: Replace/confirm link if using a specific internal/official spec release)
Lewis, P., Perez, E., Pichtulis, A., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS 2020). arXiv:2005.11401
Malkov, Y. A., & Yashunin, D. A. (2018). Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World (HNSW) graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4), 824–836. doi:10.1109/TPAMI.2018.2889473
Karpathy, Andrej. The “LLM-Wiki” Pattern and Agent Context Management. Keynote & Public Notes on LLM Memory Architectures.
Open Knowledge Foundation. Open Knowledge Definition & Open Data Principles. okfn.org(Clarification note: Unrelated open-data nonprofit sharing the OKF acronym)