Disclosure up front: I build
[agentproto], whose
knowledge and operator primitives are the back half of this piece. The problem
in the first half stands on its own, and the walkthrough uses real, checkable
commands. Corrections welcome — file an issue.
Open Claude Code. Open Codex. Open a cheap local model in the same repo. Ask all
three the same question about your codebase — your retry convention, your deploy
runbook, why that one service is the way it is.
You'll get three fluent, confident, near-identical answers. And all three will be
wrong in the same way: they'll describe how a reasonable team does it, not how
yours does. They're all running the same frontier weights, trained on the same
public internet, so out of the box they all know exactly the same thing.
The one idea, if you remember nothing else:
The weights are everyone's. Your knowledge base is the only part of the stack
that'syours— and the only thing that makes the agent yours instead of the
internet's average.
Here's the shift most people missed while arguing about benchmarks. The gap
between the best model and the tenth-best is collapsing — the hub piece
has the Stanford AI Index receipt (the #1-to-#10 spread fell from 11.9% to 5.4%
in a year). When the weights converge, the weights stop being your edge.
What replaces them is system design. One widely-cited 2026 essay on the evolution
of agents put it bluntly: agent capability has moved from model quality to system design — from writing a clever prompt to engineering what the model can
The receipt that reframes the whole market.In a production-readiness
comparison of agent frameworks, one finding held across every orchestration
style tested:retrieval quality dominates knowledge-agent performanceYou can have the best planner-worker-judge
regardless of orchestration.
topology in the world; if the agent is retrieving from the internet's average
instead of your world, it answers like a stranger.
So the interesting question stopped being "which model?" and became "what does it
know that no one else's does?" JetBrains' framing is useful here: some agents are
action-first (their value is what they do) and some are data-first
(their value is what they can access). A coding agent for your team is
data-first whether you designed it that way or not.
The uncomfortable part: if you haven't attached your knowledge to your agent, you're shipping the stranger.
Run the self-diagnosis before you build anything. There are three tiers, and most
teams are stuck on the first two without noticing.
Tier 0 — no knowledge base. The agent knows what the pre-training run knew:
public GitHub, Stack Overflow, docs up to some cutoff. Ask it about your world and
it pattern-matches to the nearest public example. It's confident and it's generic.
Ceiling: it will never know a single thing that isn't already on the internet.
Tier 1 — pasted into context. You dump the runbook, the conventions doc, a few
past PRs into the prompt (or a CLAUDE.md
) and hope. It works — until the context
fills up. Anthropic's context-engineering post is direct about why this caps out:
you have to treat context as a finite attention budget, because recall decays
as the window grows ("context rot").
Why Tier 1 has a hard ceiling.The same post prescribes the fix —keep, not pre-load everything.
lightweight references and retrieve just-in-time
Pasting your whole knowledge base into every prompt is the exact anti-pattern:
it's expensive, it rots, and it doesn't scale past a handful of documents. A
KB you paste is a KB you'll soon truncate.
Tier 2 — a served, queryable knowledge base. Your knowledge lives in one place,
gets indexed, and every agent queries it on demand — pulling only the three
relevant entries for the question at hand, with their sources attached. This is
just-in-time retrieval, done properly, shared across every agent you run.
The jump from Tier 1 to Tier 2 is the whole game — and it's the "who equips the agents" question from the files-with-contracts piece, pointed straight at knowledge. Let's build it.
Start with the packaged shape, because it shows the target. In agentproto, an
operator (the AIP-9 OPERATOR.md
) is a persona plus policies plus a set of
bound tools — and one of those tools is a knowledge binding.
---
name: House Engineer
id: house-engineer
profile:
role: Answer and act using OUR conventions and past incidents, never generic defaults.
voice: Concrete, cites the runbook or incident it's drawing from.
tools:
- knowledge-query # ← the bound knowledge tool
- shell
policies:
- "policy:engineering/house-rules"
metadata:
corpus:
knowledgeViews:
- corpus: engineering
filter: { tags: [conventions, runbooks, incidents] }
---
Read what that knowledgeViews
block does: it scopes the operator to your
engineering corpus, filtered to the tags that matter. The persona decides how it
answers; the policies decide what it's allowed to do; the knowledge view decides
what it knows. Three separate concerns, one file.
This is a real shape, not a mock.The research operators shipped in
agentproto's own corpus —source-scout
,research-analyst
— carry exactly
this:tools: [knowledge-query, ...]
plus ametadata.corpus.knowledgeViews
binding scoped by domain and quality score. The operator is thepackaging.
Underneath it is a tool, and the tool is the part that ports to any agent.
An operator is what "give the agent your knowledge" looks like when it's one declared file. Now let's build the tool that backs it — the part that reaches
This is the exact move from the files-with-contracts piece,
specialized to knowledge: split the contract (what the tool promises) from the
driver (the code that answers). Here's the contract — a pure defineTool
,
no execute body.
// tools/knowledge-search/TOOL.ts — the contract, and nothing else
import { defineTool } from "@agentproto/tool"
import { z } from "zod"
export const knowledgeSearch = defineTool({
id: "knowledge.search",
description:
"Answer from OUR knowledge base: conventions, runbooks, past incidents. " +
"Returns refined entries, each with the source it came from.",
inputSchema: z.object({
query: z.string(),
tags: z.array(z.string()).default([]),
limit: z.number().default(5),
}),
outputSchema: z.object({
entries: z.array(z.object({
kind: z.string(),
title: z.string(),
body: z.string(),
source: z.string(), // provenance — where this knowledge came from
})),
}),
mutates: [], // read-only → no approval gate
approval: "auto",
})
Now the driver — the implementation, bound to the contract by implementTool
. It
reads your corpus workspace off local disk and returns the matching entries.
// drivers/knowledge-corpus.ts — backed by your own corpus workspace
import { defineDriver, implementTool } from "@agentproto/driver"
import { knowledgeSearch } from "../tools/knowledge-search/TOOL.js"
import { resolveKnowledge } from "@agentproto/corpus"
import { NodeFsAdapter } from "@agentproto/corpus/fs"
export default defineDriver({
id: "knowledge.search.corpus",
name: "Knowledge search over your corpus workspace",
kind: "in-process",
implementations: [
implementTool(knowledgeSearch, async ({ input }) => {
// the same resolver behind `corpus knowledge <ws> --tags … --max …`
const hits = await resolveKnowledge({
fs: new NodeFsAdapter({ root: process.env.CORPUS_WS! }),
query: { tags: input.tags, maxResults: input.limit },
})
return {
entries: hits.map((h) => ({
kind: h.kind,
title: h.title,
body: h.body,
source: h.sources.join(", "),
})),
}
}),
],
network: { egress: [] }, // reads local files → your knowledge never leaves the box
})
That driver is the real corpus resolver — the same resolveKnowledge
call that
sits behind corpus knowledge <ws> --tags conventions,incidents --max 5
on the
command line, now exposed as a tool. The network.egress: []
line is doing quiet
but load-bearing work: the knowledge is queried from local files, so it never leaves your machine.
Provenance is a feature, not decoration.Every entry carries itssource
—
the runbook, the incident, the PR it was distilled from. That's what lets an
external check —[the supervision ladder's]whole
point — verify the answer later: the LLM-as-a-judge literature evaluates RAG on
twosides, context relevance going in andfaithfulness coming out(did the
answer actually follow the retrieved source, or wander off it?). No provenance,
no faithfulness check.
One contract, one driver, and now every agent that mounts the daemon can answer from your world. Serve it.
Start the daemon. It reads your tools/
and drivers/
and serves them over MCP,
knowledge.search
included.
npm i -g @agentproto/cli
agentproto serve # serves the /mcp gateway to every agent
Now spawn an agent with the tool, and ask it something it could only know from
your docs — a convention set by a past incident:
agentproto sessions start codex --cwd . \
--prompt "What's our retry policy for the payments queue? Use knowledge.search."
→ Per incident #2026-04 (double-charge on gateway timeout), payment retries are
capped at 3 with jittered backoff, and we NEVER retry a charge without an
idempotency key. Source: runbooks/payments-queue.md, incidents/2026-04.md
Now the same agent, same question, without the tool:
agentproto sessions start codex --cwd . \
--prompt "What's our retry policy for the payments queue?"
→ A common approach is exponential backoff with 3–5 retries and a dead-letter
queue after the final attempt…
The second answer isn't wrong — it's the internet's average, and it's exactly
what you'd get from any agent, anywhere. The first answer knows about the
double-charge you shipped in April, because it read your incident file. **That delta — a real past incident it could only learn from your knowledge base — is the entire difference between an agent and **
Cross-vendor, by construction.That was Codex. Point Claude Code or a cheap
local model via Hermes at the same daemon and they call the identical
knowledge.search
— because it's served over MCP, not baked into one vendor's
config. Author the KB tool once; every agent in your fleet inherits your world.
Here's the honest framing, because overclaiming is how you lose a reader. In
Guilde — our AI-company product — attaching a knowledge
base to an agent is one packaged step: an operator ships with an attached
corpus, and you apply it to a scope in a single command. Persona, policies,
knowledge, done.
What you just built by hand in agentproto is that same primitive, one floor down:
an operator (Step 1) plus a served knowledge tool (Step 2). Guilde adds the one-command packaging; agentproto is the from-primitives path you can run today,
What agentproto doesThere's no singlenotdo yet, stated plainly.
apply-knowledge-pack
verb in the open CLI — you wire the tool, the driver,
and the operator yourself, as three files. That's more assembly than the
packaged version. It's also fully inspectable and fully yours, which for a
knowledge base — the most sensitive thing you'll attach to an agent — is often
the trade you want.
The primitive is identical either way: an operator with a queryable knowledge tool. One product packages it; the other hands you the parts.
A knowledge base is not magic, and three caveats keep it honest.
A wrong KB is worse than no KB. If your knowledge base contains a stale
convention, the agent will state it with the same confidence as a correct one —
now with a source attached, which makes it more persuasive and more dangerous.
This is why the faithfulness check above matters: retrieval quality is the whole
ballgame, and a KB is an artifact you maintain, not a bucket you fill once.
Don't let the agent write its own knowledge base.ETH Zurich research
(Gloaguen et al.) found that LLM-generatedAGENTS.md
files providednoA knowledge base curated from real
benefit and could reduce success rates.
incidents, runbooks, and decisions is yours; one the agent hallucinated about
itself is just the internet's average with your repo's name on it. Curate the
KB like you'd curate docs — because that's what it is.
And it's the most sensitive thing you'll attach. Your incidents, your
conventions, your internal decisions — that's exactly the data you don't want
leaving your network. The network.egress: []
driver above isn't a detail; it's
the reason this whole pattern runs locally. Your knowledge is your edge precisely
because it's private, so the tool that serves it should keep it that way.
Every team in 2026 rents the same frontier weights. That was always going to
commoditize — the models converge, the benchmarks compress, and the clever prompt
becomes a thing anyone can copy. What doesn't commoditize is what your agent knows
that no one else's does: your codebase's conventions, your runbooks, the incident
that taught you never to retry a charge without an idempotency key.
That knowledge is the one layer of the stack you actually own. Attaching it isn't
a nice-to-have on top of a good model — after the models converged, it's the only move left that makes your agent yours. An operator with a queryable
So run the diagnosis one more time, honestly: does your agent know your world, or
just the internet's average of it? If it's the average, you don't have an agent
problem. You have a knowledge problem — and it's the good kind, because the fix is
entirely in your hands.
If your setup already attaches knowledge a cleaner way, or I've mis-described how
your KB reaches your agents, tell me where — I'll fix the piece.
Ten pieces, one argument. Start anywhere; each one cross-links the rest.
| Piece | The one idea | |
|---|---|---|
| 1 | ||
Written by the maintainer of agentproto (Apache-2.0, source). Same contract as our /compare page — dated facts, named strengths, corrections by issue. Got something wrong? File an issue.
Building agentproto in the open — follow @theagentproto and @agentik_ai on X.