Your Coding Agent Knows the Internet's Average. Here's How to Make It Know Yours.

A developer building agentproto argues that coding agents trained on public internet data produce generic answers, and the only way to make an agent specific to a team's codebase is to attach a custom knowledge base. The post outlines three tiers of knowledge integration, from no knowledge base to a served, queryable system, and claims retrieval quality is the dominant factor in agent performance.

Disclosure up front: I build agentproto , whose knowledge and operator primitives are the back half of this piece. The problem in the first half stands on its own, and the walkthrough uses real, checkable commands. Corrections welcome — file an issue. Open Claude Code. Open Codex. Open a cheap local model in the same repo. Ask all three the same question about your codebase — your retry convention, your deploy runbook, why that one service is the way it is. You'll get three fluent, confident, near-identical answers. And all three will be wrong in the same way: they'll describe how a reasonable team does it, not how yours does. They're all running the same frontier weights, trained on the same public internet, so out of the box they all know exactly the same thing. The one idea, if you remember nothing else: The weights are everyone's. Your knowledge base is the only part of the stack that'syours— and the only thing that makes the agent yours instead of the internet's average. Here's the shift most people missed while arguing about benchmarks. The gap between the best model and the tenth-best is collapsing — the hub piece https://dev.to/agentiknet/you-can-parallelize-the-typing-you-cant-parallelize-the-trust-2fkd has the Stanford AI Index receipt the 1-to- 10 spread fell from 11.9% to 5.4% in a year . When the weights converge, the weights stop being your edge. What replaces them is system design. One widely-cited 2026 essay on the evolution of agents put it bluntly: agent capability has moved from model quality to system design — from writing a clever prompt to engineering what the model can The receipt that reframes the whole market.In a production-readiness comparison of agent frameworks, one finding held across every orchestration style tested:retrieval quality dominates knowledge-agent performanceYou can have the best planner-worker-judge regardless of orchestration. topology in the world; if the agent is retrieving from the internet's average instead of your world, it answers like a stranger. So the interesting question stopped being "which model?" and became "what does it know that no one else's does?" JetBrains' framing is useful here: some agents are action-first their value is what they do and some are data-first their value is what they can access . A coding agent for your team is data-first whether you designed it that way or not. The uncomfortable part: if you haven't attached your knowledge to your agent, you're shipping the stranger. Run the self-diagnosis before you build anything. There are three tiers, and most teams are stuck on the first two without noticing. Tier 0 — no knowledge base. The agent knows what the pre-training run knew: public GitHub, Stack Overflow, docs up to some cutoff. Ask it about your world and it pattern-matches to the nearest public example. It's confident and it's generic. Ceiling: it will never know a single thing that isn't already on the internet. Tier 1 — pasted into context. You dump the runbook, the conventions doc, a few past PRs into the prompt or a CLAUDE.md and hope. It works — until the context fills up. Anthropic's context-engineering post is direct about why this caps out: you have to treat context as a finite attention budget , because recall decays as the window grows "context rot" . Why Tier 1 has a hard ceiling.The same post prescribes the fix —keep, not pre-load everything. lightweight references and retrieve just-in-time Pasting your whole knowledge base into every prompt is the exact anti-pattern: it's expensive, it rots, and it doesn't scale past a handful of documents. A KB you paste is a KB you'll soon truncate. Tier 2 — a served, queryable knowledge base. Your knowledge lives in one place, gets indexed, and every agent queries it on demand — pulling only the three relevant entries for the question at hand, with their sources attached. This is just-in-time retrieval, done properly, shared across every agent you run. The jump from Tier 1 to Tier 2 is the whole game — and it's the "who equips the agents" question from the files-with-contracts piece, pointed straight at knowledge. Let's build it. Start with the packaged shape, because it shows the target. In agentproto, an operator the AIP-9 OPERATOR.md is a persona plus policies plus a set of bound tools — and one of those tools is a knowledge binding. operators/house-engineer/OPERATOR.md — persona + policies + a KB binding --- name: House Engineer id: house-engineer profile: role: Answer and act using OUR conventions and past incidents, never generic defaults. voice: Concrete, cites the runbook or incident it's drawing from. tools: - knowledge-query ← the bound knowledge tool - shell policies: - "policy:engineering/house-rules" metadata: corpus: knowledgeViews: - corpus: engineering filter: { tags: conventions, runbooks, incidents } --- Read what that knowledgeViews block does: it scopes the operator to your engineering corpus, filtered to the tags that matter. The persona decides how it answers; the policies decide what it's allowed to do; the knowledge view decides what it knows . Three separate concerns, one file. This is a real shape, not a mock.The research operators shipped in agentproto's own corpus — source-scout , research-analyst — carry exactly this: tools: knowledge-query, ... plus a metadata.corpus.knowledgeViews binding scoped by domain and quality score. The operator is thepackaging. Underneath it is a tool, and the tool is the part that ports to any agent. An operator is what "give the agent your knowledge" looks like when it's one declared file. Now let's build the tool that backs it — the part that reaches This is the exact move from the files-with-contracts piece https://dev.to/agentiknet/your-claude-skill-is-invisible-to-codex-heres-how-to-fix-it-3l74 , specialized to knowledge: split the contract what the tool promises from the driver the code that answers . Here's the contract — a pure defineTool , no execute body. js // tools/knowledge-search/TOOL.ts — the contract, and nothing else import { defineTool } from "@agentproto/tool" import { z } from "zod" export const knowledgeSearch = defineTool { id: "knowledge.search", description: "Answer from OUR knowledge base: conventions, runbooks, past incidents. " + "Returns refined entries, each with the source it came from.", inputSchema: z.object { query: z.string , tags: z.array z.string .default , limit: z.number .default 5 , } , outputSchema: z.object { entries: z.array z.object { kind: z.string , title: z.string , body: z.string , source: z.string , // provenance — where this knowledge came from } , } , mutates: , // read-only → no approval gate approval: "auto", } Now the driver — the implementation, bound to the contract by implementTool . It reads your corpus workspace off local disk and returns the matching entries. js // drivers/knowledge-corpus.ts — backed by your own corpus workspace import { defineDriver, implementTool } from "@agentproto/driver" import { knowledgeSearch } from "../tools/knowledge-search/TOOL.js" import { resolveKnowledge } from "@agentproto/corpus" import { NodeFsAdapter } from "@agentproto/corpus/fs" export default defineDriver { id: "knowledge.search.corpus", name: "Knowledge search over your corpus workspace", kind: "in-process", implementations: implementTool knowledgeSearch, async { input } = { // the same resolver behind corpus knowledge <ws --tags … --max … const hits = await resolveKnowledge { fs: new NodeFsAdapter { root: process.env.CORPUS WS } , query: { tags: input.tags, maxResults: input.limit }, } return { entries: hits.map h = { kind: h.kind, title: h.title, body: h.body, source: h.sources.join ", " , } , } } , , network: { egress: }, // reads local files → your knowledge never leaves the box } That driver is the real corpus resolver — the same resolveKnowledge call that sits behind corpus knowledge <ws --tags conventions,incidents --max 5 on the command line, now exposed as a tool. The network.egress: line is doing quiet but load-bearing work: the knowledge is queried from local files, so it never leaves your machine. Provenance is a feature, not decoration.Every entry carries its source — the runbook, the incident, the PR it was distilled from. That's what lets an external check — the supervision ladder's whole point — verify the answer later: the LLM-as-a-judge literature evaluates RAG on twosides, context relevance going in andfaithfulness coming out did the answer actually follow the retrieved source, or wander off it? . No provenance, no faithfulness check. One contract, one driver, and now every agent that mounts the daemon can answer from your world. Serve it. Start the daemon. It reads your tools/ and drivers/ and serves them over MCP, knowledge.search included. npm i -g @agentproto/cli agentproto serve serves the /mcp gateway to every agent Now spawn an agent with the tool, and ask it something it could only know from your docs — a convention set by a past incident: agentproto sessions start codex --cwd . \ --prompt "What's our retry policy for the payments queue? Use knowledge.search." → Per incident 2026-04 double-charge on gateway timeout , payment retries are capped at 3 with jittered backoff, and we NEVER retry a charge without an idempotency key. Source: runbooks/payments-queue.md, incidents/2026-04.md Now the same agent, same question, without the tool: agentproto sessions start codex --cwd . \ --prompt "What's our retry policy for the payments queue?" → A common approach is exponential backoff with 3–5 retries and a dead-letter queue after the final attempt… The second answer isn't wrong — it's the internet's average, and it's exactly what you'd get from any agent, anywhere. The first answer knows about the double-charge you shipped in April, because it read your incident file. That delta — a real past incident it could only learn from your knowledge base — is the entire difference between an agent and Cross-vendor, by construction.That was Codex. Point Claude Code or a cheap local model via Hermes at the same daemon and they call the identical knowledge.search — because it's served over MCP, not baked into one vendor's config. Author the KB tool once; every agent in your fleet inherits your world. Here's the honest framing, because overclaiming is how you lose a reader. In Guilde https://guilde.work — our AI-company product — attaching a knowledge base to an agent is one packaged step : an operator ships with an attached corpus, and you apply it to a scope in a single command. Persona, policies, knowledge, done. What you just built by hand in agentproto is that same primitive, one floor down: an operator Step 1 plus a served knowledge tool Step 2 . Guilde adds the one-command packaging; agentproto is the from-primitives path you can run today , What agentproto doesThere's no singlenotdo yet, stated plainly. apply-knowledge-pack verb in the open CLI — you wire the tool, the driver, and the operator yourself, as three files. That's more assembly than the packaged version. It's also fully inspectable and fully yours, which for a knowledge base — the most sensitive thing you'll attach to an agent — is often the trade you want. The primitive is identical either way: an operator with a queryable knowledge tool. One product packages it; the other hands you the parts. A knowledge base is not magic, and three caveats keep it honest. A wrong KB is worse than no KB. If your knowledge base contains a stale convention, the agent will state it with the same confidence as a correct one — now with a source attached, which makes it more persuasive and more dangerous. This is why the faithfulness check above matters: retrieval quality is the whole ballgame, and a KB is an artifact you maintain, not a bucket you fill once. Don't let the agent write its own knowledge base.ETH Zurich research Gloaguen et al. found that LLM-generated AGENTS.md files providednoA knowledge base curated from real benefit and could reduce success rates. incidents, runbooks, and decisions is yours; one the agent hallucinated about itself is just the internet's average with your repo's name on it. Curate the KB like you'd curate docs — because that's what it is. And it's the most sensitive thing you'll attach. Your incidents, your conventions, your internal decisions — that's exactly the data you don't want leaving your network. The network.egress: driver above isn't a detail; it's the reason this whole pattern runs locally. Your knowledge is your edge precisely because it's private, so the tool that serves it should keep it that way. Every team in 2026 rents the same frontier weights. That was always going to commoditize — the models converge, the benchmarks compress, and the clever prompt becomes a thing anyone can copy. What doesn't commoditize is what your agent knows that no one else's does: your codebase's conventions, your runbooks, the incident that taught you never to retry a charge without an idempotency key. That knowledge is the one layer of the stack you actually own. Attaching it isn't a nice-to-have on top of a good model — after the models converged, it's the only move left that makes your agent yours. An operator with a queryable So run the diagnosis one more time, honestly: does your agent know your world, or just the internet's average of it? If it's the average, you don't have an agent problem. You have a knowledge problem — and it's the good kind, because the fix is entirely in your hands. If your setup already attaches knowledge a cleaner way, or I've mis-described how your KB reaches your agents, tell me where — I'll fix the piece. Ten pieces, one argument. Start anywhere; each one cross-links the rest. | Piece | The one idea | | |---|---|---| | 1 | | Written by the maintainer of agentproto Apache-2.0, source . Same contract as our /compare page — dated facts, named strengths, corrections by issue. Got something wrong? File an issue. Building agentproto in the open — follow @theagentproto and @agentik ai on X.