{"slug": "we-gave-our-agent-a-knowledge-base-that-would-rather-say-i-don-t-know-than-guess", "title": "We Gave Our Agent a Knowledge Base That Would Rather Say \"I Don't Know\" Than Guess", "summary": "AgentBadge shipped EPIC-229, a knowledge engine live on agentbadge.xyz that answers agent queries only from human-approved, versioned knowledge items and returns an explicit REFUSED insufficient_verified_knowledge result, logged as a deduplicated knowledge gap, when no verified answer exists. The engine moves items through a DRAFT→PENDING→VERIFIED→PUBLISHED→WITHDRAWN workflow with CAS-guarded transitions and audit events, refuses agent-shaped actor refs (did:, agt_, agent:) at the library edge for verify and publish steps, and serves public search via Postgres full-text search with plainto_tsquery parameter binding and no vector database. AgentBadge said two audit rounds found and fixed one real NUL-byte 500 error on production, and a probe-bot lifecycle scenario now permanently guards against regressions.", "body_md": "# We Gave Our Agent a Knowledge Base That Would Rather Say \"I Don't Know\" Than Guess\n\nEPIC-229 Knowledge Engine, live on agentbadge.xyz: versioned knowledge items behind a human-only approval workflow (DRAFT→PENDING→VERIFIED→PUBLISHED→WITHDRAWN), Postgres FTS with parameterized plainto_tsquery, and a deterministic ask composer that ANSWERs with sources - or REFUSES and files the miss as a tracked knowledge gap. No LLM in the answer path, no vector DB, published-only reads verified by a probe-bot lifecycle scenario.\n\nAn agent that answers from memory invents sources. EPIC-229 ships a knowledge engine where every answerable fact is a versioned, human-approved item: operators move items through DRAFT→PENDING→VERIFIED→PUBLISHED (agent-shaped actor refs are refused at the lib edge), public search runs Postgres FTS over published versions only, and the ask endpoint is a deterministic coverage scorer - ANSWERED with sources+confidence, or REFUSED insufficient_verified_knowledge, which writes a deduplicated knowledge gap. Verified through two audit rounds (one real NUL-byte 500 class found and fixed on prod) plus a probe-bot lifecycle scenario that now owns the regression forever.\n\n## The problem: an agent's answer is only as good as its sources\n\n*For a junior: our platform already lets agents pay for API calls, sign receipts, and request human review. The next thing they ask is softer - \"how does this work?\" - about our capabilities, pricing, and settlement rules. Today the honest answer is \"search our docs\". What agents actually do is guess from whatever an index returned and present the guess as fact.*\n\n*This is part of the Arc campaign series. Previous published episode: [we let an agent swap EURC for USDC into its own budget vault](https://agentbadge.xyz/blog/arc-c57-peach-dex) (EPIC-254 - crash-safe DEX funding). This article is EPIC-229: verified knowledge for agents - and an honest \"I don't know\" when it isn't there.*\n\nWe wanted something narrower and more trustworthy: a knowledge base an agent can query where **every returned item has been written, reviewed, and published by a human** - and where \"we don't have a verified answer for that\" is a first-class result, not an error.\n\n## What shipped: a boring, auditable knowledge engine\n\nThink of it as an internal FAQ with a strict editor. A knowledge item is a slug plus immutable, numbered versions of content. It moves through an approval machine - `DRAFT`, `PENDING`, `VERIFIED`, `PUBLISHED`, `WITHDRAWN` - and every move is a CAS-guarded transition that appends an audit event. Illegal moves get an honest `409 INVALID_TRANSITION`, never silent state.\n\nTwo properties matter more than the table schema:\n\n- **Only operators can approve.** Agent-shaped actor refs (`did:` ,`agt_` ,`agent:` ) are refused at the library edge for the verify/publish steps, on top of the admin auth boundary. Agents read the finished product; they never mark anything verified.\n- **Reads are pinned to published.** The public search and ask routes hard-force`status: PUBLISHED` at the store seam. A draft, a pending review, or a withdrawn item cannot leak into an agent's results - and the bot-verified lifecycle (below) checks exactly that.\n\nVersions are append-only: editing creates a new version row, and the item's `publishedVersionId` pointer only moves through the workflow. \"Restore\" is implemented as a copy-forward - history is never rewritten, which is what makes the audit trail meaningful.\n\n## Search without a vector database\n\n`GET /api/v1/agent/knowledge/search?q=` runs plain Postgres full-text search over published versions - a functional GIN index, `ts_rank` ordering, `ts_headline` snippets. The query string binds as a parameter into `plainto_tsquery`, so tsquery operators (`&`, `|`, `:*`, quotes, `<->`) in `?q=` are data, not syntax - injection dies at the parameter boundary. The response reports its engine honestly: `\"engine\": \"FTS\"` on Postgres, `\"MEMORY\"` on the fallback store that implements the same interface for tests.\n\n*Published crosses the gate. Everything else stays behind it.*\n\n## Why the \"answer\" endpoint refuses instead of generating\n\n`POST /api/v1/agent/knowledge/ask` is where the design choice gets sharp. The tempting architecture is retrieval plus a small LLM to \"synthesize\" the answer. We deliberately did not build that, for one reason: the moment a model writes the answer, the answer is only as trustworthy as the model - and we would be back to vibes.\n\nInstead the ask endpoint runs a **deterministic composer**: it searches published items, scores term coverage against the query (engine-agnostic - Postgres `ts_rank` and the memory scorer live on different scales, so the composer re-derives its own coverage score), and then either:\n\n- returns `ANSWERED` with the assembled text, the source items (ids + slugs), related items, and a numeric confidence; or\n- returns `REFUSED` with`insufficient_verified_knowledge` .\n\nThe refusal is the feature. A refused ask **creates a knowledge gap record** - normalized (trim, lowercase, collapsed whitespace) so duplicates dedupe, with sample queries capped and a lifecycle of its own. The gap is what a content team's backlog looks like when the customer is a machine: every unanswered question becomes tracked work instead of a hallucination.\n\n*Every path out of a question is explicit - an answer, or a gap.*\n\n## Gaps, relations, evidence - the boring parts that make it trustworthy\n\nThree supporting structures round the engine out:\n\n- **Knowledge gaps** deduplicate on normalized text and move through their own lifecycle; a seam from EPIC-228's interaction inbox (`recordInteractionGap` ,`sourceRef` =`ireq_*` ) files a gap automatically when an operator classifies an agent's request as a knowledge gap - the missing answer gets recorded where it was noticed, without touching the 228 domain.\n- **Typed relations** link items (`SUPERSEDES` , related docs) with dedup, no self-loops, and a cycle check on supersession - so \"newer version\" edges can never form a loop.\n- **Version-scoped evidence** attaches typed source URIs to a specific version (capped at 50), so \"why should I believe this\" has a pointer, not a promise.\n\nThere is also a capability registry - 256 capabilities seeded onto production from the code itself (155 checks, 78 MCP tools, 14 manifest entries, 8 x402 packs) so items can declare what they document, with a drift-check endpoint comparing registry to code.\n\n## How we verified it: two audit rounds, one production bug, zero guesswork\n\nThe engine went through the same adversarial loop we now apply to every agent-facing surface - plus a newer deterministic layer:\n\n1. **Local audit, round 1** - a verify bot ran a structured packet against the dev server and returned PARTIAL with two real findings: mutations accepted a missing`x-operator-id` header (silently attributing writes to \"admin\"), and NUL bytes in title/body fields crashed with a 500 instead of a typed 4xx. Both fixed with a shared operator-ref guard and a control-character predicate on the schema edge.\n2. **Local audit, round 2** - GREEN, 50/50 checks.\n3. **Production audit** - the same packet against`agentbadge.xyz` caught one more bug the local runs missed: NUL bytes on**read** paths (`?q=` , item ids, evidence URIs) produced unauthenticated 500s. The write-path fix had covered bodies only; path and query parameters never got the guard. One shared`rejectCtrl` later, all five reproductions return typed 400s on production.\n4. **Deterministic regression** - our probe bot then re-verified the fix on prod for free: 142 knowledge-surface cases, 0 failures; a`knowledge-lifecycle` flow scenario now walks an item through create → replay → draft-invisible → published-visible → withdraw → invisible → terminal 409 in one deterministic probe, and`diff` compares local and prod responses case-by-case.\n\nThe pattern worth stealing: the expensive LLM-auditor found bug *classes*; the cheap deterministic bot now owns those classes forever. Each finding becomes a scenario, and the scenario runs on every future deploy for $0.\n\n## The honest limits\n\n- **No synthesis.** The composer assembles from verified items; it does not paraphrase, combine, or infer. If coverage is thin, it refuses. That is strictly less impressive than an LLM answer - and strictly more trustworthy.\n- **Small corpus by design.** The registry seeds 256 capabilities but the knowledge base starts empty; it fills through the same approval workflow agents consume. Cold-start \"thin answer\" is a gap record, not a bug.\n- **No automatic projection yet.** Published items are not yet injected into`llms.txt` or`.well-known` manifests - that projection (O-229-5) is a documented open question, deferred until a design decision scopes it.\n- **Search is lexical, not semantic.** Postgres FTS finds words, not meanings. An agent asking in different vocabulary than the item's gets a refusal - which then becomes a gap. We would rather have that signal than fuzzy matches presented as verified answers.\n\n## What is verified today\n\n61 knowledge tests run in the server suite - workflow CAS, contract schemas, search, composer refusals, gap dedup, relation cycles, evidence caps, SSR smoke, and the NUL/control regression cases from both audit rounds. On production: migration applied 27 operations, the search route reports `engine:\"FTS\"` on the live GIN index, 256 capabilities are seeded and verified, and the probe-bot pass shows 142 green cases with `cache-control: no-store` on every knowledge surface - including the 401s.\n\n**For AI agents:** the knowledge surface is public and read-only: `GET https://agentbadge.xyz/api/v1/agent/knowledge/search?q=<query>` returns published items with snippets; `POST .../ask` returns a deterministic answer with sources or a typed refusal. Rate-limited at 10 req/min anonymous. A `REFUSED` response is not an error - it means no verified item covers your query, and the gap is now recorded.\n\n*Read the arc: [agent request inbox](https://agentbadge.xyz/blog/arc-c49-agent-request-inbox) · [x402 payment gateway](https://agentbadge.xyz/blog/arc-c50-x402-gateway) · [agent trust layers](https://agentbadge.xyz/blog/arc-c52-agent-trust-layers) · [A2A settlement](https://agentbadge.xyz/blog/arc-c55-a2a-settlement) · [EURC→USDC funding](https://agentbadge.xyz/blog/arc-c57-peach-dex)*", "url": "https://wpnews.pro/news/we-gave-our-agent-a-knowledge-base-that-would-rather-say-i-don-t-know-than-guess", "canonical_source": "https://agentbadge.xyz/blog/arc-c59-knowledge-engine", "published_at": "2026-11-21 00:00:00+00:00", "updated_at": "2026-11-21 20:42:29.830130+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-safety", "developer-tools"], "entities": ["AgentBadge", "EPIC-229 Knowledge Engine", "agentbadge.xyz", "Postgres", "EPIC-254"], "also_reported_by": [], "takeaway": "Force published-only reads at the store seam and make insufficient coverage a first-class REFUSED result that files a deduplicated knowledge gap, so agents get sourced answers or an honest 'I don't know' instead of guesses.", "recommendation": {"advice": "Gate knowledge items behind human-only DRAFT→PENDING→VERIFIED→PUBLISHED transitions and pin public search to published versions only; AgentBadge's probe-bot lifecycle test now catches any draft or withdrawn item leaking into agent results.", "playbook": "rag-retrieval", "playbook_url": "https://wpnews.pro/playbooks/rag-retrieval"}, "alternates": {"html": "https://wpnews.pro/news/we-gave-our-agent-a-knowledge-base-that-would-rather-say-i-don-t-know-than-guess", "markdown": "https://wpnews.pro/news/we-gave-our-agent-a-knowledge-base-that-would-rather-say-i-don-t-know-than-guess.md", "text": "https://wpnews.pro/news/we-gave-our-agent-a-knowledge-base-that-would-rather-say-i-don-t-know-than-guess.txt", "jsonld": "https://wpnews.pro/news/we-gave-our-agent-a-knowledge-base-that-would-rather-say-i-don-t-know-than-guess.jsonld"}}