cd /news/ai-agents/we-gave-our-agent-a-knowledge-base-t… · home › topics › ai-agents › article
[ARTICLE · art-173911] src=agentbadge.xyz ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

We Gave Our Agent a Knowledge Base That Would Rather Say "I Don't Know" Than Guess

AgentBadge shipped EPIC-229, a knowledge engine live on agentbadge.xyz that answers agent queries only from human-approved, versioned knowledge items and returns an explicit REFUSED insufficient_verified_knowledge result, logged as a deduplicated knowledge gap, when no verified answer exists. The engine moves items through a DRAFT→PENDING→VERIFIED→PUBLISHED→WITHDRAWN workflow with CAS-guarded transitions and audit events, refuses agent-shaped actor refs (did:, agt_, agent:) at the library edge for verify and publish steps, and serves public search via Postgres full-text search with plainto_tsquery parameter binding and no vector database. AgentBadge said two audit rounds found and fixed one real NUL-byte 500 error on production, and a probe-bot lifecycle scenario now permanently guards against regressions.

recommendation

Gate knowledge items behind human-only DRAFT→PENDING→VERIFIED→PUBLISHED transitions and pin public search to published versions only; AgentBadge's probe-bot lifecycle test now catches any draft or withdrawn item leaking into agent results.

more in playbook: Retrieval and RAG
read8 min views2 publishedNov 21, 2026
We Gave Our Agent a Knowledge Base That Would Rather Say "I Don't Know" Than Guess
Image: Agentbadge (auto-discovered)

EPIC-229 Knowledge Engine, live on agentbadge.xyz: versioned knowledge items behind a human-only approval workflow (DRAFT→PENDING→VERIFIED→PUBLISHED→WITHDRAWN), Postgres FTS with parameterized plainto_tsquery, and a deterministic ask composer that ANSWERs with sources - or REFUSES and files the miss as a tracked knowledge gap. No LLM in the answer path, no vector DB, published-only reads verified by a probe-bot lifecycle scenario.

An agent that answers from memory invents sources. EPIC-229 ships a knowledge engine where every answerable fact is a versioned, human-approved item: operators move items through DRAFT→PENDING→VERIFIED→PUBLISHED (agent-shaped actor refs are refused at the lib edge), public search runs Postgres FTS over published versions only, and the ask endpoint is a deterministic coverage scorer - ANSWERED with sources+confidence, or REFUSED insufficient_verified_knowledge, which writes a deduplicated knowledge gap. Verified through two audit rounds (one real NUL-byte 500 class found and fixed on prod) plus a probe-bot lifecycle scenario that now owns the regression forever.

The problem: an agent's answer is only as good as its sources #

For a junior: our platform already lets agents pay for API calls, sign receipts, and request human review. The next thing they ask is softer - "how does this work?" - about our capabilities, pricing, and settlement rules. Today the honest answer is "search our docs". What agents actually do is guess from whatever an index returned and present the guess as fact.

This is part of the Arc campaign series. Previous published episode: we let an agent swap EURC for USDC into its own budget vault (EPIC-254 - crash-safe DEX funding). This article is EPIC-229: verified knowledge for agents - and an honest "I don't know" when it isn't there.

We wanted something narrower and more trustworthy: a knowledge base an agent can query where every returned item has been written, reviewed, and published by a human - and where "we don't have a verified answer for that" is a first-class result, not an error.

What shipped: a boring, auditable knowledge engine #

Think of it as an internal FAQ with a strict editor. A knowledge item is a slug plus immutable, numbered versions of content. It moves through an approval machine - DRAFT, PENDING, VERIFIED, PUBLISHED, WITHDRAWN - and every move is a CAS-guarded transition that appends an audit event. Illegal moves get an honest 409 INVALID_TRANSITION, never silent state.

Two properties matter more than the table schema:

  • Only operators can approve. Agent-shaped actor refs (did: ,agt_ ,agent: ) are refused at the library edge for the verify/publish steps, on top of the admin auth boundary. Agents read the finished product; they never mark anything verified.
  • Reads are pinned to published. The public search and ask routes hard-forcestatus: PUBLISHED at the store seam. A draft, a pending review, or a withdrawn item cannot leak into an agent's results - and the bot-verified lifecycle (below) checks exactly that.

Versions are append-only: editing creates a new version row, and the item's publishedVersionId pointer only moves through the workflow. "Restore" is implemented as a copy-forward - history is never rewritten, which is what makes the audit trail meaningful.

Search without a vector database #

GET /api/v1/agent/knowledge/search?q= runs plain Postgres full-text search over published versions - a functional GIN index, ts_rank ordering, ts_headline snippets. The query string binds as a parameter into plainto_tsquery, so tsquery operators (&, |, :*, quotes, <->) in ?q= are data, not syntax - injection dies at the parameter boundary. The response reports its engine honestly: "engine": "FTS" on Postgres, "MEMORY" on the fallback store that implements the same interface for tests.

Published crosses the gate. Everything else stays behind it.

Why the "answer" endpoint refuses instead of generating #

POST /api/v1/agent/knowledge/ask is where the design choice gets sharp. The tempting architecture is retrieval plus a small LLM to "synthesize" the answer. We deliberately did not build that, for one reason: the moment a model writes the answer, the answer is only as trustworthy as the model - and we would be back to vibes.

Instead the ask endpoint runs a deterministic composer: it searches published items, scores term coverage against the query (engine-agnostic - Postgres ts_rank and the memory scorer live on different scales, so the composer re-derives its own coverage score), and then either:

  • returns ANSWERED with the assembled text, the source items (ids + slugs), related items, and a numeric confidence; or
  • returns REFUSED withinsufficient_verified_knowledge .

The refusal is the feature. A refused ask creates a knowledge gap record - normalized (trim, lowercase, collapsed whitespace) so duplicates dedupe, with sample queries capped and a lifecycle of its own. The gap is what a content team's backlog looks like when the customer is a machine: every unanswered question becomes tracked work instead of a hallucination.

Every path out of a question is explicit - an answer, or a gap.

Gaps, relations, evidence - the boring parts that make it trustworthy #

Three supporting structures round the engine out:

  • Knowledge gaps deduplicate on normalized text and move through their own lifecycle; a seam from EPIC-228's interaction inbox (recordInteractionGap ,sourceRef =ireq_* ) files a gap automatically when an operator classifies an agent's request as a knowledge gap - the missing answer gets recorded where it was noticed, without touching the 228 domain.
  • Typed relations link items (SUPERSEDES , related docs) with dedup, no self-loops, and a cycle check on supersession - so "newer version" edges can never form a loop.
  • Version-scoped evidence attaches typed source URIs to a specific version (capped at 50), so "why should I believe this" has a pointer, not a promise.

There is also a capability registry - 256 capabilities seeded onto production from the code itself (155 checks, 78 MCP tools, 14 manifest entries, 8 x402 packs) so items can declare what they document, with a drift-check endpoint comparing registry to code.

How we verified it: two audit rounds, one production bug, zero guesswork #

The engine went through the same adversarial loop we now apply to every agent-facing surface - plus a newer deterministic layer:

  1. Local audit, round 1 - a verify bot ran a structured packet against the dev server and returned PARTIAL with two real findings: mutations accepted a missingx-operator-id header (silently attributing writes to "admin"), and NUL bytes in title/body fields crashed with a 500 instead of a typed 4xx. Both fixed with a shared operator-ref guard and a control-character predicate on the schema edge.
  2. Local audit, round 2 - GREEN, 50/50 checks.
  3. Production audit - the same packet againstagentbadge.xyz caught one more bug the local runs missed: NUL bytes onread paths (?q= , item ids, evidence URIs) produced unauthenticated 500s. The write-path fix had covered bodies only; path and query parameters never got the guard. One sharedrejectCtrl later, all five reproductions return typed 400s on production.
  4. Deterministic regression - our probe bot then re-verified the fix on prod for free: 142 knowledge-surface cases, 0 failures; aknowledge-lifecycle flow scenario now walks an item through create → replay → draft-invisible → published-visible → withdraw → invisible → terminal 409 in one deterministic probe, anddiff compares local and prod responses case-by-case.

The pattern worth stealing: the expensive LLM-auditor found bug classes; the cheap deterministic bot now owns those classes forever. Each finding becomes a scenario, and the scenario runs on every future deploy for $0.

The honest limits #

  • No synthesis. The composer assembles from verified items; it does not paraphrase, combine, or infer. If coverage is thin, it refuses. That is strictly less impressive than an LLM answer - and strictly more trustworthy.
  • Small corpus by design. The registry seeds 256 capabilities but the knowledge base starts empty; it fills through the same approval workflow agents consume. Cold-start "thin answer" is a gap record, not a bug.
  • No automatic projection yet. Published items are not yet injected intollms.txt or.well-known manifests - that projection (O-229-5) is a documented open question, deferred until a design decision scopes it.
  • Search is lexical, not semantic. Postgres FTS finds words, not meanings. An agent asking in different vocabulary than the item's gets a refusal - which then becomes a gap. We would rather have that signal than fuzzy matches presented as verified answers.

What is verified today #

61 knowledge tests run in the server suite - workflow CAS, contract schemas, search, composer refusals, gap dedup, relation cycles, evidence caps, SSR smoke, and the NUL/control regression cases from both audit rounds. On production: migration applied 27 operations, the search route reports engine:"FTS" on the live GIN index, 256 capabilities are seeded and verified, and the probe-bot pass shows 142 green cases with cache-control: no-store on every knowledge surface - including the 401s.

For AI agents: the knowledge surface is public and read-only: GET https://agentbadge.xyz/api/v1/agent/knowledge/search?q=<query> returns published items with snippets; POST .../ask returns a deterministic answer with sources or a typed refusal. Rate-limited at 10 req/min anonymous. A REFUSED response is not an error - it means no verified item covers your query, and the gap is now recorded.

Read the arc: agent request inbox · x402 payment gateway · agent trust layers · A2A settlement · EURC→USDC funding

── more in #ai-agents 4 stories · sorted by recency
── more on @agentbadge 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/we-gave-our-agent-a-…] indexed:0 read:8min 2026-11-21 · —