Making a knowledge base queryable by any agent over MCP

A developer built AgentPostmortem, a public registry of AI-agent failure incidents, and Casebook MCP, a remote Model Context Protocol server that exposes the registry as tools any agent can call. The server runs as a single Cloudflare Worker with a stateless streamable HTTP transport, offering tools like search_cases, get_case, similar_failures, and list_tags. The developer implemented the MCP transport directly against the 2025-03-26 spec to keep the server simple and efficient.

Every team debugging an AI-agent incident is quietly rediscovering a failure mode that someone else already documented. A support bot approves hundreds of fake refunds because a ticket told it to. A retry loop runs away and spams customers. A stale cache turns into a confident hallucination. These are not novel bugs. They are recurring shapes, and the knowledge about how they played out and what fixed them is usually locked in a blog post or a postmortem doc nobody thinks to search at 2am. I built AgentPostmortem https://agentpostmortem.com as a public registry of these documented failures. But a registry a human has to remember to visit is a registry that goes unused during the exact moment it matters: while an agent is actively investigating. So I built Casebook MCP: a remote MCP https://modelcontextprotocol.io server that turns the registry into tools any agent can call. Now Claude Code, Cursor, or an agent built on the Claude Agent SDK can ask, mid-investigation, "has anything like this happened before?" Casebook exposes four tools over the Model Context Protocol: search cases query, tag? for ranked full-text search over the case files, with an optional tag filter. get case id for the full detail of one case: outcome narrative, verified facts, unknowns, and lessons. similar failures description which takes a free-text incident description and returns the closest documented failures by keyword overlap. list tags for every failure-mode tag with its description.The tool that earns its keep is similar failures . You paste in what is actually happening "our support bot was tricked by text in a ticket into approving refunds" and get back real precedents, ranked, with the shared keywords, the outcome, and the lessons attached. That is the difference between a lookup table and something an agent can reason against. The whole thing runs as a single Cloudflare Worker. I chose to implement the MCP transport directly against the 2025-03-26 streamable HTTP spec in stateless mode rather than pull in a framework. There is one POST /mcp endpoint that handles initialize , tools/list , and tools/call . No sessions, no Durable Objects, no auth, because the data is public and read-only. That decision keeps the server a plain request/response function, which is exactly what a Worker is good at. The dispatch layer is just a switch over JSON-RPC methods. Here is the shape of it: async function dispatch req: JsonRpcRequest : Promise<unknown | null { switch req.method { case "initialize": return { protocolVersion: "2025-03-26", capabilities: { tools: {} }, serverInfo: { name: "casebook-mcp", version: "0.1.0" }, instructions: "Use similar failures when investigating an incident, " + "search cases for topical research, get case for full detail.", }; case "tools/list": return { tools: TOOLS }; case "tools/call": { const name = String req.params?.name ?? "" ; const args = req.params?.arguments ?? {} as Record<string, unknown ; return callTool name, args ; } default: if req.method.startsWith "notifications/" return null; throw { code: -32601, message: Method not found: ${req.method} }; } } Notifications JSON-RPC requests with no id get swallowed and produce no response body, which is what the spec wants. The request handler also accepts batched arrays, so a client can pipeline initialize and tools/list in one POST. The ranking logic lives in its own pure module with no I/O, which makes it trivial to unit test. Scoring is deliberately simple and legible: a query token hitting a case title weighs 3, a tag hit weighs 2, and a body hit weighs 1, with ties broken by case number for stable ordering. export function scoreCase c: CaseFile, tokens: string : number { if tokens.length === 0 return 0; const title = new Set tokenize c.title ; const tags = new Set c.tags.flatMap t = tokenize t ; const body = new Set tokenize caseText c ; let score = 0; for const tok of tokens { if title.has tok score += 3; else if tags.has tok score += 2; else if body.has tok score += 1; } return score; } similar failures uses a related but even simpler measure: tokenize the description, tokenize each case, and rank by the size of the keyword intersection. No embeddings, no vector store. Data comes from the live public endpoints on agentpostmortem.com /api/export for the corpus, /api/search for rich detail, /api/tags behind a 5 minute in-memory cache. When the network is unavailable, it falls back to a bundled dataset of representative case files that ships in the repo and doubles as the deterministic fixture for tests. So the server keeps answering even offline, just from a smaller snapshot. There is also a companion investigator agent agent/investigate.ts built on the Claude Agent SDK query API. Given an incident description it connects to the MCP server, finds similar failures, pulls the top cases with get case , and writes a postmortem-draft.md grounded in the documented lessons. It runs on your local Claude Code subscription auth, so no API key appears in the code, and it has a --dry-run mode that stubs the model but still exercises the MCP server end to end, which is what CI uses. The similarity matching is pure keyword overlap, not semantic search. If your incident description and a documented case describe the same failure in different vocabulary, for example one says "loop" and the other says "recursion", the overlap can miss it. That is a conscious tradeoff: keyword ranking is transparent, dependency-free, and fully unit-testable, and it keeps the Worker cold-start cheap with no vector database to stand up. For a corpus of curated, tagged case files it works well in practice, but I would not claim it generalizes to fuzzy paraphrase the way an embedding index would. Adding an optional embedding pass is the obvious next step if recall becomes the bottleneck. The other honest note: the rate limit is a simple in-memory per-IP counter 60 requests per minute scoped to a single Worker isolate. It resets when the isolate recycles. That is intentionally lightweight for a public read-only endpoint, not a hardened quota system. Add it to Claude Code with a single command once it is running: claude mcp add --transport http casebook http://localhost:8787/mcp Then your agent can consult a growing body of real agent failures instead of rediscovering each one from scratch. Code, the four tools, the investigator agent, and the bundled dataset are all here: github.com/AgentPostmortem/casebook-mcp https://github.com/AgentPostmortem/casebook-mcp .