{"slug": "rfc-ai-tooling-40x-fewer-tokens-while-reading-the-rfc-corpus", "title": "RFC AI Tooling: 40x fewer tokens while reading the RFC corpus", "summary": "A new open-source project, rfc-ai-tooling, gives AI agents access to the roughly 9,800-document IETF RFC corpus through a skill and an MCP server that share a stdlib-only Python core, cutting token use by a median of 41x by reading single sections instead of whole documents. Measured from rfc.py 0.5.0 output, reading RFC 9110 §15.5.5 on 404 Not Found drops from 125.7k tokens for the whole RFC to 177 for the section, and RFC 9000 §19.3 on ACK Frames from 100.9k to 1,742. Every result carries the RFC index's obsolescence field so agents follow replacements before quoting, addressing cases like RFC 2616, obsolete since 2014, which the project says most models still reach for on HTTP; it replaces the unmaintained mcp-server-ietf.", "body_md": "Two ways to give an AI agent the IETF RFC corpus: a skill and an MCP server. Both run the same stdlib-only Python core, so they find, read and flag documents identically. No RFC text ships with either; documents come from the RFC Editor on demand.\n\n- **Obsolescence on every result.** A superseded RFC says so in a banner the model can't miss:`!! OBSOLETED BY: RFC 7230, 7231, ...`\n- **Section reads.** List an RFC's headings with line numbers, then read only the section that answers the question.\n- **Search.** By title out of the box, or across the full text of all ~9,800 RFCs once you sync a local mirror.\n- **Clean text.** Page headers, footers and form feeds are stripped.\n- **A guard on oversized reads.** A read over 1500 lines that isn't a named section is refused, with its size and how to narrow it.\n\n## The numbers behind the chart\n\n| RFC and section | Whole RFC | One section | \n|---|---|---|\n| RFC 9110 HTTP Semantics, §15.5.5 404 Not Found | 125.7k | 177 | \n| RFC 9000 QUIC, §19.3 ACK Frames | 100.9k | 1,742 | \n| RFC 8446 TLS 1.3, §4.1.2 Client Hello | 78.1k | 1,865 | \n| RFC 5321 SMTP, §4.2 SMTP Replies | 52.8k | 4,003 | \n| RFC 6749 OAuth 2.0, §5.2 Error Response | 38.0k | 935 | \n| RFC 9111 HTTP Caching, §4.2 Freshness | 21.1k | 2,531 | \n\nThe median saving is 41x, the figure behind \"40x fewer tokens\".\nTokens are approximated as bytes divided by 4, measured from `rfc.py` 0.5.0 output.\n[`assets/charts/context_sizes.json`](https://github.com/shbernal/rfc-ai-tooling/blob/main/assets/charts/context_sizes.json) has the raw measurements.\n\nAgents cite dead specifications with confidence. RFC 2616 has been obsolete since 2014, and it's still the first thing most models reach for on HTTP. Every result here carries the RFC index's obsolescence field, so the agent follows the replacement before it quotes anything.\n\nThis project replaces [`mcp-server-ietf`](https://github.com/tizee/mcp-server-ietf), which is unmaintained.\n\n|  | rfc-ai-tooling | mcp-server-ietf | Model fetches rfc-editor.org | \n|---|---|---|---|\n| Flags obsoleted RFCs | On every result | No | Only if it reads the header | \n| RFCs below 1000, like IP and TCP | Yes | No, its index parser expected zero-padded numbers | Yes | \n| Full titles | Yes | Cut at the first period, so RFC 2616 is \"HTTP/1\" | Yes | \n| Newly published RFCs | Index refreshes | Index downloaded once | Yes | \n| Read one section | By number or heading | Line ranges only | Whole document | \n| Full-text search | Ranked by section, with a downloaded index | Titles only | No | \n| Install | One config line, or none for the skill | Clone and `pip install` | Nothing | \n\nUse the one that matches where your agent's tools run.\n\n| Your client | Use | Why | \n|---|---|---|\n| Claude Code, Codex CLI, OpenClaw, Cursor, Zed, or anything that runs commands on your machine | The skill | The agent gets the whole shell, including `rg` over the corpus, instead of three fixed tools | \n| Claude Desktop | The MCP server over stdio | Desktop has no shell, but it spawns MCP servers on your machine | \n| claude.ai on the web, Cowork, other cloud sessions | The MCP server over HTTP, hosted by you | A cloud session can't reach a process on your machine | \n\nThere's no public instance of the server, and none is planned.\n\nInstall from this repo with [`skills`](https://github.com/vercel-labs/skills):\n\n```\nnpx skills add shbernal/rfc-ai-tooling\n```\n\nThe skill is a single stdlib-only Python script, with nothing to `pip install`.\n\nAdd this to your client's MCP configuration.\n`uvx` fetches [`mcp-server-rfc`](https://pypi.org/project/mcp-server-rfc/) from PyPI on first run.\n\n```\n{\n  \"mcpServers\": {\n    \"rfc\": {\n      \"command\": \"uvx\",\n      \"args\": [\"mcp-server-rfc\"]\n    }\n  }\n}\n```\n\nI've tested it end to end in Claude Desktop on Linux.\nmacOS and Windows should work, but I haven't checked them.\n[`docs/desktop-verification.md`](https://github.com/shbernal/rfc-ai-tooling/blob/main/docs/desktop-verification.md) shows how to tell whether Desktop really called the tools.\n\n## Self-host over HTTP for cloud sessions\n\nThe same binary serves HTTP:\n\n```\nmcp-server-rfc --transport http --host 0.0.0.0 --port 8080\n```\n\n[`mcp/Dockerfile`](https://github.com/shbernal/rfc-ai-tooling/blob/main/mcp/Dockerfile) and [` mcp/fly.toml`](https://github.com/shbernal/rfc-ai-tooling/blob/main/mcp/fly.toml) deploy it as a container:\n\n```\ndocker build -t mcp-server-rfc mcp/ && docker run --rm -p 8080:8080 mcp-server-rfc\n```\n\nThen add `https://<your-app>/mcp` as a custom connector in claude.ai, which needs a Pro, Max, Team or Enterprise plan.\n\n- **There's no authentication.** A public instance is an open proxy in front of a volunteer-run mirror, so put it behind auth or keep the URL private.\n- **Full-text search needs the section index on the server.** Run`sync --index` there, or it returns an error. Everything else needs only the 2 MB RFC index.\n\n1. Install the skill or the MCP server.\n2. Ask your agent a spec question, like \"What does HTTP say about how long a cached response stays fresh?\"\n3. Check that it cites a current RFC by section, like `RFC 9111 §4.2` , and not RFC 2616.\n\nThe agent runs the same loop you can run yourself: search, check obsolescence, list sections, read one.\n\nTitle search works with no setup. To search the text of every RFC, download the section index once.\n\nFor the skill, where `<skills-dir>` is where it was installed, like `~/.claude/skills`:\n\n```\npython3 <skills-dir>/rfc-lookup/scripts/rfc.py sync --index\n```\n\nFor the MCP server:\n\n```\nuvx mcp-server-rfc sync --index\n```\n\nBoth write to the same directory, so one download serves both.\n\nThe index is a SQLite file holding every RFC, compressed, and an FTS5 index over each section. A search returns sections ranked by BM25, with the number to read next:\n\n``` bash\n$ rfc search \"how long can a cache serve a stale response\" --fulltext\nRFC 9111 — HTTP Caching  [INTERNET STANDARD]\n   §4.2.4 Serving Stale Responses  (line 784, 17 lines)\n   790: A cache MUST NOT generate a stale response if it is prohibited by an\n```\n\nReads come from the index too, so nothing is fetched from the RFC Editor after that.\nOn the [retrieval eval](https://github.com/shbernal/rfc-ai-tooling/blob/main/docs/search-evaluation.md) it finds the answering section in its top 10 for more than half of the held-out questions; ripgrep over the corpus finds it for none.\n\nWithout the index, `--fulltext` falls back to `rg` or `grep` over a synced mirror of the corpus as plain files, which is also what `--regex` needs:\n\n```\nuvx mcp-server-rfc sync\n```\n\n|  | Download | On disk | Source | \n|---|---|---|---|\n| `sync --index` | About 270 MB | 360 MB | This project's GitHub releases, checked against a published SHA-256 | \n| `sync` | 512 MB, about 5 minutes | 512 MB | The RFC Editor's public rsync mirror, capped at 2 MB/s by default ( `--bwlimit` ) | \n\nNeither starts on its own, and the skill tells agents not to run them unasked.\n`rfc index build` makes the index from a synced mirror, in about two and a half minutes.\n\n```\nflowchart LR\n    editor[(\"RFC Editor\")]\n    releases[(\"GitHub releases\")]\n    index[(\"Section index<br/>360 MB, optional\")]\n    mirror[(\"Local mirror<br/>512 MB, optional\")]\n    core[\"core/rfc.py<br/>stdlib only\"]\n    skill[\"Skill<br/>scripts/rfc.py\"]\n    mcp[\"mcp-server-rfc<br/>stdio or HTTP\"]\n    shell[\"Agent with a shell\"]\n    client[\"MCP client\"]\n\n    editor -- \"HTTPS, on demand\" --> core\n    editor -. \"rsync, when you run sync\" .-> mirror\n    releases -. \"when you run sync --index\" .-> index\n    index --> core\n    mirror --> core\n    core -- \"vendored copy\" --> skill\n    core -- \"vendored copy\" --> mcp\n    skill --> shell\n    mcp --> client\n```\n\n`core/rfc.py` is the only implementation.\nThe skill and the MCP server each carry an identical copy, and CI fails if one drifts.\nThe core parses the RFC Editor's index for titles, status and what obsoletes what, then fetches documents over HTTPS and caches them.\nWith the section index, it reads from that file and ranks sections with SQLite's FTS5.\nWith only a synced mirror, it reads from disk and searches with `rg` or `grep`.\n\n- [`skill/SKILL.md`](https://github.com/shbernal/rfc-ai-tooling/blob/main/skill/SKILL.md) : every CLI command and how the agent is told to use them\n- [`mcp/README.md`](https://github.com/shbernal/rfc-ai-tooling/blob/main/mcp/README.md) : the MCP tools and their arguments\n- [`docs/desktop-verification.md`](https://github.com/shbernal/rfc-ai-tooling/blob/main/docs/desktop-verification.md) : proving Claude Desktop called the tools\n- [`CHANGELOG.md`](https://github.com/shbernal/rfc-ai-tooling/blob/main/CHANGELOG.md) : breaking changes are listed there, so read it before upgrading\n- [`CONTRIBUTING.md`](https://github.com/shbernal/rfc-ai-tooling/blob/main/CONTRIBUTING.md) : layout, tests and release rules\n- [`NOTICE`](https://github.com/shbernal/rfc-ai-tooling/blob/main/NOTICE) : attribution, and the terms that apply to RFC text", "url": "https://wpnews.pro/news/rfc-ai-tooling-40x-fewer-tokens-while-reading-the-rfc-corpus", "canonical_source": "https://github.com/shbernal/rfc-ai-tooling", "published_at": "2026-10-27 09:29:57+00:00", "updated_at": "2026-10-27 09:41:38.880589+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "agent-protocols", "developer-tools", "ai-search"], "entities": ["rfc-ai-tooling", "IETF", "RFC Editor", "mcp-server-ietf", "mcp-server-rfc", "Claude Desktop", "Claude Code", "Cursor"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/rfc-ai-tooling-40x-fewer-tokens-while-reading-the-rfc-corpus", "markdown": "https://wpnews.pro/news/rfc-ai-tooling-40x-fewer-tokens-while-reading-the-rfc-corpus.md", "text": "https://wpnews.pro/news/rfc-ai-tooling-40x-fewer-tokens-while-reading-the-rfc-corpus.txt", "jsonld": "https://wpnews.pro/news/rfc-ai-tooling-40x-fewer-tokens-while-reading-the-rfc-corpus.jsonld"}}