I was burning context feeding HTML to LLMs, so I built a Markdown converter for agents

A developer built a Markdown converter for AI agents that reduces token usage by over 93% compared to raw HTML. The tool, available as TypeScript and Python packages, uses content negotiation to serve clean Markdown to agents while returning HTML to browsers. It works across Node, Bun, Deno, and edge runtimes without requiring a headless browser.

If you build AI agents, scrapers, or RAG pipelines, you have run into this: feeding raw HTML to a model is mostly waste. The <div , the class="..." , the nav, the cookie banner. None of it is the content you wanted, and all of it costs tokens. I knew this was inefficient. I did not appreciate how inefficient until I measured it. Here is a single page of GitHub's documentation, run through an audit: HTML Markdown Savings ─────────────────────────────────────────────────── Tokens 138,550 9,364 -93.2% Chars 554,200 37,456 -93.2% Words 27,123 4,044 Size 541.3 KB 36.6 KB -93.2% Ninety-three per cent of the tokens, gone, with no loss of meaning. The intuition is simple: <h2 About Us</h2 carries the same information as About Us and costs several times more to read. Multiply that across a real page and the boilerplate is most of your payload. Across a scraping or RAG workload, that is the difference between a cost model that works and one that does not. Before writing anything, I looked at the options. Heavy scrapers. Spinning up Puppeteer or Cheerio to strip HTML works, but it drags a headless browser and a pile of dependencies into a project that may not want either. Cloudflare's Markdown for Agents. Cloudflare ships exactly this conversion at the edge, and it is genuinely good. It is also free on their paid plans, so if your site already sits behind Cloudflare, you may not need anything else. The catch is in the requirement itself: it only helps if your traffic runs through Cloudflare. I wanted something that lived in my own code, ran anywhere, and did not assume a particular network sat in front of it. The library credits Cloudflare's work as the inspiration, because that is where the idea came from. What I actually wanted was a small, framework-agnostic way to serve Markdown whenever an agent asked for it, with nothing else attached. The mechanism is content negotiation, the boring HTTP feature that turns out to be exactly right for this. A normal browser visits your site and gets HTML. An agent visits with Accept: text/markdown , and the middleware intercepts the request, strips the boilerplate, and returns clean Markdown from the same URL. No second endpoint, no separate build, no fork in your routing. js import { convert } from 'markdown-for-agents'; const { markdown, tokenEstimate } = convert html, { extract: true } ; The design goals, in order: One dependency. The core relies on a single HTML parser and nothing else. No headless browser, no DOM. Runs anywhere. Node, Bun, Deno, Cloudflare Workers, Vercel Edge, the browser. If it speaks Web Standards, it works. The same idea in Python. There is a zero-dependency Python package alongside the TypeScript one, with middleware for FastAPI, Flask, and Django. Half the RAG and scraping world lives in Python, and I did not want to leave it out. Drop-in middleware. Express, Fastify, Hono, Next.js, and any Web Standard server. The middleware reads the Accept header, passes normal browser traffic through untouched, and converts only when an agent asks. Point the audit tool at any URL and see what you would save: npx @markdown-for-agents/audit https://docs.github.com/en/copilot/get-started/quickstart Or paste a URL or raw HTML into the playground and watch the conversion happen live: https://markdown-for-agents.vercel.app/playground https://markdown-for-agents.vercel.app/playground I built it because I wanted my own websites to be prepared and agent-friendly, and I was seeing my own usage limits get burned by HTML-only websites. If you are dealing with context limits or token costs from web content, it might save you the same. The code is here: https://github.com/KKonstantinov/markdown-for-agents https://github.com/KKonstantinov/markdown-for-agents%60%60