This note has an interactive demo. Try it in the original
The llms.txt proposal got its first big revision in August 2026, and the changes answer the obvious question nobody could answer before: given a page, how does an agent find the llms.txt that describes it? I read the new text, ran this site's own /llms.txt through the small reference parser that lives in the proposal's repository, and it crashed. The demo above shows both things: what a parser really gets out of the file, and how an agent walks from a page to its llms.txt and its Markdown version.
It's a Markdown file that gives agents a curated entry point into a site: a name, a short summary, some context and lists of links to LLM-friendly content. Jeremy Howard published the original in September 2024. It's still a proposal "open for community input", not a W3C or IETF standard. The text is clear about what it's for: robots.txt says what automated access is acceptable, while llms.txt "is instead used on demand, when an agent needs information about a topic while assisting a user". It's meant for inference, not training.
The Changes page lists four things:
/docs/llms.txt covers everything under /docs/, and "where more than one file applies, agents should use the most specific one". This is what lets a GitHub Pages project site, which can't touch the host root, take part..md appended (page.html.md). v2 also allows replacing the extension ( page.md). URLs without a file name use index.html.md or index.md. rel="alternate" type="text/markdown" points to a page's Markdown version and rel="describedby" points to the llms.txt that covers it, either as <link> elements or as an HTTP header.## Optional lost its special meaning.Optional is now just a convention for secondary links.
The header form is the interesting one, because you can add it in your server or CDN config without touching a single page, and it also works on non-HTML files. This is the proposal's own example:
Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"
And the equivalent in the page's <head>:
<link rel="alternate" type="text/markdown" href="https://dev.to/docs/page.html.md">
<link rel="describedby" href="https://dev.to/docs/llms.txt">
The "How it gets found" tab in the demo builds that header for each page of a small example site. These are the two functions behind it, which I ran in Node with the outputs shown:
// The llms.txt files that cover a path, most specific first.
function coveringLlms(path, files) {
return files
.filter(f => path.startsWith(f.slice(0, f.lastIndexOf('/') + 1)))
.sort((a, b) => b.length - a.length);
}
// page.html -> page.html.md ('append') or page.md ('replace').
function markdownUrl(path, mode) {
if (path.endsWith('/')) return path + (mode === 'append' ? 'index.html.md' : 'index.md');
return mode === 'append' ? path + '.md' : path.replace(/\.[^./]+$/, '') + '.md';
}
const files = ['/llms.txt', '/docs/llms.txt'];
coveringLlms('/docs/api/fetch.html', files); // ['/docs/llms.txt', '/llms.txt']
markdownUrl('/docs/api/fetch.html', 'replace'); // '/docs/api/fetch.md'
markdownUrl('/docs/', 'append'); // '/docs/index.html.md'
Untick "/docs/llms.txt exists" and the docs pages fall back to the root file.
The proposal says the file is "in a precise format allowing fixed processing methods (i.e. classical programming techniques such as parsers and regex)". In order: an H1 with the name (the only required part), a blockquote with a short summary, zero or more paragraphs or lists with no headings, then H2 sections whose lists hold [name](url) links, optionally followed by : and notes.
The repository still ships a parser of about twenty lines, llms_txt/miniparse.py, and the demo's first tab is a line-for-line JavaScript port of it. I ran both on the same seven files, this site's llms.txt and the proposal's own included, and they return the same result in every case. Here is what I found by feeding it real input:
from miniparse import parse_llms_txt
parse_llms_txt("# My site\n\n> What it is.\n\nOne line of details.\n\n## Docs\n\n- [Guide](https://example.com/guide.md): start here\n")
[name](url) link.- GitHub: https://github.com/me raises AttributeError. So does a long list item wrapped onto a second, indented line, because the parser splits sections line by line. This site's /llms.txt does both, which is why the "richi.codes today" example opens in red.> lines keeps its first line as the summary and dumps the rest into the details.##, the summary comes back as None and the blockquote ends up in info. With only an H1 before the first llms.txt in the proposal's own repository comes back with summary: None for this reason.### also splits.^##, so ### Getting started becomes a section called # Getting started. The spec doesn't allow headings before the first H2 anyway.
An agent reading the file as plain Markdown won't care about any of this. But if a tool parses llms.txt the way the proposal invites it to, these details decide whether your links survive. The "richi.codes fixed" example shows the cheap fix: one line per link, real Markdown links, and a short paragraph after the summary.
Originally published at richi.codes.