llms.txt v2: what an agent actually reads

A developer tested the August 2026 revision of the llms.txt proposal, running this site's own /llms.txt through the proposal's roughly twenty-line reference parser (llms_txt/miniparse.py) and finding that it crashed. The v2 changes define how agents discover the file: hierarchical /docs/llms.txt scoping with most-specific-file precedence, .md and .html.md Markdown variants, and rel="alternate" type="text/markdown" plus rel="describedby" link relations delivered via HTML head or HTTP Link headers, while ## Optional loses its special meaning. The note includes a JavaScript port of the parser and helper functions for resolving covering llms.txt files and Markdown URLs.

This note has an interactive demo. Try it in the original https://richi.codes/article/2026-10-23-llms-txt-v2-what-an-agent-actually-reads The llms.txt proposal got its first big revision in August 2026, and the changes answer the obvious question nobody could answer before: given a page, how does an agent find the llms.txt that describes it? I read the new text, ran this site's own /llms.txt through the small reference parser that lives in the proposal's repository, and it crashed. The demo above shows both things: what a parser really gets out of the file, and how an agent walks from a page to its llms.txt and its Markdown version. It's a Markdown file that gives agents a curated entry point into a site: a name, a short summary, some context and lists of links to LLM-friendly content. Jeremy Howard published the original in September 2024. It's still a proposal "open for community input", not a W3C or IETF standard. The text is clear about what it's for: robots.txt says what automated access is acceptable, while llms.txt "is instead used on demand, when an agent needs information about a topic while assisting a user". It's meant for inference, not training. The Changes page lists four things: /docs/llms.txt covers everything under /docs/ , and "where more than one file applies, agents should use the most specific one". This is what lets a GitHub Pages project site, which can't touch the host root, take part. .md appended page.html.md . v2 also allows replacing the extension page.md . URLs without a file name use index.html.md or index.md . rel="alternate" type="text/markdown" points to a page's Markdown version and rel="describedby" points to the llms.txt that covers it, either as <link elements or as an HTTP header. Optional lost its special meaning. Optional is now just a convention for secondary links. The header form is the interesting one, because you can add it in your server or CDN config without touching a single page, and it also works on non-HTML files. This is the proposal's own example: Link: </docs/page.html.md ; rel="alternate"; type="text/markdown", </docs/llms.txt ; rel="describedby" And the equivalent in the page's <head : <link rel="alternate" type="text/markdown" href="https://dev.to/docs/page.html.md" <link rel="describedby" href="https://dev.to/docs/llms.txt" The "How it gets found" tab in the demo builds that header for each page of a small example site. These are the two functions behind it, which I ran in Node with the outputs shown: // The llms.txt files that cover a path, most specific first. function coveringLlms path, files { return files .filter f = path.startsWith f.slice 0, f.lastIndexOf '/' + 1 .sort a, b = b.length - a.length ; } // page.html - page.html.md 'append' or page.md 'replace' . function markdownUrl path, mode { if path.endsWith '/' return path + mode === 'append' ? 'index.html.md' : 'index.md' ; return mode === 'append' ? path + '.md' : path.replace /\. ^./ +$/, '' + '.md'; } const files = '/llms.txt', '/docs/llms.txt' ; coveringLlms '/docs/api/fetch.html', files ; // '/docs/llms.txt', '/llms.txt' markdownUrl '/docs/api/fetch.html', 'replace' ; // '/docs/api/fetch.md' markdownUrl '/docs/', 'append' ; // '/docs/index.html.md' Untick "/docs/llms.txt exists" and the docs pages fall back to the root file. The proposal says the file is "in a precise format allowing fixed processing methods i.e. classical programming techniques such as parsers and regex ". In order: an H1 with the name the only required part , a blockquote with a short summary, zero or more paragraphs or lists with no headings, then H2 sections whose lists hold name url links, optionally followed by : and notes. The repository still ships a parser of about twenty lines, llms txt/miniparse.py , and the demo's first tab is a line-for-line JavaScript port of it. I ran both on the same seven files, this site's llms.txt and the proposal's own included, and they return the same result in every case. Here is what I found by feeding it real input: python from miniparse import parse llms txt parse llms txt " My site\n\n What it is.\n\nOne line of details.\n\n Docs\n\n- Guide https://example.com/guide.md : start here\n" {'title': 'My site', 'summary': 'What it is.', 'info': 'One line of details.', 'sections': {'Docs': {'title': 'Guide', 'url': 'https://example.com/guide.md', 'desc': 'start here'} }} name url link. - GitHub: https://github.com/me raises AttributeError . So does a long list item wrapped onto a second, indented line, because the parser splits sections line by line. This site's /llms.txt does both, which is why the "richi.codes today" example opens in red. lines keeps its first line as the summary and dumps the rest into the details. , the summary comes back as None and the blockquote ends up in info . With only an H1 before the first llms.txt in the proposal's own repository comes back with summary: None for this reason. also splits. ^ , so Getting started becomes a section called Getting started . The spec doesn't allow headings before the first H2 anyway. An agent reading the file as plain Markdown won't care about any of this. But if a tool parses llms.txt the way the proposal invites it to, these details decide whether your links survive. The "richi.codes fixed" example shows the cheap fix: one line per link, real Markdown links, and a short paragraph after the summary. Originally published at richi.codes https://richi.codes/article/2026-10-23-llms-txt-v2-what-an-agent-actually-reads .