# llms.txt v2: what an agent actually reads

> Source: <https://dev.to/richicodes/llmstxt-v2-what-an-agent-actually-reads-3c43>
> Published: 2026-10-24 07:30:39+00:00

**This note has an interactive demo.** [Try it in the original](https://richi.codes/article/2026-10-23-llms-txt-v2-what-an-agent-actually-reads)

The llms.txt proposal got its first big revision in August 2026, and the changes answer the obvious question nobody could answer before: given a page, how does an agent find the llms.txt that describes it? I read the new text, ran this site's own `/llms.txt` through the small reference parser that lives in the proposal's repository, and it crashed. The demo above shows both things: what a parser really gets out of the file, and how an agent walks from a page to its llms.txt and its Markdown version.

It's a Markdown file that gives agents a curated entry point into a site: a name, a short summary, some context and lists of links to LLM-friendly content. Jeremy Howard published the original in September 2024. It's still a proposal "open for community input", not a W3C or IETF standard. The text is clear about what it's for: robots.txt says what automated access is acceptable, while llms.txt "is instead used on demand, when an agent needs information about a topic while assisting a user". It's meant for inference, not training.

The Changes page lists four things:

`/docs/llms.txt` covers everything under `/docs/`, and "where more than one file applies, agents should use the most specific one". This is what lets a GitHub Pages project site, which can't touch the host root, take part.`.md` appended (`page.html.md`). v2 also allows replacing the extension (` page.md`). URLs without a file name use `index.html.md` or `index.md`.` rel="alternate" type="text/markdown"` points to a page's Markdown version and `rel="describedby"` points to the llms.txt that covers it, either as `<link>` elements or as an HTTP header.`## Optional` lost its special meaning.`Optional` is now just a convention for secondary links.
The header form is the interesting one, because you can add it in your server or CDN config without touching a single page, and it also works on non-HTML files. This is the proposal's own example:

```
Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"
```

And the equivalent in the page's `<head>`:

```
<link rel="alternate" type="text/markdown" href="https://dev.to/docs/page.html.md">
<link rel="describedby" href="https://dev.to/docs/llms.txt">
```

The "How it gets found" tab in the demo builds that header for each page of a small example site. These are the two functions behind it, which I ran in Node with the outputs shown:

```
// The llms.txt files that cover a path, most specific first.
function coveringLlms(path, files) {
  return files
    .filter(f => path.startsWith(f.slice(0, f.lastIndexOf('/') + 1)))
    .sort((a, b) => b.length - a.length);
}

// page.html -> page.html.md ('append') or page.md ('replace').
function markdownUrl(path, mode) {
  if (path.endsWith('/')) return path + (mode === 'append' ? 'index.html.md' : 'index.md');
  return mode === 'append' ? path + '.md' : path.replace(/\.[^./]+$/, '') + '.md';
}

const files = ['/llms.txt', '/docs/llms.txt'];
coveringLlms('/docs/api/fetch.html', files); // ['/docs/llms.txt', '/llms.txt']
markdownUrl('/docs/api/fetch.html', 'replace'); // '/docs/api/fetch.md'
markdownUrl('/docs/', 'append');                // '/docs/index.html.md'
```

Untick "/docs/llms.txt exists" and the docs pages fall back to the root file.

The proposal says the file is "in a precise format allowing fixed processing methods (i.e. classical programming techniques such as parsers and regex)". In order: an H1 with the name (the only required part), a blockquote with a short summary, zero or more paragraphs or lists with no headings, then H2 sections whose lists hold `[name](url)` links, optionally followed by `:` and notes.

The repository still ships a parser of about twenty lines, `llms_txt/miniparse.py`, and the demo's first tab is a line-for-line JavaScript port of it. I ran both on the same seven files, this site's llms.txt and the proposal's own included, and they return the same result in every case. Here is what I found by feeding it real input:

``` python
from miniparse import parse_llms_txt

parse_llms_txt("# My site\n\n> What it is.\n\nOne line of details.\n\n## Docs\n\n- [Guide](https://example.com/guide.md): start here\n")
# {'title': 'My site', 'summary': 'What it is.', 'info': 'One line of details.',
#  'sections': {'Docs': [{'title': 'Guide', 'url': 'https://example.com/guide.md', 'desc': 'start here'}]}}
```

`[name](url)` link.`- GitHub: https://github.com/me` raises `AttributeError`. So does a long list item wrapped onto a second, indented line, because the parser splits sections line by line. This site's `/llms.txt` does both, which is why the "richi.codes today" example opens in red.`>` lines keeps its first line as the summary and dumps the rest into the details.`##`, the summary comes back as `None` and the blockquote ends up in `info`. With only an H1 before the first `llms.txt` in the proposal's own repository comes back with `summary: None` for this reason.`###` also splits.`^##`, so `### Getting started` becomes a section called `# Getting started`. The spec doesn't allow headings before the first H2 anyway.
An agent reading the file as plain Markdown won't care about any of this. But if a tool parses llms.txt the way the proposal invites it to, these details decide whether your links survive. The "richi.codes fixed" example shows the cheap fix: one line per link, real Markdown links, and a short paragraph after the summary.

*Originally published at [richi.codes](https://richi.codes/article/2026-10-23-llms-txt-v2-what-an-agent-actually-reads).*