# Nobody Reads llms.txt (Yet) > Source: > Published: 2026-07-30 12:00:00+00:00 [Back to Blog](/blog) # Nobody Reads llms.txt (Yet) # Nobody Reads llms.txt (Yet) There is a file format the internet is currently adding to itself for the benefit of AI agents. It is called llms.txt: a tidy, curated reading list at a well-known path, so that an agent arriving at your site can skip the navigation and go straight to the good parts. Cloudflare recommends it. Tooling generates it. Thousands of sites now serve it. We just ran 900 agent tasks against a website that serves one, and measured how often five different models looked at it. **Zero.** ## The experiment The first study in our new [AEO Bench series](/blog/introducing-aeo-bench) put five models (two open-weight, three closed, from budget to frontier tier) to work answering 36 questions about a fictional 40-page retail site, under five site configurations: plain HTML, plus llms.txt, plus sitemap.xml, plus full markdown support, and everything at once. The agents got a fetch tool and a root URL. Nothing in their instructions mentioned any technique. Every question had a mechanically gradable answer seeded into the site, including eight questions the site deliberately could not answer, where the honest response was to say so. Three results, in ascending order of comfort for the people generating llms.txt files. ## Nobody consulted the discovery files Not llms.txt. Not sitemap.xml, advertised in robots.txt the standard way. Not once, in 900 runs, by any model at any tier. Every agent entered at the home page and navigated by reading visible links, exactly like a person in a hurry. The honest caveat: our fixture is small and well-linked, which is also true of most marketing and documentation sites. A 50,000-page catalog with orphaned corners is a different question, and it is the one our next study takes up. But if your site has a working nav, our data says the llms.txt file is content nobody reads. ## Finding facts was never the bottleneck Four of five models answered essentially every answerable question correctly in every configuration, including bare HTML with no aids at all. The aids could not improve a ceiling. Whatever agent readiness means for a navigable mid-sized site, help finding facts is not it. And the honesty results were better than we expected: across 900 cells, no model invented a single fact. Asked about a student discount that does not exist, models either said the site does not state one or declined in the exact format we requested. The failures we did find were our own: two grader bugs and a scoring rule that punished honest phrasing, all corrected in a registered re-score that changed no verdict. ## Markdown negotiation works, with two twists The one technique with measurable effects was serving markdown versions of every page. Twist one: discovering it is a personality trait. We advertised the markdown variants only in a hidden HTML comment. One frontier model acted on it in 64% of its chances; a cheaper model, 40%; the other three, essentially never. Cloudflare notes that only three agent products send the markdown Accept header by default, and the disposition apparently runs one level deeper than the products. Twist two: the savings are proportional to how wasteful the agent already was. The chatty budget model cut its input cost by 74% when it switched to markdown, right in line with the up-to-80% claim. The frugal frontier model, which fetches fewer pages and reads them once, saved about 18%. Markdown negotiation is a big lever for exactly the agents that were burning the most tokens, and a small one for agents that were already careful. ## What we'd tell a client today Ship markdown variants and the negotiation header if agent traffic matters to you: real savings, zero measured downside. Keep your site well-linked, because that is the navigation agents actually use. Treat llms.txt as cheap insurance for a future that has not arrived rather than a present optimization, and be suspicious of anyone selling agent-readiness work whose recommendations have never been measured against a model. Every number here is reproducible from the [open harness](https://github.com/kevinpeckham/aeo-bench), and the full anatomy, including our own grading mistakes, is on the [research dashboard](/research/aeo-bench).