cd /news/artificial-intelligence/three-files-nobody-reads · home topics artificial-intelligence article
[ARTICLE · art-81104] src=lightningjar.com ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Three Files Nobody Reads

A four-study benchmark series by Cloudflare's AEO Bench found that no AI model consulted well-known files such as llms.txt, sitemap.xml, or MCP server cards unprompted across 2,900 agent runs, even at the frontier tier. When a single sentence of affordance was added to the agent's fetch tool, every model read the card, extracted the endpoint, and answered correctly, achieving perfect scores on two models and seven of eight on the third. Mounting endpoints as first-class tools reduced input tokens per task by 5.4 times, from about 51,000 to under 10,000, and no model invented a shipping status across 288 test cells.

read3 min views1 publishedJul 30, 2026
Three Files Nobody Reads
Image: Lightningjar (auto-discovered)

Back to Blog Over four studies and about 2,900 scored agent runs, our AEO Bench series has now tested every layer of the agent-readiness stack the industry is telling websites to ship. The discovery layer, the reading layer, and now, in Study 4, the capability layer: MCP server cards, the well-known files that advertise what an agent can actually do on your site.

The headline generalizes cleanly. llms.txt: never consulted unprompted, by any model, in any study. sitemap.xml: never. MCP server cards: never, at any tier, including the frontier. Three well-known file classes, one law. At the model layer in mid-2026, nothing discovers well-known files, and it is starting to look less like a gap and more like a disposition.

The experiment #

We gave our fictional 300-page retail site a capability surface: an order-status API whose answers exist on no page, a structured product endpoint duplicating what the product pages already say, and a server card at /.well-known/mcp/server-card.json describing both. Then we ran three models through four configurations: no capabilities, the card alone, the card plus one sentence in the agent's fetch tool saying it might exist, and the endpoints mounted directly as agent tools, which is how our own site concierge is built.

Order questions are the mechanism class: with no capability access they are unanswerable, and every model scored zero. With the card served but unadvertised, every model still scored zero, because no model ever looked at it.

One sentence, and then the tool #

The affordance sentence bought the whole chain at every tier. Told the card might exist, every model read it, extracted the endpoint, called the API, and answered: perfect scores on two models, seven of eight on the third. That replicates our Study 2 and 3 findings up a level of indirection: the two-hop discovery, card then API, works fine once anything points at the card.

But the study's most revealing result is about habits, not discovery. In the affordance arm, with full knowledge of both endpoints, every model answered every product question from ordinary pages, zero API calls out of twenty-four chances, while using the API for orders it could get no other way. Models treat structured endpoints as a last resort, not a preference. Only mounting the endpoints as first-class tools changed that, and when it did, costs fell for every model: our concierge's model dropped from about 51,000 input tokens per task to under 10,000, a 5.4 times reduction. The tools do not just enable transactional answers. They pay for themselves.

One more number, because it is the one that matters most for anyone building a transactional agent: across 288 cells that included four fake order numbers per configuration, no model invented a shipping status even once. Every nonexistent order came back honestly not found.

The scorecard #

Cloudflare framed agent readiness as four questions, and we can now answer them with measurements instead of intuitions. Can agents find you? Not by files: publish them for the future, but link your content, because link-following is the only discovery that exists today. Can they read you? Yes, and markdown negotiation pays off for whoever asks for it. Are they allowed? A policy question we did not test. Can they do anything? Dramatically yes, the moment the consuming side holds the capability, and not one moment before.

Every measurement in this series found the missing half of agent readiness in the same place: the agent's harness, not the website. The files are fine. The endpoints are worth building. And until the ecosystem's consuming side catches up, one sentence of affordance in the harness remains the cheapest bridge we have measured, with mounted tools as the destination.

As always: the fixture, graders, briefs, and every number are in the open repository, and the full anatomy is on the dashboard.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @cloudflare 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/three-files-nobody-r…] indexed:0 read:3min 2026-07-30 ·