What Is llms.txt, and Does It Work for Your Docs? Google's AI optimization guide states that Search and its AI features do not treat llms.txt as a special signal, and SE Ranking's study of about 300,000 domains found no correlation between having the file and earning AI citations, according to an analysis by documentation-software vendor Jamdesk. The evidence is more favorable for coding agents: in Mintlify's July 2026 benchmark, a linked llms.txt cut Claude Code's 404s on Markdown docs from 0.79 to 0.10 per task, while Ahrefs found that 97% of about 38,000 sites with llms.txt received no requests for it in May 2026. Jeremy Howard, co-founder of Answer.AI, proposed llms.txt in September 2024 as a Markdown file listing a site's most useful pages with one-line descriptions. What Is llms.txt, and Does It Work for Your Docs? You’ve probably heard of llms.txt by now. Maybe someone on your team saw a thread calling it the new robots.txt, or an SEO tool flagged that your site doesn’t have one. The idea is simple. It’s a small Markdown file on your site that tells AI tools which pages matter most and what each one covers. Whether it matters for your website and docs depends on what you want from it. If developers point coding agents like Claude Code at your docs, the file can help those agents find the right page with fewer wrong turns. If you’re hoping it gets you cited in ChatGPT or Google’s AI answers, the evidence so far says it won’t. For most docs sites, that makes it worth adding if your platform generates it for you, and not worth much more time than that. We make documentation software https://www.jamdesk.com/?utm source=blog&utm medium=article&utm campaign=does-llms-txt-work that generates llms.txt, so we have a reason to want it to work. That’s why we went to the sources. We read the original proposal, Google’s guidance, two large studies, a benchmark whose data we recomputed, and our own docs traffic. Below is what the file is, where it helps, where it doesn’t, and how much effort it deserves. The Short Version - Google’s AI optimization guide https://developers.google.com/search/docs/fundamentals/ai-optimization-guide says Search and its AI features don’t treat llms.txt as a special signal. - SE Ranking studied about 300,000 domains and found no link https://seranking.com/blog/llms-txt/ between having llms.txt and getting AI citations. - In Mintlify’s July 2026 benchmark https://www.mintlify.com/blog/llms-txt-agent-benchmark , a linked llms.txt cut Claude Code’s 404s on Markdown docs from 0.79 to 0.10 per task. - Ahrefs found that 97% of about 38,000 sites with llms.txt got no requests for it https://ahrefs.com/blog/llmstxt-study/ in May 2026. What Is an llms.txt File? llms.txt is a Markdown file that lists a site’s most useful pages, each with a one-line description, under a short introduction. Jeremy Howard, co-founder of Answer.AI, proposed it in September 2024 https://www.answer.ai/posts/2024-09-03-llmstxt.html so models could use a website while answering someone’s question. A small one looks like this: Acme API Documentation for Acme's payments API. Get Started - Quickstart https://docs.acme.com/quickstart.md : Make your first request - Authentication https://docs.acme.com/auth.md : Create and use an API key - Webhooks https://docs.acme.com/webhooks.md : Verify and handle events The current proposal https://llmstxt.org/ puts the file at /llms.txt or under a path such as /docs/llms.txt , where it covers the pages beneath that path. Only the project-name heading is required, but the descriptions and links are what make it useful. The file doesn’t grant access to private content or tell crawlers what they may read. You still need sign-in for access control, robots.txt for crawl rules, and pages that answer the reader’s question. It also helps to keep two jobs apart. One is helping an assistant find your authentication guide. The other is getting your product recommended in an AI answer. Either can happen in a chat app or a coding agent, and the evidence for each is very different. Does llms.txt Help You Get Cited by AI? No study we found shows that it does. Google is explicit about it. Its AI optimization guide https://developers.google.com/search/docs/fundamentals/ai-optimization-guide says Search, AI features included, doesn’t use llms.txt or similar files as a special mechanism. Google may crawl the file like any other URL, and that’s all. For other answer engines, SE Ranking’s study https://seranking.com/blog/llms-txt/ , first published in November 2025, looked at about 300,000 domains and found no correlation between having llms.txt and getting AI citations. Its prediction model even did better once the llms.txt variable was removed. That’s observational research, so it can’t say what adding the file would do for your own site. Traffic studies answer a smaller question. Ahrefs’ June 2026 study https://ahrefs.com/blog/llmstxt-study/ covered 137,210 domains and detected llms.txt on roughly 38,000 of them. Of those, 97% got no requests for the file during May, which is a lot of files waiting by the phone. Ahrefs notes that its customers are more technical and SEO-minded than the web at large. None of that justifies selling llms.txt as a citation tactic. It also doesn’t prove the file is useless when an agent does ask for it, and answering that takes watching an agent at work. Does llms.txt Help Agents Find the Right Page? Yes, in the one test with public data. Mintlify’s July 2026 benchmark https://www.mintlify.com/blog/llms-txt-agent-benchmark served docs four ways: HTML, Markdown, Markdown with a linked llms.txt, and Markdown with the index pasted into the page. Claude Code and Codex, two coding agents, answered questions across 20 Mintlify-hosted sites, for 2,400 valid attempts. The useful comparison is Markdown with and without the index. The table counts the average requests per task that came back 404 page not found , which covers wrong guesses at a page’s address and requests for an llms.txt that wasn’t there. We recalculated these from the released results https://github.com/mintlify/url-discovery-bench/blob/5f055a89336cb94050e36eaa479374b390e92462/data/full/results.jsonl rather than rerunning the agents, using the same 2,400 valid attempts the publisher used and leaving out the 272 that failed. | Documentation Format | Claude Code: 404s/Task | Codex: 404s/Task | |---|---|---| | HTML | 1.12 | 3.33 | | Markdown without the index | 0.79 | 2.05 | | Markdown with llms.txt available and linked | 0.10 | 0.15 | The detailed results https://github.com/mintlify/url-discovery-bench/blob/5f055a89336cb94050e36eaa479374b390e92462/RESULTS.md show that most of the HTML version’s 404s were requests for .md pages or an llms.txt that didn’t exist yet, so much of the drop comes from those files appearing. On plain Markdown, more of the 404s were real wrong guesses at a page’s address. Both agents found the right page 94% to 99% of the time in every format, so the index mostly cut the wrong turns on the way. The test measured picking the URL, not the code or answer the agent wrote afterward. It’s also a vendor’s benchmark on its own hosting, with agents told to fetch through a proxy using curl and no search or normal browsing. For docs served as Markdown, I’d still include the linked index. Our own traffic shows the file gets used, without saying how well. From September 12 to October 12, 2026, Jamdesk-hosted docs received 160 llms.txt requests from three tools that fetch pages for a person crawlers not counted , after we removed known audit and internal test traffic. Claude Code sent 83 of them and ChatGPT-User 62, with Claude-User sending the other 15. Those are requests, not people, and we didn’t see the prompts or what the agent did next. Anthropic documents web search https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool and fetching a given page https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-fetch-tool as separate tools, so a request alone can’t tell you whether someone was comparing products, writing an integration or asking the agent to read the file. How Do You Help Agents Find Your llms.txt? Give the tool a starting point. Link the index from your quickstart or agent setup page, and give developers a prompt to paste into a browsing-enabled assistant: Start with https://docs.example.com/llms.txt. Find the webhook signature verification guide, then show me how to verify an incoming request using the current Node SDK. Cite the documentation page you used. That points the tool straight at the index instead of hoping an answer engine finds it. The proposal also describes discovery links https://llmstxt.org/ for every page: rel="describedby" points to the index, and rel="alternate" with type="text/markdown" points to a Markdown copy of the page. On the authentication page, that looks like this: