{"slug": "googles-mueller-says-llms-txt-cant-help-llms-differentiate-sites", "title": "Google’s Mueller Says llms.txt Can’t Help LLMs Differentiate Sites", "summary": "Google's John Mueller argued that llms.txt files cannot help large language models differentiate between websites for discovery, as the files are self-reported and inherently untrustworthy. Mueller suggested llms.txt may still be useful for navigation once an LLM is already on a site, but not for choosing which site to visit.", "body_md": "Google’s John Mueller argued that LLM systems can’t use files like llms.txt to decide which websites to surface for a given query.\n\nHe made the comments on a recent episode of [Search Off the Record](https://search-off-the-record.libsyn.com/should-i-use-markdown-for-my-site), the podcast from Google’s Search Relations team.\n\nHis comment points to a broader signal problem, not just intentional gaming. Even a well-written llms.txt file is still self-reported information from the site that wants to be chosen.\n\nFor discovery, Mueller pointed back to normal HTML pages and internal links.\n\n## What Mueller Said\n\nThe conversation started with a question about whether publishers should convert websites to Markdown for LLMs. Mueller and co-host Martin Splitt agreed that HTML is still the foundation for crawling and discovery.\n\nThe discussion got specific when Mueller turned to llms.txt. He described the discovery use case as a dead end:\n\n“It’s basically you’re telling these systems, like, I have the best website ever. And here are all of the pages that everyone must go to. And you must buy all of my products or whatever you put in there. So in LLM system, it basically, by design, can’t trust what is here as a way of differentiating between different websites.”\n\nHis argument comes down to differentiating. If sites use llms.txt to promote themselves, the files can make similar claims. An LLM deciding which site best answers a query still needs another way to differentiate between them.\n\n## What ‘By Design’ Might Mean\n\n“By design” could mean two different things, and Mueller didn’t clarify which.\n\nOne reading is architectural. LLM systems evaluate web content and can’t use self-reported files when picking sources.\n\nThe other reading treats it as a signal problem. Self-reported signals lose value when everyone provides them. Meta keywords stopped working for the same reason. Every site stuffed them, and search engines couldn’t extract a useful ranking signal.\n\nBoth readings reach the same conclusion on discovery. But they imply different things about whether the limitation could change over time.\n\n## Where Mueller Sees A Role\n\nMueller didn’t reject all uses of llms.txt. He carved out one case where it could help:\n\n“If someone is already on your website, maybe some kind of automated system is helpful.”\n\nHe used the example of an agent trying to buy a photograph from a specific site. The LLM would visit the site and look for instructions on how to complete the purchase.\n\nThe argument splits discovery from navigation. llms.txt can’t help an LLM choose which site to visit. But it could help once the agent is already there, like a store directory for someone who already walked in.\n\n## Beyond The Gaming Argument\n\nMueller has [called building Markdown pages for bots “a stupid idea”](https://www.searchenginejournal.com/googles-mueller-calls-markdown-for-bots-idea-a-stupid-idea/566598/). He’s also [compared llms.txt to the keywords meta tag](https://www.searchenginejournal.com/google-says-llms-txt-is-purely-speculative-for-now/577576/).\n\nSEJ’s Roger Montti [wrote](https://www.searchenginejournal.com/llms-txt-for-ai-seo/556576/) that llms.txt is “inherently untrustworthy” because nothing stops site owners from adding self-serving content. SE Ranking’s [analysis of 300,000 domains](https://www.searchenginejournal.com/llms-txt-shows-no-clear-effect-on-ai-citations-based-on-300k-domains/561542/) found no link between llms.txt adoption and citation frequency in LLM answers.\n\nThose arguments focused on what happens when people game the files. Mueller’s podcast comment adds the nuance that there’s no mechanism within the files to help an LLM pick one site over another.\n\n## Why This Matters\n\nThe gaming argument against llms.txt has always had a counterargument available. Platforms could learn to penalize manipulation, the way search engines handled spammy structured data.\n\nThe differentiation argument leaves a harder problem. Penalizing manipulation may address abuse, but it doesn’t explain how self-reported files help an LLM choose one site over another. Your most accurate llms.txt file still can’t tell an LLM to pick your site over a competitor’s.\n\n## Looking Ahead\n\nStandards for how agents navigate sites haven’t settled yet, Mueller acknowledged. He mentioned [WebMCP](https://www.searchenginejournal.com/google-says-llms-txt-is-purely-speculative-for-now/577576/) alongside other file types under discussion.\n\nNone have become a standard. By his estimate, it could take six months to a year, or longer, for agentic systems to settle on a format. The discovery layer, where HTML and internal linking already work, isn’t part of that discussion.", "url": "https://wpnews.pro/news/googles-mueller-says-llms-txt-cant-help-llms-differentiate-sites", "canonical_source": "https://www.searchenginejournal.com/googles-mueller-says-llms-txt-cant-help-llms-differentiate-sites/579304/", "published_at": "2026-06-15 16:28:02+00:00", "updated_at": "2026-06-17 13:02:44.770223+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-policy"], "entities": ["Google", "John Mueller", "Martin Splitt", "Search Off the Record", "SEJ", "Roger Montti", "SE Ranking"], "alternates": {"html": "https://wpnews.pro/news/googles-mueller-says-llms-txt-cant-help-llms-differentiate-sites", "markdown": "https://wpnews.pro/news/googles-mueller-says-llms-txt-cant-help-llms-differentiate-sites.md", "text": "https://wpnews.pro/news/googles-mueller-says-llms-txt-cant-help-llms-differentiate-sites.txt", "jsonld": "https://wpnews.pro/news/googles-mueller-says-llms-txt-cant-help-llms-differentiate-sites.jsonld"}}