{"slug": "when-ai-asks-for-a-clean-copy-of-your-page-does-your-site-answer", "title": "When AI asks for a clean copy of your page, does your site answer?", "summary": "Citesprout tested the homepages of 50 B2B marketing agencies on November 10, 2026, and found that only 11 of the 47 it could test (23%) responded to the HTTP header Accept: text/markdown with a clean text copy of the page. Citesprout's own site also failed the check until the company fixed it the same day, and it now answers both named bots and the standard header. The company recommends sites either enable Cloudflare's Markdown for Agents or add a server rule that returns a prepared text copy when the header is present.", "body_md": "Some AI tools now ask websites for a clean text copy of each page, using the standard header Accept: text/markdown. We checked 50 B2B marketing agencies on November 10, 2026. Of the 47 we could test, only 11 (23%) answered with a clean copy. Citesprout's own site didn't either, until we fixed it the same day.\n\nBy Citesprout · November 10, 2026 · Page for people: [https://citesprout.com/resources/ai-asks-for-a-clean-copy](https://citesprout.com/resources/ai-asks-for-a-clean-copy)\n\nA normal web page is mostly layout code, which costs AI tools tokens and buries the words. Some AI tools now send the header `Accept: text/markdown`, meaning \"send the plain text version of this page if you have one.\"\n\nAnswering the standard header is cleaner: anyone who asks gets the same words in a simpler format, and browsers never send the header. Until November 10, 2026, Citesprout only answered named bots; it now answers both ways.\n\n```\ncurl -s -o /dev/null -w \"%{content_type}\\n\" -H \"Accept: text/markdown\" https://yoursite.com/\n```\n\ntext/markdown means your site answers. text/html means it doesn't. Free check without a terminal: [https://citesprout.com/ai-view](https://citesprout.com/ai-view)\n\nCitesprout sets this up in a Crawlability Fix: [https://citesprout.com/services](https://citesprout.com/services)\n\nLean Labs, Virayo, Ten Speed, Understory, Skydog Ops, RevenueHoop, WarmUp, Kula Partners.\n\nNovember 10, 2026. Homepages of 50 B2B marketing agencies from our own list. Our research bot identified itself, read only public pages (homepage, robots.txt, llms.txt, possible clean copy), paused between sites, skipped sites whose robots.txt asked bots not to crawl, and did not pretend to be any AI company's bot. Three sites couldn't be tested. A one-day snapshot of the basics, homepages only. Re-checks and corrections: [https://citesprout.com/contact](https://citesprout.com/contact)\n\nThey send the HTTP header Accept: text/markdown. A site that supports it answers with a plain text version of the same page. Browsers never send this header.\n\nIn Checkly's February 2026 test: Claude Code, Cursor and OpenCode. Not OpenAI Codex, Gemini CLI, GitHub Copilot or Windsurf. This changes as tools update.\n\nGoogle has not said. The safest approach keeps the content the same, changes only the format, and answers anyone who asks through the standard header.\n\nCloudflare Pro or above: turn on Markdown for Agents. Otherwise: a server rule that answers the header with a prepared text copy of each page, which Citesprout sets up in a Crawlability Fix.", "url": "https://wpnews.pro/news/when-ai-asks-for-a-clean-copy-of-your-page-does-your-site-answer", "canonical_source": "https://dev.to/citesprout/when-ai-asks-for-a-clean-copy-of-your-page-does-your-site-answer-4m2e", "published_at": "2026-11-11 18:42:30+00:00", "updated_at": "2026-11-11 18:52:26.159073+00:00", "lang": "en", "topics": ["ai-crawlers", "agent-protocols", "ai-search", "generative-engine-optimization"], "entities": ["Citesprout", "Cloudflare", "Lean Labs", "Virayo", "Ten Speed", "Understory", "Skydog Ops", "RevenueHoop"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/when-ai-asks-for-a-clean-copy-of-your-page-does-your-site-answer", "markdown": "https://wpnews.pro/news/when-ai-asks-for-a-clean-copy-of-your-page-does-your-site-answer.md", "text": "https://wpnews.pro/news/when-ai-asks-for-a-clean-copy-of-your-page-does-your-site-answer.txt", "jsonld": "https://wpnews.pro/news/when-ai-asks-for-a-clean-copy-of-your-page-does-your-site-answer.jsonld"}}