Some AI tools now ask websites for a clean text copy of each page, using the standard header Accept: text/markdown. We checked 50 B2B marketing agencies on November 10, 2026. Of the 47 we could test, only 11 (23%) answered with a clean copy. Citesprout's own site didn't either, until we fixed it the same day.
By Citesprout · November 10, 2026 · Page for people: https://citesprout.com/resources/ai-asks-for-a-clean-copy
A normal web page is mostly layout code, which costs AI tools tokens and buries the words. Some AI tools now send the header Accept: text/markdown, meaning "send the plain text version of this page if you have one."
Answering the standard header is cleaner: anyone who asks gets the same words in a simpler format, and browsers never send the header. Until November 10, 2026, Citesprout only answered named bots; it now answers both ways.
curl -s -o /dev/null -w "%{content_type}\n" -H "Accept: text/markdown" https://yoursite.com/
text/markdown means your site answers. text/html means it doesn't. Free check without a terminal: https://citesprout.com/ai-view
Citesprout sets this up in a Crawlability Fix: https://citesprout.com/services
Lean Labs, Virayo, Ten Speed, Understory, Skydog Ops, RevenueHoop, WarmUp, Kula Partners.
November 10, 2026. Homepages of 50 B2B marketing agencies from our own list. Our research bot identified itself, read only public pages (homepage, robots.txt, llms.txt, possible clean copy), d between sites, skipped sites whose robots.txt asked bots not to crawl, and did not pretend to be any AI company's bot. Three sites couldn't be tested. A one-day snapshot of the basics, homepages only. Re-checks and corrections: https://citesprout.com/contact
They send the HTTP header Accept: text/markdown. A site that supports it answers with a plain text version of the same page. Browsers never send this header.
In Checkly's February 2026 test: Claude Code, Cursor and OpenCode. Not OpenAI Codex, Gemini CLI, GitHub Copilot or Windsurf. This changes as tools update.
Google has not said. The safest approach keeps the content the same, changes only the format, and answers anyone who asks through the standard header.
Cloudflare Pro or above: turn on Markdown for Agents. Otherwise: a server rule that answers the header with a prepared text copy of each page, which Citesprout sets up in a Crawlability Fix.