cd /news/generative-engine-optimization/serving-http-accept-text-markdown-co… · home › topics › generative-engine-optimization › article
[ARTICLE · art-172976] src=dev.to ↗ pub= topic=generative-engine-optimization verified=true sentiment=↑ positive

Serving HTTP Accept: text/markdown, Content-Signal & /llms-full.txt on Cloudflare Pages (2026 GEO Guide)

A developer built GeoReady, a free real-time edge inspector, and an open-source zero-dependency MCP server (yixuanrobot/geoready-mcp) that audits any URL in about five seconds for AI-agent readiness. The tool checks robots.txt rules for bots like OAI-SearchBot, PerplexityBot and ClaudeBot, llms.txt and llms-full.txt presence, JSON-LD structured data, and Content-Signal HTTP headers. The project promotes serving Markdown via HTTP Accept: text/markdown content negotiation, which the developer says cuts payloads from roughly 16,000 to 2,000 tokens.

key takeaway

Serving Markdown via Accept: text/markdown with Vary: Accept and a Content-Signal header cuts agent payloads from about 16,000 to 2,000 tokens, and the GeoReady MCP server audits robots.txt, llms.txt, JSON-LD and Content-Signal for any URL in about five seconds.

read1 min views1 publishedNov 20, 2026

Modern AI coding and shopping agents (ClaudeBot, OAI-SearchBot, PerplexityBot, Claude Code) waste up to 85% of their context window parsing inline CSS, SVGs, and layout boilerplate when fetching raw HTML.

In February 2026, HTTP content negotiation via Accept: text/markdown and the Content-Signal header emerged as the cleanest white-hat Generative Engine Optimization (GEO) pattern for SaaS and Shopify sites:

Accept: text/html) receive your normal interactive page; Accept: text/markdown) receive 1:1 visible content converted into clean YAML-frontmatter Markdown tables (Vary: Accept), cutting payload size from ~16,000 tokens to ~2,000 tokens with zero SEO cloaking risk. We built a free real-time edge inspector — GeoReady — plus an open-source zero-dependency MCP server (yixuanrobot/geoready-mcp) that audits any URL in 5 seconds across:

robots.txt`` OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended + firewall HTTP 403 challenge detection)/llms.txt & /llms-full.txt`` index.html fallbacks)Organization, Product/ SoftwareApplication, FAQPage) Content-Signal HTTP Headersai-train=yes, search=yes, ai-input=yes)_headers Snippet Drop this into your static output folder's _headers file:

/*
  Content-Signal: ai-train=yes, search=yes, ai-input=yes
  Vary: Accept
  Link: <https://yourdomain.com/llms.txt>; rel="llms-txt", <https://yourdomain.com/llms-full.txt>; rel="llms-full-txt"

uvx)

{
  "mcpServers": {
    "geoready-auditor": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/yixuanrobot/geoready-mcp",
        "geoready-mcp"
      ]
    }
  }
}

Test your own domain's live score and generate a tailored llms.txt + robots.txt + JSON-LD kit free at https://geoready.aitruelabs.com.

── more in #generative-engine-optimization 4 stories · sorted by recency
── more on @geoready 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/serving-http-accept-…] indexed:0 read:1min 2026-11-20 · —