{"slug": "i-split-my-site-s-ai-bot-traffic-three-ways-only-one-turns-into-a-citation", "title": "I split my site's AI bot traffic three ways. Only one turns into a citation.", "summary": "A developer analyzed Cloudflare logs from their own site over a 23.5-hour window and split AI bot traffic into three categories: live answers (189 fetches), search indexing (541 fetches), and training (1,178 fetches). The developer found that only live-answer fetches, such as ChatGPT-User and Perplexity-User, can lead to citations and real visitors, while most AI traffic is training noise. They also discovered that a large portion of training fetches came from unnamed crawlers and that Cloudflare's bot categories can be unreliable, recommending raw User-Agent matching.", "body_md": "Everyone keeps saying \"AI is reading your site now.\" I wanted the actual number, not the vibe. So I read my own Cloudflare logs and split every AI bot fetch by what the fetch is FOR. One 23.5 hour window, my own site.\n\nHere is what came back.\n\n**Live answers: 189.** These are ChatGPT-User, Perplexity-User and Claude-User. A real person asked a question, and the assistant fetched my page right then to write the answer. This is the fetch that can become a citation, because there is a human on the other end of it, mid decision.\n\n**Search index: 541.** OAI-SearchBot, PerplexityBot, bingbot, Applebot. These build the index the assistants search over. Indirect, but it is how you become retrievable in the first place. Bing matters more than it looks here: it grounds Copilot and it receives ChatGPT's search queries.\n\n**Training: 1,178.** GPTBot, ClaudeBot, Google-Extended, CCBot and friends. Collecting data for a model that ships months from now. The biggest bucket by far, and the least useful to you today.\n\nHere is the honest part of that last number: 974 of the 1,178 training fetches came from crawlers Cloudflare flags as \"AI\" but does not name. I cannot attribute them precisely, and that is its own finding. A large slice of the AI traffic hitting a site is currently unverifiable, because nobody publishes the list you would need to check it against.\n\nIf you look at a dashboard that says \"2,000 AI bots visited you this week,\" you are mostly looking at training noise. It feels like progress. It is not the thing that sends you a customer.\n\nThe 189 live-answer fetches are the ones worth chasing. In one week I measured, 23 to 29 July, six real people arrived on my site from an AI answer: three from ChatGPT, two from Bing, one from Perplexity, across four referring sources. A browser beacon counts them and no bot triggers it, so a referral there is a human. Five of the six landed on a page built from my own measurement data. Zero landed on a page that sells, not the home page, not pricing, not the check. Every assistant that sent me a person sent them to a page of data, and none of those people met an offer. That is the most useful number in this whole post.\n\nLump all three buckets into \"AI traffic\" and you lose that story. Split them, and the story is obvious: publish something worth fetching in the middle of an answer, and the live-answer number is the one that moves.\n\nYou do not need a tool for this. Match the User-Agent in your logs:\n\n`ChatGPT-User`\n\n, `Perplexity-User`\n\n, `Claude-User`\n\n`OAI-SearchBot`\n\n, `PerplexityBot`\n\n, `Claude-SearchBot`\n\n, `bingbot`\n\n, `Applebot`\n\n`GPTBot`\n\n, `ClaudeBot`\n\n, `Google-Extended`\n\n, `CCBot`\n\n, `Bytespider`\n\nOne warning: do not trust Cloudflare's own bot category alone. On my site `PerplexityBot`\n\ncame back with an empty category while `ChatGPT-User`\n\nwas labelled correctly, so leaning on the label would have silently dropped one of the three assistants I care about most. Match the raw User-Agent yourself.\n\nAlso filter to successful responses (status under 400). A credential scanner was spoofing `PerplexityBot`\n\nand `OAI-SearchBot`\n\nuser agents to probe for `/.env`\n\nand `/config`\n\nfiles, all 404, and every one of those was inflating my \"assistant\" count until I filtered them out. It cut the number by a third.\n\nThe live-answer fetches cluster on my pages that carry original data, the ones where I measured something and published the raw result. They do not cluster on my pricing page or my homepage. So the move is not \"optimise the landing page for AI.\" It is \"publish a number nobody else has, on a page an assistant can quote in one sentence.\"\n\nI build a tool that measures exactly this, so I run it on myself and publish the numbers, training noise and all. If you want your own three-way split without reading logs by hand, that is what it does, and the first check is free at [glotier.com](https://glotier.com). But everything above is the method, and your Cloudflare logs already have the data. Go look.", "url": "https://wpnews.pro/news/i-split-my-site-s-ai-bot-traffic-three-ways-only-one-turns-into-a-citation", "canonical_source": "https://dev.to/oguzhantusen/i-split-my-sites-ai-bot-traffic-three-ways-only-one-turns-into-a-citation-1113", "published_at": "2026-08-09 13:57:21+00:00", "updated_at": "2026-08-09 14:22:36.238697+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "ai-infrastructure"], "entities": ["Cloudflare", "ChatGPT", "Perplexity", "Claude", "Bing", "Applebot", "GPTBot", "Google-Extended"], "alternates": {"html": "https://wpnews.pro/news/i-split-my-site-s-ai-bot-traffic-three-ways-only-one-turns-into-a-citation", "markdown": "https://wpnews.pro/news/i-split-my-site-s-ai-bot-traffic-three-ways-only-one-turns-into-a-citation.md", "text": "https://wpnews.pro/news/i-split-my-site-s-ai-bot-traffic-three-ways-only-one-turns-into-a-citation.txt", "jsonld": "https://wpnews.pro/news/i-split-my-site-s-ai-bot-traffic-three-ways-only-one-turns-into-a-citation.jsonld"}}