# I tracked AI crawlers on my new GEO site for 30 days — here's what I learned

> Source: <https://dev.to/jflu/i-tracked-ai-crawlers-on-my-new-geo-site-for-30-days-heres-what-i-learned-5e9l>
> Published: 2026-08-28 08:27:00+00:00

I launched a site focused on Generative Engine Optimization (GEO) about 2 months ago. Instead of just tracking Google rankings, I set up Cloudflare D1 logging to track every AI crawler visit.

Here's what 30 days of data shows:

**Crawler breakdown:**

**Key insights:**

**Bing is the new Google for AI.** Bingbot volume dwarfs everything else. If you're optimizing for GEO, Bing Webmaster Tools is your best free analytics source.

**ClaudeBot is remarkably consistent.** Same crawl pattern every day, suggesting scheduled indexing rather than on-demand. If you publish content, expect 24-48hr lag before Claude "sees" it.

**Bytespider (ByteDance) is aggressive.** 500+ hits/day on a small site. They're clearly doing massive data collection. If you want visibility in Doubao/DeepSeek, your content needs to be crawlable.

**External referrers matter more than I expected.** A single Reddit post drove 8 AI crawler visits from a domain I'd never heard of (sahammurah.com). External mentions = AI discovery signals.

**What I'm doing differently now:**

Anyone else tracking AI crawlers? What patterns are you seeing?