cd /news/ai-crawlers/we-fetched-104-ai-built-sites-the-wa… · home › topics › ai-crawlers › article
[ARTICLE · art-152070] src=dev.to ↗ pub= topic=ai-crawlers verified=true sentiment=· neutral

We fetched 104 AI-built sites the way GPTBot does. 61% were empty.

A team at 402registry fetched 120 sites built with AI app builders the way AI crawlers like GPTBot, ClaudeBot and PerplexityBot do—a single plain HTTP request with no JavaScript—and found that 61% of the 104 that responded returned HTML with fewer than 300 characters of visible text. The study, run on 20 September 2026, included a control group and proposed fixes for the empty-page problem.

by read1 min views5 publishedOct 17, 2026
We fetched 104 AI-built sites the way GPTBot does. 61% were empty.
Image: Dev.to

On 20 September 2026 we fetched 120 sites made with AI app builders the way AI crawlers do: plain HTTP, no JavaScript. 61% of the live ones were empty. The data, the control group and the fixes.

On 20 September 2026 we fetched 120 websites built with AI app builders the way GPTBot, ClaudeBot and PerplexityBot fetch a page: one plain HTTP request, no browser, no JavaScript. 104 of them answered. For 63 of those 104 (61%), the HTML held fewer than 300 characters of visible text.

Read the full post on 402registry: https://402registry.com/blog/ai-built-sites-empty-without-javascript

── more in #ai-crawlers 4 stories · sorted by recency
── more on @402registry 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/we-fetched-104-ai-bu…] indexed:0 read:1min 2026-10-17 · —