22:23
2026-07-31
dev.to
ai-products
Your robots.txt decides whether ChatGPT can cite you. Here's the 5-minute check.
A developer warns that robots.txt files often accidentally block AI retrieval agents, preventing sites from being cited in ChatGPT, Claude, and Perplexity answers. The developer found that major news …