19:10
2026-08-14
searchenginejournal.com
ai-policy
OpenAI Says Robots.txt May Not Apply To ChatGPT’s Fetch Bot
OpenAI's ChatGPT-User bot accessed disallowed pages on nearly half of European sites that explicitly blocked it, and TollBit's State of the Bots report for the first half of 2026 found that about 15% …