AI Companies Announce New Blocking Standard
Anthropic, OpenAI, Google and Microsoft announced a new robots.txt token, __usage_ai-training, that lets webmasters block or allow all four companies' AI crawlers with two lines of code instead of listing each user agent…