TollBit's State of the Bots report reveals that unauthorized AI crawlers are increasingly difficult to block, with bad bots masquerading as legitimate ones like Google's, rotating IP addresses, and us…
TollBit's H1 2026 data found that ChatGPT-User ignored explicit robots.txt disallow instructions on 54% of its scrapes and reached disallowed pages on more European publisher sites than any tracked bo…
OpenAI's ChatGPT-User bot accessed disallowed pages on nearly half of European sites that explicitly blocked it, and TollBit's State of the Bots report for the first half of 2026 found that about 15% …
European publishers are experiencing significantly higher rates of AI bot scraping than their North American counterparts, according to a recent report from TollBit, which analyzed AI bots from 40 scr…
Time has begun serving ads to AI agents, with Ally Bank and Project Management Institute among the first buyers, using AI-powered ad tech platform Mobian to convert pages into markdown and generate FA…
Time has begun serving ads to AI bots, with Ally Bank and Project Management Institute among the first buyers, using AI-powered ad tech platform Mobian to convert its pages into markdown and generate …
Cloudflare and AWS now let website owners charge AI crawlers per request using HTTP 402, turning a dormant 1997 status code into a toll booth for machines. Cloudflare launched pay-per-crawl in July 20…
Google is quietly pitching publishers on a new AI program requiring broad rights to use their content for AI training or risk losing existing Showcase payments, but the explicit terms inadvertently gi…
TIME completed a four-month migration from WordPress to Contentstack, unifying over 100 years of fragmented content data to support its AI search product and markdown pages for large language models. …
Time, The Economist, and other major news publishers are re-engineering their websites for AI agents by creating markdown versions of content and implementing bot whitelists. Time is converting all we…
A growing number of schoolboys are using AI chatbots to create personalized virtual girlfriends, raising concerns among educators and child safety experts about the psychological and social impact on …