AI Crawlers Are Expensive. Agent Builders Should Care.
Kernel.org administrator Konstantin Ryabitsev reported that AI crawlers consumed 14-16 of 90 CPU cores across five servers, continuously rendering Git commit pages for scrapers despite repositories being available for cl…