{"slug": "list-of-companies-that-block-internet-archive", "title": "List of companies that block Internet Archive", "summary": "An incomplete list compiled by Consumer Rights Wiki identifies companies that have blocked the Internet Archive's Wayback Machine, with 23 major news outlets including The New York Times and USA Today updating their robots.txt files in April 2023 to explicitly block the crawler, according to a report by Am John Philip. The blocking is largely motivated by publishers' desire to prevent their content from being used to train AI models.", "body_md": "Inherently incomplete. Sourced to https://archive.is/Lz4nY\n\n| ← Older revision | Revision as of 21:27, 17 August 2026 | ||\n| Line 1: | Line 1: | ||\n{{Incomplete}} | {{Incomplete}} | ||\nThis is | This is an early/inherently incomplete list of companies who took the lead their websites purposefully blacklisted/excluded from the Internet Archive (IA) for various reasons, whether for malice reasons (plausible deniability) or not. For these websites, utilize other archive websites found [[Consumer_Rights_Wiki:Tools_for_writing_articles#Archived_citations|here]]. | ||\nUnless there's a way for IA to have the content but not make it available to train AI, a large proportion of content sites will be blocking it, in order to avoid their content being available to train AI. So the list is very incomplete. For example, it was reported in April of 2023 \"Twenty-three major news outlets, including [[The New York Times]] and [[USA Today]] have updated their [[robots.txt]] files to explicitly block the Internet Archive’s Wayback Machine crawler.\"<ref>{{Cite news |title=The Internet Archive just got blocked by Big News. (Paywalled; Archive.is has complete article) |url=https://amjohnphilip.medium.com/the-internet-archive-just-got-blocked-by-big-news-f74fe59bbd2f |archive-url=https://archive.is/Lz4nY |archive-date=17 Aug 2026}}</ref> | |||\nThere might be websites with [[Cloudflare]] DDoS protection, which falls in a grey area for archival purposes, though should not be listed on this article. | There might be websites with [[Cloudflare]] DDoS protection, which falls in a grey area for archival purposes, though should not be listed on this article. |", "url": "https://wpnews.pro/news/list-of-companies-that-block-internet-archive", "canonical_source": "https://consumerrights.wiki/index.php?title=List_of_companies_that_block_Internet_Archive&diff=64013&oldid=63285", "published_at": "2026-08-17 21:27:34+00:00", "updated_at": "2026-08-17 21:41:21.752235+00:00", "lang": "en", "topics": ["ai-ethics", "ai-policy"], "entities": ["Internet Archive", "Consumer Rights Wiki", "The New York Times", "USA Today", "Am John Philip", "Cloudflare"], "alternates": {"html": "https://wpnews.pro/news/list-of-companies-that-block-internet-archive", "markdown": "https://wpnews.pro/news/list-of-companies-that-block-internet-archive.md", "text": "https://wpnews.pro/news/list-of-companies-that-block-internet-archive.txt", "jsonld": "https://wpnews.pro/news/list-of-companies-that-block-internet-archive.jsonld"}}