The 490,000-page study found a 10% rate overall, with commercial sites far ahead of government and education domains.
By Ryan Merket · Published
Primary source: Pew Research Center
Why it matters #
A 35% AI-authorship rate among newer dated pages raises provenance problems for search, publishing and future model training, while Pew's limits show why detectors cannot adjudicate individual authorship.
Pew Research Center found signs of AI writing or substantial editing on 35% of webpages with publication dates after ChatGPT's November 2022 release, according to a study published on August 20th.
The result, reported by TechCrunch, puts a number on a shift that publishers, search companies and AI developers have helped accelerate: generative models are producing a growing share of the material available for people, crawlers and future models to read.
Samuel Bestvater, a senior data scientist at Pew and the study's lead researcher, analyzed 490,000 English-language pages collected by Common Crawl, the nonprofit repository of public web data. Bestvater's previous research has covered link rot, social media use and electric-vehicle charging infrastructure, giving the project a longer view of how the public web changes and disappears.
The headline figure comes with a narrower denominator than "the internet." Pew randomly selected 10,000 pages from each of 49 Common Crawl snapshots created between January 2021 and July 2026. Only about 10% to 15% of the sampled pages had publication dates in their HTML, allowing Pew to identify them as published before or after ChatGPT's November 30th, 2022 launch.
That means the 35% finding describes pages with detectable dates, rather than every page published during the period. Paywalled sites and pages requiring a login are also likely underrepresented because Common Crawl collects publicly accessible material.
Across the full random sample of 10,000 pages in the July 2026 crawl, including older material that predates widely available generative AI, 10% showed significant signs of AI authorship. The gap between that figure and the post-ChatGPT result captures how quickly synthetic or AI-edited writing has entered newer portions of the web.
How Pew tested the web
Pew ran each page's body text through Open Pangram's editlens_Llama-3.2-3B model, which assigns text a score from zero for fully human-written material to one for fully AI-generated material. Pew treated a score of 0.2 or higher as meaningful evidence of AI authorship or editing.
That threshold can capture pages in which a person wrote the original text and used a model for substantial revisions. It also makes the finding broader than a count of pages produced entirely from prompts.
Pew tested the open model's results against Pangram's commercial Pangram 3.3 detector on 62,370 pages from seven crawls. The two systems agreed on classifications in 96% of cases, though Pew cautioned that their page-level judgments remain probabilistic. The open model also classified about 1% of pre-ChatGPT pages as AI-authored, which Pew identified as likely evidence of a higher false-positive rate on older, human-written material.
Pangram released Open Pangram in March 2026 as a source-available, open-weight research tool based on its EditLens work. Pangram explicitly warns against using the open models to enforce AI policies in schools or workplaces. That caveat matters here: Pew's large sample can show a population-level trend even when the detector cannot establish the authorship of any single page with certainty.
Commercial sites are moving fastest
The growth was concentrated on commercial domains. In Pew's 2026 samples, roughly one in 10 pages on .com domains showed signs of AI authorship. The rate was 4.6% for .org pages and around 1% for both .edu and .gov pages.
Pew also tracked writing habits associated with model output. Compared with webpages sampled in 2023, em dashes appeared about twice as frequently, Oxford comma use rose 63%, and a basket of words often favored by AI models more than doubled in frequency. Negative parallelism, the "it's not just X, it's Y" construction, nearly tripled, although it remained uncommon overall.
Those habits do not prove machine authorship. Human writers use every one of them. Pew used the linguistic shifts as aggregate evidence alongside the detector's analysis, rather than as a checklist for judging individual articles.
A separate study by researchers from Imperial College London, the Internet Archive and Stanford University reached a similar result using Internet Archive data and several detection methods. That group classified about 35% of newly published websites by mid-2025 as AI-generated or AI-assisted. It found that AI-classified pages had higher semantic similarity and more positive sentiment, while finding no statistically significant relationship with factual errors or reduced stylistic diversity.
The production shift is arriving alongside an equally large change in consumption. Cloudflare reported in July that non-human systems had crossed 50% of internet traffic measured by the company. Cloudflare said AI-training crawlers represented 52% of crawler requests in June 2026, up from 22% in spring 2025.
Pew's research measures content rather than visitors, so the two findings are not directly comparable. Together, they describe a web where automated systems account for a growing share of both the pages being published and the requests retrieving them. For publishers, search engines and model developers, authorship and provenance are becoming basic infrastructure questions rather than metadata that can be safely ignored.