The conversation around AI visibility is changing. Publishers once focused on whether AI answer engines like ChatGPT would send enough referral traffic to offset declining clicks from traditional search, have mostly moved on.
Publishers are now beginning to treat AI answer engines less as a source of human referrals and more as a distribution layer. Success now depends on controlling how AI agents consume their content, determining how brands value being surfaced in AI answers and how publishers can monetize an evolving ecosystem where mentions and citations matter more than clicks.
A number of new reports and analyses of agentic traffic trends highlight why publishers are building updated strategies around AI visibility, in a web ecosystem now made up of more bots than humans.
Agentic traffic is surging #
A report from DataDome published last week shows AI agent traffic grew 45% in Q2 2026. DataDome’s network of over 400 companies recorded 17.7 billion AI agent requests in April-June 2026, a 45% increase from Q1’s 12.2 billion. June had 6.6 billion AI agent requests alone.
Meta now generates the majority of AI agent traffic on DataDome’s network, both from its training and RAG crawlers. Meta’s training crawler grew 74% from Q1 to Q2, while its RAG crawler grew faster, up 163% in the same time period.
“We’re entering a new phase of the AI web,” said DataDome vp Jérôme Segura. “Meta is shifting from ‘scrape once for training’ to continuous indexing for real-time AI answers. As AI traffic becomes more persistent, publishers need to set guidelines for crawling and indexing like they have been doing for Google. It’s imperative to understand which agents access content, how often, and what they receive in return.”
Meanwhile, a July report from Decodo examined Cloudflare data and found AI-driven traffic grew by approximately 187% in 2025, around eight times faster than human traffic over the same period. Automated systems now generate 57.4% of web requests, compared with 42.6% from humans, according to Decodo’s analysis. AI agent traffic grew by approximately 7,851% year over year.
More websites are building for agents #
To prepare for the agentic web, more companies are creating agent-readable versions of their web content to make it easier for AI agents to access and understand. Markdown and LLMs.txt — a specific type of markdown file — are web formats that strip away HTML information to give agents content and metadata, without the surrounding page design.
AI detection company Originality.ai published a report and dashboard last month that tracks the growth of AI web standards like LLMs.txt.
The report found that LLMs.txt instances grew from 4,088 in June 2025 to 36,120 by May 2026, an 8.8X increase across the 3 million websites it analyzed. Originality.ai identified 38,980 sites adopting web standards like llms.txt, llms-full.txt, and ai.txt.
However, some publishers may be ahead of the game here. DataDome’s report found that LLMs.txt has reached only 3.2% adoption among publishers in July, below the 7.9% on the open web.
And Ahrefs’ server-log research found 97% of LLMs.txt files received zero requests in May 2026, according to Originality.ai’s report. Jon Gillham, CEO at Originality.ai, said this means markdown pages may not be the key to improving AI visibility. (Google’s own GEO guidance is that LLMs.txt is not necessary for generative AI search optimization.)
“The 8.8 [times] growth in LLMs.txt adoption shows that publishers are searching for some control over how AI interacts with their content. But adoption isn’t usage. If 97% of these files are never requested, this isn’t an AI visibility strategy yet. It’s a low-cost bet on an agent-driven future which might include llms.txt,” Gillham said.
AI visibility is a tough game to play #
Another sign that AI visibility is a tough game to play — despite it becoming a metric of growing importance for some publishers — is the fact that it’s not easy to get mentioned or cited in an AI answer.
Webflow, a visual web development platform, created an “AEO Maturity Index” that assesses a company’s SEO and GEO performance. It analyzed 2,000 websites, looking at their published content, technical GEO readiness, brand authority and measurement capabilities. It found that the median company appeared in 16% of relevant AI answers, and appeared in a citation with a link only 6% of the time, according to Guy Yalif, chief evangelist at Webflow. Issues with the fundamental basics of SEO were usually culprits for underperforming GEO, he said, such as broken links, missing metadata, and stale content.
“We are in early days in a new medium,” Yalif said. “[Publishers] can, at [their] core, answer questions, make it really easy for the LLMs to consume their content, show up in a bunch of places, and measure the right stuff.”
Webflow developed a rubric called the Webflow AEO Maturity Model in these four categories. The more prepared a company was for GEO, the higher the mention and citation rates they were seeing.
Large companies tended to perform better, according to Webflow’s data, with two-times higher mention rates, 60% higher citation rates and 80% higher share of voice in AI search.
AI referral traffic still meager #
New data also illustrates why publishers seem to be shifting away from seeing AI referral traffic as a growth opportunity. DataDome’s report found that ChatGPT-User, which fetches live pages to answer real-time questions, visited websites 6% less in Q2 than it did in Q1.
But, it’s not all bad news for publishers. Humans using ChatGPT clicked through to websites 17% more, with ChatGPT accounting for upwards of 80% of all AI-driven referral traffic.
DataDome’s Segura said this points to why publishers are moving beyond blanket AI bot policies. Reuters and Time, as examples, have recently started blocking all AI bots by default but created whitelists of approved bots to access content on their sites.
“Not all AI agents are the same,” said DataDome’s Segura. “The question needs to evolve beyond simply allowing or blocking AI agents to determining which ones should be trusted and why. The agents worth paying attention to aren’t only the ones hitting your servers hardest, but also the ones actually sending visitors. The publishers best positioned for the agentic web are those that build policies around an agent’s purpose, identity, and business value, rather than treating every AI agent the same.”
Some publishers are still trying to block bots #
Speaking of bot blocking, more new publishers are moving to block AI bots — even if the methods to do so aren’t iron-clad. A report from web data firm HasData found that so far this month, over half of news publishers were blocking at least one AI crawler in robots.txt.
Of the 10,894 domains that were analyzed, 56.4% of news publishers now block at least one AI crawler, compared to 10.3% of the open web. GPTBot specifically is banned by 50.5% of publishers, compared to 7.9% of the web.
But bot blocking doesn’t always work. Of the sites that declared a GPTBot ban and were live-tested, 39.5% served GPTBot a page anyway, HasData found.
“Long-term, publishers are in a ‘change-or-die’ spot because their protection only works with polite bots. So, most likely they are losing their AI search visibility, and at the same time, not stopping the traffic they fear most,” said Roman Milyushkevich, CTO and CEO of HasData. “It can be extremely difficult to distinguish agent traffic from browser traffic. So publishers should think about proper access deals and monetization that would work for both of these categories instead of simple walls.”
More in Media
‘It’s a CEO conversation,’ Accenture and Whalar on their playbook for the creator economy
Brian Yasko, managing director at Accenture and Emma Harman, co-CEO of Whalar, on the massive creator economy acquisition and future plans.
[
WTF is LLM honeypotting? ](https://digiday.com/media/wtf-is-llm-honeypotting/)
Publishers and e-commerce brands under siege from AI crawlers are starting to fight back with an old security trick updated for the LLM era: “LLM honeypotting.”
[
Why a once-anonymous creator unmasked herself to build a bigger media brand ](https://digiday.com/media/why-a-once-anonymous-creator-unmasked-herself-to-build-a-bigger-media-brand/)
Kristi Cook used to YouTube anonymously. Once she revealed her face, her account became wildly popular.