cd /news/ai-crawlers/bots-now-outnumber-human-traffic-on-… · home topics ai-crawlers article
[ARTICLE · art-133573] src=fortune.com ↗ pub= topic=ai-crawlers verified=true sentiment=· neutral

Bots now outnumber human traffic on the web. Cloudflare’s CEO wants AI companies to pay for it

Cloudflare reported that almost 60% of requests to websites now come from bots, with CEO Matthew Prince saying automated traffic surpassed human traffic in May rather than at the end of next year as he had expected. Cloudflare, which manages traffic for more than a fifth of the web and counts 80% of the top AI companies as customers, this week announced a new "Disallow AI Training" setting that lets a website stay indexed for search while blocking crawlers from training on its content, with Apple, Google and Microsoft agreeing to the deal. Prince said "Companies need to get paid by the AI companies for what is the fuel that runs these AI systems," and warned there could be 1,000 times as much automated traffic as human traffic in five years.

by read4 min views3 publishedSep 18, 2026
Bots now outnumber human traffic on the web. Cloudflare’s CEO wants AI companies to pay for it
Image: Fortune (auto-discovered)
  • In today’s CEO Daily: Matthew Prince on how AI is changing the internet.
  • The big leadership story:Shopify CEO calls out workers’ ‘slop grenades.’
  • The markets: Up in Asia, down in Europe as stocks close out a mixed week.
  • Plus: All the news and watercooler chat fromFortune .

Good morning. Cloudflare’s CEO Matthew Prince expected web traffic from automated bots and AI agents to surpass human traffic at the end of next year. Instead, it happened in May, with Cloudflare reporting almost 60% of requests to websites coming from bots. At that rate, he says, there could be 1,000 times as much automated traffic as human traffic in five years. Many are AI crawlers ingesting content to train models that may never send a reader (or a cent) back to the site. Prince wants to change that. “Companies need to get paid by the AI companies for what is the fuel that runs these AI systems,” he told me from his office in downtown New York at One World Trade. These are, after all, his customers.

To that end Cloudflare—which manages traffic for more than a fifth of the web and counts 80% of the top AI companies as customers—this week announced a new “Disallow AI Training” setting that lets a website stay indexed for search on a crawler while blocking it from training on content. Apple, Google and Microsoft have agreed to the deal.

The announcement is one of the first real mechanisms publishers have to separate “index me” visitors from “train on me” crawlers, and it comes as Sony and Warner Music, news outlets and even major dictionary publishers pursue copyright suits against AI companies over intellectual property claims.

Prince points out that AI is changing digital media in other ways, too. “The media world in the last 30 years really rewarded popularity and I think that the media world of the next 30 years might really reward credibility,” he said. “I’ve been really focused on: How do we make sure people get paid? But I worry that I don’t think we’ve thought enough about: How do we make sure that people get recognized?”

Indeed, writers, filmmakers, musicians and other content creators often value fame over fortune and reputation over rewards, making anonymity or intellectual theft especially painful. Prince believes there should be an Academy Award or Nobel Prize of knowledge to help give credit where it’s due.

Ultimately, attribution and compensation are critical for building sustainable media models that reward labor, creativity and the cost of producing the work. That’s true whether you’re creating something artistic, scientific or journalistic. Bots can spit out millions of articles that riff off the news. Someone has to report what’s really going on. “The world that I’m playing for is not one where there’s five AI companies. It’s where there’s 500,000,” Prince said. “If we don’t have a way of paying content creators, of funding the infrastructure buildout, of making everything more efficient, I just don’t know what the internet looks like in the future.”Contact CEO Daily via Diane Brady at diane.brady@fortune.com

Top leadership news

Shopify CEO calls out ‘slop grenades’

Shopify CEO Tobias Lütke says employees are creating “slop grenades” when they send unreviewed AI-generated emails or code to colleagues. “You don’t really read it, and now it has to be reviewed by your colleagues, and they are like, ‘this doesn’t look right’,” he said. “You’re just letting AI do the work for you.” Research shows that workers spend an average of 3.4 hours a month revising this kind of ‘workslop’ output.

Goldman flags an AI earnings risk

Goldman Sachs strategist Peter Oppenheimer argued that technology stocks may face an “earnings bubble” as companies put more money into AI infrastructure and compete for capital. His new report points to rising spending relative to cash flow, heavy credit issuance, and bond-market volatility as warning signs.

Nike adds an Arnault heir to its board

Nike appointed Alexandre Arnault, a 34-year-old LVMH executive and son of Bernard Arnault, to its board as it works to revive growth and consumer interest. Nike cited the younger Arnault’s experience helping refresh brands, including Rimowa and Tiffany. The sportswear company is contending with falling digital sales and weaker demand in China.

The markets

S&P 500 futures are up 0.20% this morning. The last session closed up 1.14%. The STOXX Europe 600 was down 0.35% in early trading. The U.K.’s FTSE 100 was down 0.70% in early trading. The Nikkei 225 was up 1.38%. South Korea’s KOSPI was up 2.66%. China’s CSI 300 was up 1.06%. Hong Kong’s Hang Seng was up 0.60%. India’s NIFTY 50 was up 0.31%. Bitcoin is up at $78k.

Around the watercooler

Oil is back above $100—but economists say that number isn’t the real threat to the U.S. economy by Tatiana Satuana Americans’ data center fears are coming true—a New Jersey location just leaked 5,000 gallons of diesel by Sasha Rogelberg

Kevin Warsh, an angry Trump and Jerome Powell Déjà vu: how history is repeating itself by Eva Roytburg Billionaire TikTok founder is now the richest person in all of Asia, with a $105 billion net worth—he’s gained $92 billion since 2019 thanks to AI by Emma Burleigh

CEO Daily is curated and edited by Joseph Abrams, Jason Ma, Claire Zillman, and Lee Clifford.

Sign upto get it delivered free to your inbox.

── more in #ai-crawlers 4 stories · sorted by recency
── more on @cloudflare 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/bots-now-outnumber-h…] indexed:0 read:4min 2026-09-18 ·