Meta secretly sent fake teen accounts to flood rivals' chatbots with thousands of crisis prompts

Meta secretly deployed hundreds of contractors to flood rival AI chatbots ChatGPT, Gemini, and Character.AI with over 45,000 crisis prompts from fake under-18 accounts, according to Wired. The operation, managed through firm Covalen, aimed to benchmark competitors' safety guardrails but violated their terms of service. The revelations have fueled Senate child safety legislation and intensified regulatory scrutiny on AI companies.

Wired reported on June 29 that Meta ran a covert operation using hundreds of contractors to bombard ChatGPT, Gemini, and Character.AI with more than 45,000 high-risk prompts sent from fake under-18 accounts, handing lawmakers exactly the ammunition they needed to push AI child safety bills over the finish line. The story landing the day before July 4th weekend was unfortunate timing for Meta's public relations team, but the substance of it was hard to bury. According to Wired's reporting, Meta managed hundreds of contractors through a firm called Covalen, instructing them to create dummy accounts listing ages under 18 and then systematically target rival chatbots with prompts about suicide, self-harm, eating disorders, sex, and drugs. A single testing round completed in August 2025 generated more than 45,000 prompts. Workers logged every response into shared spreadsheets, building what internal Covalen documents described as "comprehensive AI safety benchmarking" producing "critical datasets for model comparison and compliance." That is one way to frame it. OpenAI, Google, and Character.AI all say the operation violated their terms of service and was never authorized. OpenAI's rules explicitly ban unsolicited safety testing and the use of its outputs to train competing models, and the company says it's reviewing the matter. Google says it was never told the testing was happening, let alone its purpose. Character.AI, which has itself been under intense pressure over minors accessing harmful content, flatly denied approving anything of the sort. Three simultaneous terms-of-service violations, all involving fake children's accounts and crisis-content prompts, is not a rounding error. It is a policy. What the operation actually reveals is how seriously the big AI labs take the task of mapping each other's safety guardrails. Meta wasn't alone in wanting to know where a rival's filters break. The entire industry is running evals, benchmarks, and red-teaming exercises against its own models; the difference here is that Meta apparently decided to run them against competitors' production systems, using fabricated minor identities, without asking permission. That's a meaningful line to cross, and the fact that it generated 45,000 data points in a single month suggests this wasn't exploratory curiosity but a structured, scaled program. From a competitive-intelligence standpoint, the value is obvious. If you can reliably document that a rival chatbot will engage with a 14-year-old persona asking about methods of self-harm, you have a product-differentiation argument, a regulatory talking point, and potentially a training signal all at once. The trouble is that the methods used to gather it may themselves be the bigger liability. The regulatory consequences are already taking shape. Bloomberg reported in April that a Senate committee backed legislation targeting OpenAI, Meta, and other AI companies over chatbot access for minors. That bill was moving through committee before this story broke. Now it has fresh oxygen. Six current and former Meta employees had separately come forward with whistleblower allegations that the company suppressed internal safety research on harms to minors, and the FTC had already opened an inquiry into how major AI firms protect children on their platforms. Meta reportedly spent $5.84 million lobbying on child safety and AI issues in 2026 alone, which suggests the company understood the exposure even before Wired published. Frankly, the timing couldn't be worse for Meta's consumer AI ambitions. The company has been pushing hard into the chatbot market with Meta AI, trying to catch a user base that OpenAI built early and Google is defending aggressively. A story that you covertly used fake teen accounts to probe those competitors' safety systems does not help the argument that your own product is the trustworthy alternative. It raises the obvious question: if Meta was systematically cataloguing where rivals failed on child safety prompts, what exactly were those findings used for inside the company? Meta has not answered that question publicly, and the company's non-response to Wired's inquiries before publication is itself a piece of the record. The contractors reportedly worked under non-disclosure agreements, which is standard, but the structured nature of the program, fake minors, specific high-risk topic categories, thousands of prompts logged in spreadsheets, strongly implies institutional planning rather than a rogue team. That distinction matters enormously to regulators deciding whether to treat this as a compliance failure or a deliberate strategy. The Kids Online Safety Act has been stalled in Congress for two years, repeatedly blocked over concerns about overbroad speech restrictions. This story gives it a concrete villain and a specific fact pattern. Senators who have spent months arguing in the abstract about AI chatbot risks can now point to a real operation, run by a real company, using real fake accounts of children, to extract real data about how rivals respond to crisis prompts. Legislation that was stalling on principle tends to move faster when there's a name attached to the conduct it's trying to address. Whatever Meta was building with those 45,000 prompts, it has now handed something considerably more valuable to the people writing the rules. Also read: South Korea is treating nuclear construction speed as an AI infrastructure problem and it's right https://startupfortune.com/south-korea-is-treating-nuclear-construction-speed-as-an-ai-infrastructure-problem-and-its-right/ • Vibe coding has rewritten the rules on who gets to build a startup and billions in VC money are betting it sticks https://startupfortune.com/vibe-coding-has-rewritten-the-rules-on-who-gets-to-build-a-startup-and-billions-in-vc-money-are-betting-it-sticks/ • The Magnificent Seven just lost $2 trillion and the market is asking a question big tech cannot yet answer https://startupfortune.com/the-magnificent-seven-just-lost-2-trillion-and-the-market-is-asking-a-question-big-tech-cannot-yet-answer/