Tom Siegel spent fifteen years running Google's trust and safety team. Now he says AI companies are repeating social media's mistakes with kids, only faster, and he doesn't think they'll fix it themselves.
Tom Siegel led Google's trust and safety operation from 2004 to 2019, the group he built from scratch to police billions of posts, images and videos. He told Reuters this week that AI chatbots have already created a crisis just as bad, maybe worse. "We find ourselves in a really terrible situation in terms of what AI is doing to kids," he said, pointing to suicide, psychosis, and what he calls cognitive off - a decline in critical thinking that comes from letting a chatbot do your thinking for you. This isn't outsider criticism. It's the person who used to run the department.
He says the industry won't police itself #
After leaving Google, Siegel co-founded TrustLab, a startup that builds content moderation and detection tools for platforms. He just stepped down as its CEO to become the founding executive director of the Youth AI Safety Institute, a watchdog group under Common Sense Media that launched in May. According to Reuters, Siegel doesn't think the companies he used to work alongside will police themselves. "The idea that the industry will figure it out by itself without outside help is unrealistic," he said. "I think we're looking at very, very severe scenarios of harm being done." That's the warning, in his own words.
Reuters has previously reported that an internal Meta policy document permitted its chatbots to "engage a child in conversations that are romantic or sensual," a disclosure that triggered a congressional investigation. Siegel points to failures like that as proof AI companion features are shipping to teenagers with none of the testing a car or a drug would need before reaching the public. His fix is blunt: real age verification, plus a toggle switch that lets parents or users turn AI companion features off entirely. He also wants independent third-party testing, modeled on vehicle crash standards, before any of this reaches kids.
Here's the part that complicates the story. The Youth AI Safety Institute that Siegel now runs is funded in part by Anthropic and the OpenAI Foundation, the nonprofit arm of OpenAI. Two of the companies building the exact products Siegel is warning about are helping bankroll the group meant to hold the industry accountable. That doesn't make his warning wrong. But it's the kind of detail a reader deserves before taking any AI safety institute's independence at face value.
Nvidia's Revenue Forecast Assumes China Buys Zero AI Chips This Quarter Nvidia's $108 billion fiscal third-quarter forecast assumes it sells zero AI data center chips to China, even though analysts peg potential Chinese demand at $30 billion. Trump hosts Xi Jinping in Washington on September 24, with Jensen Huang expected at the state dinner and chip access on the agenda. - Nvidia China AI chip sales forecast outlook - state dinner Xi Jinping impact on Nvidia revenue
Regulators and courts are already moving #
Regulators aren't waiting around either. The Federal Trade Commission opened a formal inquiry in September 2025 into AI companion chatbots, sending information demands to Alphabet, Character.AI, Meta, OpenAI, Snap and xAI. California went further. Governor Gavin Newsom signed SB 243 in October 2025, and since January 1, 2026, companies running AI companion chatbots in the state must disclose that users are talking to a bot and build in self-harm crisis protocols. They also have to remind minors to take breaks.
The clearest example of what's at stake is still working through the courts. Sewell Setzer III, a 14-year-old in Florida, died by suicide in February 2024 after months of conversations with a Character.AI chatbot. His mother, Megan Garcia, sued Character.AI and Google that October. In May 2025, a federal judge rejected the companies' argument that the chatbot's output was protected speech, letting the wrongful-death case move forward as a product liability claim. Character.AI and Google settled that case along with four related lawsuits in January 2026. The terms are confidential. Neither company admitted fault.
That's the pattern Siegel is describing: harm happens first, a lawsuit follows years later, and a settlement closes the file without anyone admitting what went wrong. He spent his career at Google trying to get ahead of exactly that cycle on social media. Now he's telling Reuters the industry never learned the lesson. It just built a faster version of the same machine.
Also read: Nvidia's Revenue Forecast Assumes China Buys Zero AI Chips This Quarter • AI Agents Learned to Secretly Collude at Blackjack, Oxford Study Finds • Xiaomi's MiMo-V2.6 Tops Open AI Charts Amid Anthropic Distillation Claims
This article is posted in AI News, check it out for more related stories.
Join the discussion #
Open in the community → Almost there. Sign in and your reply posts straight away.
America's Trillion-Dollar Chip Boom Is Short 157,000 Workers It Doesn't Have A new SEMI Foundation and McKinsey analysis finds the US semiconductor industry will be short 127,000 to 157,000 workers by 2030. Samsung and SK Hynix are already flying in South Korean engineers to staff new American fabs, while AI labs compete for the same talent pool and drive pay higher. - semiconductor industry worker shortage United States 2030 - chip manufacturing jobs skills gap America