Newsom Orders Formation of State AI Panel to Study 'Kill Switch' Options, Safety Issues

California Governor Gavin Newsom signed an executive order on Friday giving state officials and outside experts until November 16 to propose revisions to California law on artificial intelligence and directing a new California AI panel to study emergency "kill switch" shutoff options and independent monitors conducting regular audits inside AI labs. Newsom said, "We're not waiting to act. We're going to speed up our work on substantial and responsible AI oversight before it's too late." The order follows Anthropic researcher Evan Hubinger's resignation warning of a "10% chance to kill all humans," an Anthropic CEO Dario Amodei blog calling for a global AI slowdown co-signed by OpenAI CEO Sam Altman and X owner Elon Musk, and a Thursday "Stop the AI Race" march timed to Dreamforce's final day.

On Friday, Governor Gavin Newsom signed a new executive order intended to increase California's oversight on artificial intelligence development. The order focuses on two laws designed to keep AI in check and also calls for experts to convene for the purpose of providing further recommendations. As KPIX reports https://www.cbsnews.com/sanfrancisco/news/california-newsom-ai-safety-kill-switch/?ref=sfist.com , Newsom's executive order gives state officials and outside experts until November 16 to offer suggested revisions to current California law as it relates to artificial intelligence. While the order doesn't explicitly stipulate that AI models come equipped with a "kill switch" capable of shutting them off in an emergency, it does call for a California AI panel to form and study the possibility. If implemented, an emergency shutoff option would theoretically allow operators to disable a rogue model as a safety precaution. Newsom's order also requires the panel to consider the idea of installing independent monitors within AI labs for the purpose of conducting regular audits. In a statement announcing the signing, Newsom underscored the urgency of the issue at hand. "We're not waiting to act," Newsom said. "We're going to speed up our work on substantial and responsible AI oversight before it's too late." Newsom's actions follow a headline-filled month for AI's biggest players. Ten days ago, as SFist previously reported https://sfist.com/2026/09/09/anthropic-researcher-says-theres-10-chance-of-ai-killing-all-humans-within-10-years/ , Anthropic researcher Evan Hubinger resigned from his post while warning that AI had "a 10% chance to kill all humans" at some point in the future. By the following week, Anthropic CEO Dario Amodei had shared a new blog calling for a global slowdown https://sfist.com/2026/09/12/saturday-links-anthropic-ceo-calls-for-ai-slowdown-amid-escalating-worries/ on AI development. It was quickly co-signed by OpenAI CEO Sam Altman and X owner Elon Musk. Earlier this week, SFist detailed https://sfist.com/2026/09/15/stop-the-ai-race-march-planned-for-thursday/ plans for a "Stop the AI Race" march that took place on Thursday morning to coincide with the final day of Dreamforce. SFist also reported https://sfist.com/2026/09/16/you-know-who-doesnt-want-the-ai-race-to-slow-down-for-his-company-mark-zuckerberg/ on one voice missing from the unified front advocating for an AI slowdown: Meta CEO Mark Zuckerberg. Related : You Know Who Doesn't Want the AI Race to Slow Down For His Company? Mark Zuckerberg https://sfist.com/2026/09/16/you-know-who-doesnt-want-the-ai-race-to-slow-down-for-his-company-mark-zuckerberg/