A week ago, three of the industry's biggest names agreed AI needed to pump the brakes. This week, the two governments that would actually have to enforce that both said no — and the fallout says more about where AI is headed than the original call for caution did.
The story starts with Dario Amodei, Sam Altman, and Elon Musk publicly endorsing a plan to slow frontier AI development, a rare moment of alignment among rivals. It didn't last. President Trump used Truth Social to call warnings about AI "taking over the World, destroying Humanity" a "HOAX," comparing it directly to climate change, and phoned Nvidia CEO Jensen Huang live on stage at a tech summit just to hear him agree that a slowdown "is not going to happen." China's government was no warmer to the idea: its Foreign Ministry dismissed the plan as against everyone's interests, and state media called it a "Cold War playbook" dressed up as safety concern. Layer in skepticism from a former White House AI czar, who argued labs could simply slow down without needing new rules, and speculation from at least one investor that the sudden three-way agreement had more to do with an unnamed spooked incident than genuine principle, and the neat consensus from a week ago looks a lot shakier. The debate isn't staying theoretical, either — Wall Street is already voting with its money, sending cybersecurity stocks up double digits this week on the bet that AI-capable hacking is the more immediate risk, while at least three major companies are reportedly restricting how their staff use one leading AI assistant on sensitive work over data-retention concerns.
Amid all that noise, actual products kept shipping. Apple finally began rolling out its rebuilt Siri, an assistant that can read what's on your screen and act across apps like WhatsApp and Audible — built in part on Google's models and notably absent from the EU and China at launch. It's a genuinely significant upgrade for the world's most-used phone, even if it's arriving well behind the AI chatbots it's trying to catch up with. Elsewhere, the range of what companies are betting on kept widening: a former OpenAI researcher launched a deterministic "judgment call" AI that claims to be nearly free to run and effectively immune to hallucination; Salesforce rolled out its own in-house reasoning model trained entirely on synthetic data to keep customer information away from outside AI providers; a Chinese lab released an open-weight research model it claims rivals the best from the U.S. and China; and Meta's personal AI agent climbed to the number-two spot on the App Store, trailing only ChatGPT.
The safety conversation also got more concrete this week, in two different directions. Microsoft's AI division published a detailed draft rulebook — dubbed a "Humanist AI" code of conduct — that would require its models to accept being shut off, forbid them from editing their own reasoning logs or reasoning in unreadable shorthand, and explicitly deny them anything resembling rights or personhood. It's open for public comment, though for now it only binds Microsoft's own systems. Meanwhile, continued fallout from a major AI misuse report detailed just how far bad actors have already pushed these tools in the real world: a group in Yemen reportedly used an AI coding assistant to help develop guidance software for weapons, and a separate operation ran a commercial disinformation-for-hire scheme across dozens of fabricated websites spanning six continents. The company behind the report says it caught and shut down every case described and shared its findings with authorities — but the specifics are a reminder that AI's risks aren't only the hypothetical, superintelligent kind people are debating over Truth Social posts.
And in a preview of where the "improve itself" conversation goes next, a group of more than thirty Chinese researchers published a roadmap sorting AI systems into five levels of self-improvement, from following human-designed upgrades to eventually redesigning their own improvement process from scratch. Surveying nearly 500 existing research papers, they found the field is overwhelmingly still stuck at the earliest levels — under 6% reach the top tier — but flagged coding as the domain most likely to get there first, since its fixes can be tested instantly. That top level is exactly the stage Western labs have named as a top safety risk in their own frameworks; this group treated it as a milestone worth racing toward instead.