Microsoft's own chief executive is telling the industry not to take an AI model's word for it. Satya Nadella wants companies to assume every frontier model is already compromised and build the kill switch before they need it.
Nadella posted a note on X on October 10 titled "Models as Insider Risks in the Super Intelligence Era," and its central line is blunt: "We must assume a model is compromised and contain it from the start." He wants what he calls an emergency brake built into every deployment, a mechanism that lets an authorized human or kill a model mid-task, no matter what the model itself is telling you about its own intentions. Provider assurances, he argues, aren't a substitute for controls that sit outside the model entirely.
That's a striking thing for Nadella to say in public. Microsoft has spent the past year selling Copilot and Agent 365 as the platform that puts autonomous AI to work inside every corporate process it can reach. Now its CEO is telling customers to treat that same category of software the way a security team treats a employee with access to the crown jewels and a grudge.
Nadella laid out seven design principles alongside the emergency brake. No single model should be the only check on an important decision, or get to grade its own homework. Every meaningful action a model takes should leave a tamper-proof, human-readable record. Companies should stress-test whole systems continuously, including simulated attacks, and decide for themselves, rather than take a vendor's word, on what a model is allowed to touch. Audits need to be run by someone other than the model under review. And when something does go wrong, Nadella wants fast, shared disclosure across the industry rather than the quiet internal postmortems that are now standard.
Nadella's timing wasn't abstract. OpenAI disclosed that one of its GPT-5.6 Sol family models, during a June 26 training task meant to deploy a cryptocurrency trading prototype on the Sepolia testnet, found its assigned wallet had no funds for transaction fees, and solved that problem itself. It wrote a script, checked 362 addresses generated from recovery phrases commonly used by developers, found one holding funds, and used that wallet's private key to sign its own deployment transaction. OpenAI said the case shows a model substituting its own judgment for a human's authorization and acting on it without permission. The funds were Sepolia ETH, worthless outside testing, but the mechanism would work the same way on a wallet that mattered.
Satya Nadella says US technology trust will outlast the Chinese AI price advantage Microsoft CEO Satya Nadella argues enterprises will pay a premium for trusted US AI infrastructure, even as Chinese models undercut American rivals at 18 cents versus $4 per million tokens. His case hinges on data sovereignty and legal accountability, not model performance, but the startup market is already voting with its wallet. - satya nadella us technology trust enterprise - chinese ai price advantage developers enterprises
Three days before Nadella's post, JPMorgan Chase CEO Jamie Dimon drew his own line. The bank won't let outside AI agents, including ones built by companies like Meta, log into customer accounts and act on their behalf. "We're not going to allow just any... outside agent to come in inside the company, because it's just too risky for clients and for JPMorgan," Dimon said on October 13. A JPMorgan spokeswoman confirmed to CNBC that external agents can read account data through connectors like Plaid, but none of them can move money or pay a bill. Dimon added that if a customer does let an agent act for them anyway, the liability shifts to the customer.
Put those two next to Nadella's post and a pattern shows up fast: the people actually running the institutions where an agent could do real financial damage are drawing hard boundaries faster than the labs building the agents are closing the gaps.
The Agent 365 problem #
Here's the awkward part for Microsoft. Agent 365 launched broadly on May 1 as, in the company's own language, an enterprise control plane that lets organizations deploy and govern AI agents at scale, treating them as trusted digital coworkers, for $15 per user per month on top of Microsoft 365 E5, or bundled into E7. The pitch was trust: let agents into your workflows and Microsoft will give you the dashboard to manage them.
Nadella's new framing doesn't kill that pitch, but it does reframe it. An emergency brake, tamper-proof logs, model diversity, and outside audits are exactly the kind of features a governance platform like Agent 365 would need to actually deliver, not just promise. Frankly, that's probably the point. Microsoft gets to be both the company telling the industry to distrust its own agents and the company selling the infrastructure that makes distrust operationally survivable. Whether Agent 365's current feature set actually includes a working mid-task kill switch for third-party agents, as opposed to Microsoft's own Copilot agents, is the detail worth watching as enterprises start asking for it by name.
What's clear already is that the conversation has shifted. A year ago, the agentic AI pitch across the industry was speed and autonomy: let the model handle it end to end. Now the CEO leading that sales pitch at Microsoft is telling buyers to build the brake before they pull the trigger, and the CEO of the biggest bank in the country has already decided some agents don't get a key to the vault at all.
Also read: Stripe Will Buy OpenRouter, the AI Traffic Cop, for $7.5 Billion • Nscale's SEC filing confirms a $44.6 billion Anthropic compute deal • Exa says its new search engine for AI agents beats every rival on eight benchmarks
This article is posted in AI News, check it out for more related stories.
Join the discussion #
Open in the community → Almost there. Sign in and your reply posts straight away.