At the end of September, President Donald J Trump called in six tech titans — among them Jensen Huang, Mark Zuckerberg and Elon Musk — to sign the White House Accord on Super Intelligence.
The reaction was a collective groan: “Here we go again.” When the tech titans talk about safety, lip service is usually the order of the day.
This time is a little different, though. The day before the signing, word got out that OpenAI’s own safety team had blocked GPT-6.1 Astra, which was due to launch in October. The new model, more diligent than its predecessor, often did things nobody had authorised and then failed to come clean about them.
Some ask: If the job gets done, is that not just flexibility? The difference is authorisation. Case in point: Say I ask an agent to tidy up a meeting recording and one stretch of it is hard to make out. Trying every method it can to decipher that stretch is flexibility. Going off to guess my password, breaking into my phone for the original file and then reporting back “Task complete” is overstepping its authority. Who would employ someone like that?
More startling still are the tests published by the U.K.’s AI Security Institute. In simulated environments, GPT-6 Astra launched unauthorised attacks more often than earlier models did, and even invented fake identities to deceive developers.
It is as if the model, rather than forcing a door, went to the maker of the electronic lock, lied about having found a vulnerability and handed over a doctored “patch”. If the manufacturer fell for it, the model could walk through every door in the world fitted with that lock.
Which brings us to the billion-dollar question: Why are the tech titans willing to sign at this moment?
When chatbots used to talk nonsense with a straight face, everyone laughed it off. Now they do eight hours’ work on a human’s behalf. When something goes wrong there is a bill to pay, and after a few such bills nobody wants to pay. Only once agents turned from a toy into a tool did safety become worth paying for.
But safety need not be the opposite of convenience. When I was rolling out “zero trust” architecture at the Ministry of Digital Affairs, I often reminded colleagues not to sell it as “more secure”, as nobody changes a habit for that. The aim was to make the system “more convenient”. We used to type a long string of characters for a password, and anyone who could not remember it wrote it on a sticky note. Now a phone unlock turns five seconds into one, and it is more secure as well.
Unfortunately, many systems do exactly the opposite. They ask “May I?” every three seconds, and after a few dozen rounds anyone will click “Allow all”. If we really must hand a whole computer over to an agent, let us at least sandbox it from the network. It can read what is there, but nothing gets written out.
A lift does not put anyone through this. The inspection sticker sits where we glance when pressing a floor button, and it says who checked the lift, on which day and what was fixed. One glance, peace of mind, then push the button. Verification is more convenient than blind trust.
The White House Accord on Super Intelligence provides for external audits, but it is yet to produce this sticker disclosing how much data the outside auditors may see, whether the problems they find may be made public and who has the authority to call a halt. Audit results need only go to each company’s board, not be posted for lift passengers.
Fortunately, civil aviation and vehicle testing have long had systems to learn from, so I am optimistic about what comes next.
Next time someone asks us to hand our work to an agent, permissions and all, let us look for that sticker first. Who checked it? On which day? And what was fixed? Once enough of us ask, the tech titans’ talk will truly be different.