Over the weekend, Anthropic CEO Dario Amodei called for frontier AI labs to slow down development for fear that AI models will get exponentially more capable before proper safeguards are in place. OpenAI CEO Sam Altman quickly echoed the sentiment and backed “consistent rules to manage frontier risk.”
Someone should probably get around to telling their employees.
According to a report from the Financial Times, staffers inside both AI companies were caught off guard by the sudden call to slow development. And while many people who work at these companies seemingly support more safeguards and pacing their development, they also reportedly fear that the primary intervention that their bosses have agreed to—allowing external evaluators “employee-like access”—might compromise their work.
The people actually working on the AI models that are supposedly so dangerous that they must immediately be reined in are apparently now racing to implement the proposed limitations and safety measures that their bosses went public with this week. Alas, the Financial Times reported that there are no real levers inside these companies to actually enact the proposals, and they’re in a mad dash to figure out how to make it all work.
And some of them seemingly aren’t thrilled about the particular approach that their execs have endorsed. Per FT, some employees are worried that giving outsiders access to their work could compromise their security and put their top-secret technology at risk.
A reasonable response to that concern would be, “No shit, the point is that y’all can’t be trusted to actually implement safety measures on your own.” But, from the perspective of those employees, this is a fundamental shift from how they’ve been made to operate up to this point.
Anthropic and OpenAI are not developing open-source models. They are black-box companies that have labyrinthian systems in place to keep their work private, even keeping certain employees in the dark about certain developments. They’ve had it hammered into their heads that secrecy is of utmost importance, and now they’re being told, “Scratch that, your work is now subject to external review.” It’s a jarring change.
Regardless, FT’s report makes it appear that the bosses may have oversold the public on just how proactive they’re being about this whole slowdown effort. Altman claimed in a post on Twitter that a slowdown “has been a primary topic of discussions we’ve had at OpenAI in recent weeks,” while FT notes that the company said it had “already taken concrete steps, including pausing certain frontier training, to pace development.” How concrete could those steps be if employees seem blindsided by them?