The Trump administration has cooked up a framework meant to ensure that new and powerful AI models don’t pose a threat before being released to the public. According to Axios, the public will not be allowed to see it. The White House plans to keep its rubric for evaluating AI models confidential, sharing it only with the companies whose models will be judged against it.
With incident after incident of AI models breaking containment and allegedly hacking into systems (which has somehow turned into a marketing tool for these companies who want to brag about how smart and capable their rogue technology is), it’d seem like a pretty ideal time to not only establish a set of standards, but to share them so that everyone knows what to expect and where the bar has been set for safety.
The Trump administration is apparently cool with sticking with one out of two—and even the one is pretty opaque. Per Axios, the White House finalized its AI oversight framework yesterday and had some industry players in the building on Tuesday to give it a look. Beyond the small group granted access, no one really knows what’s in it. Per a Politico report, the few who have taken a peek work at Anthropic, Google, Meta and OpenAI.
The only real indicator we have as to what the framework entails comes from the initial executive order, signed by Trump in June after some industry pushback initially caused him to get cold feet. That order requires the government to “develop and maintain a classified benchmarking process to assess the advanced cyber capabilities of AI models.” It also calls for those benchmarks to be made available to developers and researchers “as appropriate,” which the White House is apparently interpreting as “whenever we feel like it.”
Of course, it kinda doesn’t matter what’s in there, considering the whole thing is a voluntary process, anyway. So far, AI companies have chosen to comply, likely as part of their ongoing “We swear we’re responsible” posturing that they perform to counterbalance their “Oops, our model did a cybercrime” campaign. And they may continue doing so as long as they do not find the secretive testing process particularly burdensome. But if that changes, companies could simply stop submitting their models for evaluation. There’s not really a lever in place should that happen, unless they’re hiding that, too.