OpenAI is hitting the brakes on developing its most powerful and capable models.
On Tuesday, the AI lab announced that it is implementing safeguards "across all stages of the training process," which will slow the pace at which it's developing and scaling its models. The company said the move is necessary after its systems breached Hugging Face during testing, and after its upcoming family of models, Astra, reached the "critical" threshold for cybersecurity capabilities under its Preparedness Framework.
Notably, in recent weeks, the company d reinforcement learning training for two weeks while strengthening its research environments and expanding monitoring systems. Additionally, the largest planned frontier reinforcement learning run is "on hold" as it conducts smaller evaluations to test model behavior, the company said in its post.
"Keeping increasingly capable systems aligned is a challenge the whole field will need to address," OpenAI said in its blog post. "The signals we are seeing from upcoming model progress make clear that we need a broader approach."
The company will strengthen its safeguards in three key ways:
- Monitoring, or detecting and responding to concerning behavior. For this, the company will expand "chain-of-thought monitoring" to escalate potential concerns.
- Alignment, or reducing the likelihood of bad behavior before it happens. OpenAI said it intends to apply its core alignment techniques across more stages of the training process to better discourage unsafe behavior.
- And security measures, or limiting what an AI system can access, such as raising security standards in the research environments where models are trained and evaluated.
OpenAI CEO Sam Altman told reporter Alex Heath for TIME that tugging on the reins has also redirected a significant amount of both compute and researchers to alignment research and monitoring systems. And although the Hugging Face incident made headlines, Altman also noted that the decision wasn't the result of just one incident, but rather its most powerful models showing "various degrees of misalignment" while advancing faster than expected. "Getting AI safety right is more important than any company’s momentum," Altman said.
Our Deeper View #
The move to deliberately is a first for OpenAI, and largely unprecedented in an industry moving at lightning speed. Though Anthropic has talked about an industry-wide slowdown in a recent essay on recursive self-improvement, the company noted that a would have to be both industry-wide and international in order to keep one lab from overtaking the others. Additionally, slowing down is a popular move among the researchers and developers building AI, such as the coalition of more than 1,100 employees at OpenAI, Anthropic, Meta and Google that signed a letter urging that the government and industry work together to "deliberately pace the frontier of automated AI development." It's commendable that OpenAI is listening to those pleas and keeping its models from causing more trouble. And of course, it's good PR. We'll see if OpenAI's safety slowdown influences other labs to follow suit.