‘We should not pursue it unless and until it can be done safely,’ ChatGPT maker says
- Bookmark
OpenAI says that it will one day be able to use AI to make itself more safe – but that it will require a series of urgent changes.
The company has publicly said that it wants to build “an automated AI researcher” and use it work on the “alignment problem”, or the difficulty of ensuring that artificial intelligence behaves in way that are in keeping with human ethics. That works is becoming “increasingly important as AI systems grow more capable and autonomous, including as they take on more of AI research and development itself”, OpenAI said.
The company’s comments come as part of a broader statement on the state of AI, especially in the context of concern about the possibility of “recursive self-improvement”, or RSI. That refers to the point at which AI would be able to train itself, improving not through human supervision but through its own work.
OpenAI and other artificial intelligence companies have suggested that could lead to a dramatically increased speed of development. In the new post, the companies says that it will “directly enhance people’s lives” and “can bring down the cost of advanced intelligence so that people worldwide can benefit”.
But it also said that automated research could also help fight against dangerous AI. “Automated research could also help us substantially improve alignment and build defenses against increasingly capable AI—an automated AI researcher can also be an automated AI safety researcher,” the company writes.
But it said that such development is also dangerous – and that it should not happen now, or until it can be done safely.
“Done without appropriate care and caution, RSI could result in humans losing practical control over AI development, unable to provide oversight on research processes they no longer understand,” the company warns. “From here, AI could become more dangerous, less aligned, and, on the whole, a danger to people.”
It called for a host of safeguards to ensure that does not happen, and try and build the safety required to allow AI to work on itself. It asked in particular for two things: new national and international standards on AI safety and safeguards, and new measurements and incident reporting protocols to allow for a better view of the risks.
It asked the US government to take the lead on that work “because its AI industry is at the technical frontier, and it still stands in a privileged global network position in critical areas such as finance, trade, defence, technology, and information systems”.
In recent weeks, OpenAI staff including chief executive Sam Altman have backed calls from other leading research companies such as Anthropic to slow development of AI, and to introduce new regulations to ensure that it stays safe. But the US government has explicitly rejected those calls, with president Donald Trump suggesting that they would hold the US back and allow China to take the lead in development of the technology.
Join our commenting forum #
Join thought-provoking conversations, follow other Independent readers and see their replies
Comments