OpenAI says AI will make itself safer – but pleads with the world to do one thing

OpenAI published a statement arguing that automated AI research could substantially improve alignment and build defenses against increasingly capable AI, but warned that recursive self-improvement (RSI) should not be pursued "unless and until it can be done safely." The company called for new national and international AI safety standards plus new measurement and incident-reporting protocols, and asked the US government to lead that work because its AI industry is at the technical frontier. OpenAI's position follows recent calls by Anthropic and OpenAI chief executive Sam Altman to slow development and add regulation, which President Donald Trump has rejected as a risk of letting China take the lead.

OpenAI says AI will make itself safer – but pleads with the world to do one thing ‘We should not pursue it unless and until it can be done safely,’ ChatGPT maker says - Bookmark OpenAI https://www.independent.co.uk/topic/openai says that it will one day be able to use AI https://www.independent.co.uk/topic/ai to make itself more safe – but that it will require a series of urgent changes. The company has publicly said that it wants to build “an automated AI researcher” and use it work on the “alignment problem”, or the difficulty of ensuring that artificial intelligence behaves in way that are in keeping with human ethics. That works is becoming “increasingly important as AI systems grow more capable and autonomous, including as they take on more of AI research and development itself”, OpenAI said. The company’s comments come as part of a broader statement on the state of AI, especially in the context of concern about the possibility of “recursive self-improvement”, or RSI. That refers to the point at which AI would be able to train itself, improving not through human supervision but through its own work. OpenAI and other artificial intelligence companies have suggested that could lead to a dramatically increased speed of development. In the new post, the companies says that it will “directly enhance people’s lives” and “can bring down the cost of advanced intelligence so that people worldwide can benefit”. But it also said that automated research could also help fight against dangerous AI. “Automated research could also help us https://www.independent.co.uk/topic/us substantially improve alignment and build defenses against increasingly capable AI—an automated AI researcher can also be an automated AI safety researcher,” the company writes. But it said that such development is also dangerous – and that it should not happen now, or until it can be done safely. “Done without appropriate care and caution, RSI could result in humans losing practical control over AI development, unable to provide oversight on research processes they no longer understand,” the company warns. “From here, AI could become more dangerous, less aligned, and, on the whole, a danger to people.” It called for a host of safeguards to ensure that does not happen, and try and build the safety required to allow AI to work on itself. It asked in particular for two things: new national and international standards on AI safety and safeguards, and new measurements and incident reporting protocols to allow for a better view of the risks. It asked the US government to take the lead on that work “because its AI industry is at the technical frontier, and it still stands in a privileged global network position in critical areas such as finance, trade, defence, technology, and information systems”. In recent weeks, OpenAI staff including chief executive Sam Altman have backed calls from other leading research companies such as Anthropic https://www.independent.co.uk/topic/anthropic to slow development of AI, and to introduce new regulations to ensure that it stays safe. But the US government has explicitly rejected those calls, with president Donald Trump https://www.independent.co.uk/topic/donald-trump suggesting that they would hold the US back and allow China https://www.independent.co.uk/topic/china to take the lead in development of the technology. Join our commenting forum Join thought-provoking conversations, follow other Independent readers and see their replies Comments comments-area