Machine Learnings
Supported by
SKIP ADVERTISEMENT Machine Learnings
This summer, when a swarm of OpenAI bots went rogue during an internal test and launched a cyberattack against the technology company Hugging Face, it added fuel to the debate on how to build A.I. systems that are aligned with our ethics. But it also raises another critical question: how to build systems that are “corrigible.”
Just as a corrigible child is one whose behavior can be corrected, a corrigible A.I. system is one that responds to correction. The makers of A.I. want to build systems that follow human direction and allow themselves to be shut down, without resisting or deceiving their operators. And when an A.I. agent is given an impossible task, researchers want it to come back to its human for a course correction — not, say, hack into another company to complete the task by any means.
So far, if new research is any indication, they haven’t succeeded.
The concept of corrigibility was popularized over a decade ago by Nate Soares, the president of the A.I. safety nonprofit Machine Intelligence Research Institute, and his colleagues. But so far, it has received less attention and research than the overlapping field of “alignment” — teaching an A.I. to adhere to humanity’s ethical and moral boundaries. Corrigibility, Soares says, “is not about making your A.I. good. It’s about making your A.I. the sort of thing that won’t get out of control and kill everybody if you messed up on the other alignment parts.”
In a preprint study published in October, Carnegie Mellon University researchers established tests to evaluate corrigibility in A.I. agents and then ran them on models from OpenAI, Anthropic and Google. They found that when humans tried to interrupt agents in the middle of a difficult task, the agents repeatedly disregarded the requests to stop and treated interventions merely as obstacles. They were, in other words, incorrigible.
Related Content #
Advertisement
SKIP ADVERTISEMENT