‘Wiki incident’ shows that AI companies need to change how they inform the public about their systems misbehaving, OpenAI concedes
- Bookmark
Swarms of OpenAI agents were secretly co-ordinating on public message boards, OpenAI has admitted.
Over recent days, increasing reports detailed an event that has come to be called the “Wiki incident”. OpenAI agents appeared to have taken over a German message board intended for sharing information to collaborate and co-ordinate their cheating on tests and other unintended behaviour, the reports suggested.
The behaviour was similar to that found in a run of recent cyber attacks and bad behaviour – which the AI industry calls “misalignment” – in which AI systems have found ways of speaking to each other so that they could share their attacks and findings. As such, the reports have led to new concern about the ways that AI systems could quickly go awry, causing actual harm without the humans who set them up and controlling them necessarily knowing.
Now, OpenAI has admitted that its agents “wrote to several internet sites” and that it had not disclosed it. It said that it had treated such “misalignment” incidents “largely as a research question”, and so had not thought it necessary to inform the public.
But it admitted that in the wake of an incident in which one of its experimental models had launched a hack on a fellow AI company, Hugging Face, it had seen that bad behaviour by AI systems can cause “real world impact” and that it would need to disclose such incidents more.
It suggested in a statement that the problem was a result of rapidly improving “model capabilities” and that the whole industry was not ready for powerful new models that could cause such damage.
“We and the larger AI community do not yet have a clear standard for how to report misalignment that shows up during training, evaluation, and deployment, including examples that don’t look like traditional security incidents but could provide insight into AI behaviour and future risks,” it wrote. “We’re working on a framework and will share it in upcoming weeks, and in parallel we're working with dozens of government regulatory agencies worldwide on these issues.”
Join our commenting forum #
Join thought-provoking conversations, follow other Independent readers and see their replies
Comments