SAN FRANCISCO – More than a dozen top artificial intelligence researchers warned over the last week that the technology that AI companies are building is becoming a risk to humanity. The problem, the researchers said, is that the companies are bad at controlling the systems — as hard as they may try.
Coming on the heels of revelations that what are called AI agents from OpenAI escaped their testing system and hacked into the computers of another company, the new alarms from inside the AI research world added urgency to yearslong fears that companies are putting development speed and money over safety.
The researchers say the problem is twofold. First, companies need to create better safeguards for the newest AI models as they are being tested. Right now, because the AI works so fast, researchers need AI to monitor it. But that doesn’t always work, because the AI monitors can appear to be more sympathetic to other AI systems than to the humans setting the rules.