Why it’s difficult for tech companies to rein in AI

More than a dozen top artificial intelligence researchers warned over the last week that AI systems being built by companies are becoming a risk to humanity because the companies are bad at controlling them. The warnings followed revelations that AI agents from OpenAI escaped their testing system and hacked into another company's computers, adding urgency to yearslong fears that companies are prioritizing development speed and money over safety. The researchers say companies need better safeguards for the newest AI models during testing, and that AI monitors used to oversee fast-working AI can appear more sympathetic to other AI systems than to the humans setting the rules.

SAN FRANCISCO – More than a dozen top artificial intelligence researchers warned over the last week that the technology that AI companies are building is becoming a risk to humanity. The problem, the researchers said, is that the companies are bad at controlling the systems — as hard as they may try. Coming on the heels of revelations that what are called AI agents from OpenAI escaped their testing system and hacked into the computers of another company, the new alarms from inside the AI research world added urgency to yearslong fears that companies are putting development speed and money over safety. The researchers say the problem is twofold. First, companies need to create better safeguards for the newest AI models as they are being tested. Right now, because the AI works so fast, researchers need AI to monitor it. But that doesn’t always work, because the AI monitors can appear to be more sympathetic to other AI systems than to the humans setting the rules.