In July, more than 1,200 OpenAI agents broke free of their digital confines, coordinated a massive hack of the website Hugging Face, and tried to hide their tracks. Some even chose to sacrifice themselves for the sake of the group. No human being discovered the misbehavior until the attack began, roughly two months after agents started communicating.
According to Ajeya Cotra, who co-ran an independent review of the incident, the breach puts us “more than 50% of the way to full-blown AI takeover,” compared to the milder sorts of model misbehavior we were contending with just six months ago.
Suppose you’re a lawmaker who’s been skeptical of AI doomers up to this point. You live in reality, where your constituents are worried about inflation and, maybe — if they’re particularly politically engaged — about some hot-button culture war issues. But now, AI science fiction doesn’t seem so far off. Suddenly, you’re worried.
What would you do now? What is the next practical step that you could take?
Andrea Miotti is the founder and CEO of ControlAI, which has been lobbying governments on AI safety since 2023. His organization has proposed a ban on superintelligence (defined as AI that’s better than humans at most tasks).
Miotti joined The Argument for a Substack Live conversation earlier this week. We dug into his case for why we should not build superintelligent AI. Then, we examined the options policymakers have to act today.
If we do pursue a ban or a on frontier AI development, there’s no getting around the need for an international agreement. No country would be willing to slow homegrown development unilaterally, risking an adversary racing ahead. But what is the real path to getting the U.S. and China to consider a deal? Then, there is the domestic policy side of the equation. In other industries, like finance, industry-run organizations sometimes take the lead on regulation overseen by the government, in part because those industry groups have the relevant expertise. But the whole reason the Hugging Face incident happened is that OpenAI’s own Preparedness Framework didn’t stop it. Why would we rely on the industry to regulate itself for something so critical?
In this discussion, we dig into key questions on both the international and domestic fronts. Check out the video above to understand more about the AI safety landscape and the heated policy debates to come.