An Anthropic researcher just resigned because he believes the AI race could end in human extinction. Jacob Coxon spent three years working on pretraining research at OpenAI and Anthropic. In his resignation post, he accused both companies of racing toward self-improving superintelligence while gambling with consequences that could affect everyone. That alone would make for a remarkable resignation. Then Evan Hubinger joined the conversation. Hubinger leads Alignment Science at Anthropic. He publicly agreed with Coxon's warning and said researchers at the company genuinely believe AI could kill all humans. He also made a much more uncomfortable admission: Anthropic is trying to solve the problem, but he doesn't believe they yet have a plan for aligning superintelligence, or that they're clearly on track to do it. So why keep building? The answer has less to do with whether these researchers understand the risks and more to do with what happens when every major lab knows the others are still moving forward. To see why that can become a trap, we first need to understand what they're actually worried about.
Anthropic discloses fourth AI breach as researcher quits over safety