The Dangerous Ideology Behind the AI Warnings

Palantir chief technology officer and executive vice president Shyam Sankar argues in a Free Press Big Read that the AI safety warnings from researchers and executives are driven by the ideology of effective altruism and are actually an attempt to take control away from the American people. The piece cites the September 8 X post by former Anthropic software engineer Jacob Coxon, who wrote that "[t]he people building AI earnestly believe that it could kill us all by the end of the decade," and a current frontier-lab employee who put the odds at greater than 10 percent, warnings that prompted calls for AI safety regulation from the leaders of almost every major AI company. Sankar ties the movement to effective altruism, the philosophy crypto entrepreneur Sam Bankman-Fried cited as his motivation before he stole billions from FTX customers and was sentenced to 25 years in prison.

In a world with little agreement, there is one thing that unites almost everyone: AI is the most transformative technology of our time. What we don’t agree on is what to do about it. Over the past few days we’ve aired a range of perspectives, from Niall Ferguson https://www.thefp.com/p/niall-ferguson-aipocalypse-now-china-trump-pandemic-financial-collapse and Daniel Kokotajlo https://www.thefp.com/p/yes-ai-might-really-kill-us-all on the rising risk of AI catastrophe to Tyler Cowen https://www.thefp.com/p/tyler-cowen-ai-slowdown-computing-anthropic-openai on why efforts to slow AI won’t necessarily impede progress. But there’s another view of this issue—and especially the developments of the past week—that argues that the warnings from AI researchers and leaders should not be taken at face value. Shyam Sankar, chief technology officer and executive vice president at Palantir, is an influential member of this camp. In today’s Big Read, he lays out what he sees as the destructive ideology that lies behind calls for safety regulation—and why he believes what looks like a responsible plea to slow progress is actually about taking control out of the hands of the American people. —The Editors If you turned on cable news last week, you might have seen Jacob Coxon, a software engineer who resigned from Anthropic and immediately embarked on a media tour. It all began on September 8, when Coxon warned on X https://x.com/hilbertspaess/status/2097476203863224394 that “ t he people building AI earnestly believe that it could kill us all by the end of the decade.” A current employee at a frontier lab chimed in https://x.com/EvanHub/status/2097497037956891126 to confirm that, yes, he and many of his co-workers do believe there is a significant chance AI will kill us all. The odds, he said, are perhaps greater than 10 percent. These vague and apocalyptic warnings caused a firestorm online, including calls for AI safety regulations from the leaders of almost every major AI company. But there is reason for skepticism. The warnings from Coxon and others are infused with an ideology called effective altruism EA . The term will be familiar to some: A few years ago, it was the idea that crypto entrepreneur Sam Bankman-Fried said motivated him. SBF, you may remember, was supposed to be the good billionaire https://80000hours.org/stories/sam-bankman-fried/ . He was to make a fortune with his company, FTX, and give it away to protect humanity’s future, especially from big, existential risks. Instead, he stole billions from his customers and was sentenced to 25 years in prison https://www.justice.gov/usao-sdny/pr/samuel-bankman-fried-sentenced-25-years-prison .