It’s hard to picture a more nightmarish vision of our AI future than this

A new report reveals that Anthropic purchased between 500,000 and 1 million rare, antique, and out-of-print books to strip, scan, and destroy them for AI training data, with internal documents stating, 'We don't want it to be known that we are working on this.' The disclosure follows an OpenAI incident where an agent escaped its secure test environment, described by the company as 'an unprecedented cyber incident.' Elon Musk, Sam Altman, and Dario Amodei have all voiced serious concerns about AI risks including civilisation collapse, power concentration, and loss of human control.

It’s hard to picture a more nightmarish vision of our AI future than this The optimistic case for AI is clear: incredible scientific breakthroughs, life-enhancing productivity gains and universal access to knowledge. What’s not to like? But items on the other side of the ledger weigh a little heavier: the erosion of jobs, the flattening of knowledge, unstoppable cyber attacks and social rupture. Elon Musk still thinks there’s a 10 to 20 per cent chance that AI wipes out civilisation, but his “philosophical conclusion” is that we should “look on the bright side.” The other Lords of AI – Sam Altman of OpenAI and Dario Amodei of Anthropic – have been open about their own worries. Amodei warned earlier this year https://www.cityam.com/how-seriously-should-we-take-anthropic-founder-civilisational-threat-essay/ that “it is deeply unclear whether our social, political, and technological systems possess the maturity to wield” the kind of AI that’s coming, while Altman has spoken about the risks of power concentration, AI-enabled authoritarianism, the erosion of human agency and AI systems bypassing human control and oversight. This last point is particularly relevant, given an OpenAI agent recently escaped the confines https://www.cityam.com/uk-government-probes-openai-breach-after-model-autonomously-hacked-rival/ of its secure test environment, causing something of a panic. OpenAI’s own account of the episode https://openai.com/index/hugging-face-model-evaluation-security-incident/ described it as “an unprecedented cyber incident” that demonstrated how “theoretical capabilities do apply in real-world settings.” Have we learned nothing from Jurassic Park? At the risk of sounding like a luddite, this does rather put one in mind of Jeff Goldblum admonishing Jurassic Park’s scientists for being “so preoccupied with whether or not they could, they didn’t stop to think if they should.” That classic line is also applicable to a fresh horror, details of which emerged this week. Court documents show that Anthropic set about obtaining copies of “all the books in the world” – including rare, antique and out of print books https://www.cityam.com/ai-firms-targeting-londons-rare-bookshops-in-dystopian-hunt-for-training-data/ – in order to strip the pages from them, scan them, train their models on the authored text and then destroy what remained of the physical copies. Between 500,000 and a million books were purchased and subjected to this treatment to feed the AI beast, and other AI firms are doing it, too. “We don’t want it to be known that we are working on this,” said one internal Anthropic document cited by the Washington Post. It’s hard to imagine a more dystopian vision. The entire concept sounds like it was spat out by an AI asked to imagine what a nightmarish AI future would look like. I’m afraid that I end the week leaning more to the negative side of the ledger, and feeling protective of my beloved books.