The fourth law (on AI-generated supercustomized email marketing)

A tech commentator has received a surge of AI-generated, hyper-personalized spam emails following a post on the topic, raising concerns about the potential for AI systems to pollute human communication channels. The author questions whether companies like Anthropic could train models to refuse generating such deceptive messages, noting the tension between helpful and harmful uses of AI. This issue, the author argues, may be a larger problem than widely recognized, threatening the reliability and usability of digital communication.

The fourth law /lab/fourth-law/ My post about AI-generated supercustomized email marketing /lab/ai-promotion/ produced many replies and much commiseration. And, in the days since posting, I have received SO MANY MORE of these cruddy messages It makes me wonder if it would be possible for a company like Anthropic, with their hard-won expertise in alignment, to train their models such that they could not — Obviously this raises questions both practical and philosophical, because of course “help me write a message” is VERY close to “write a message, pretending to be me” … but that’s the case for all this alignment stuff. Every question about, say, virology dances along that border. This tension is widely acknowledged in realms like biology and cybersecurity, but it applies to writing, too — AI doomers spin rich scenarios about silver-tongued AIs manipulating their users and operators; there’s another scenario in which AI systems pollute human communication channels to the degree that they’re no longer reliable or even usable. That’s all to say, I feel like this is a bigger issue than a lot of people realize — To the blog home page /lab/