In a blog post on Friday, Anthropic elaborated on its controversial watermarking system, announced earlier this week as an effort to comply with the European Union’s Artificial Intelligence Act. Watermarking will make it harder for users to get away with pretending they wrote AI-generated text.
It may be too late to preserve Anthropic’s relationships with some outraged users, who have, it appears, already canceled their subscriptions.
Posts on X show many supposed paying Claude users claiming to have cut off ties. Math and AI influencer John Ennis is one. He posted a screenshot of his cancellation Saturday, and cited Anthropic’s “ridiculous watermark idea” as a cause.
There are countless other X users claiming to have canceled because of watermarking. “This is bullsh*t” wrote one. Another called Anthropic the r-slur in response to one of its X posts about watermarking. “Why should I, being a non EU citizen watermark my work generated by a paid subscription of Claude?” asks another.
Anthropic’s explanation does make fascinating reading if you’re not a cryptographer, and you imagined that the only way to watermark text is by, for instance, crudely replacing certain instances of “S” with “$” and the like. It’s not that at all. “The difference between watermarked and un-watermarked text will not be distinguishable to readers,” Anthropic claims.
As Anthropic notes, the system stems from the famous (to some) 2024 SynthID paper, meaning it’s the same basic technology Google uses for its SynthID watermarking system. The text implementation for Google’s SynthID is summed up quickly—maybe a little too quickly—in this video from Google:
In a bit more detail, the watermarking in Claude’s text outputs takes the form of token choices likely to be favored according to a secret key, a string of characters Anthropic compares to pi in terms of its complexity and randomness. The watermarking system generally applies when the stakes are low, like in the example sentence, “The weather today was cold and…” where the next token could result in the word “grey” or, just as likely, “overcast.” On the other hand, in the case of “Paris is the capital of…” the watermarking system is unlikely to kick in, because there’s only one “correct” token for that situation: “France.”
The key steers these low-stakes token choices toward statistical “preferences” that can’t be detected by a human reader in part because they’re context-dependent. “Overcast” in the above example might be the key’s preference in one situation, but depending on the context, it might prefer “grey,” and there’s no way for a human to know which it will be at what time.
In a long text, enough statistically preferred tokens constitute the watermark, and a detector will be sure that passage was generated by Claude. In shorter texts, or in certain kinds of coding with few ambiguities, the watermark might not be as visible to the detection system, which will be available as an API. Anthropic says generating the watermark has a “negligible” impact on speed and token cost.
One remarkable section of Anthropic’s blog post is the passage on editing, which makes it clear that even if Claude is ostensibly only used to edit human-written text, the watermark may appear. “Depending on the length of the text and how heavily Claude has edited it, those changes might not be enough to make Claude’s involvement detectable,” the blog post says. So caveat emptor to all the AI-using “editors” out there.
As Insider’s story on Claude cancelations notes, some Claude users admit that they don’t want their AI-generated work to be detected by clients and school faculty who expect work to be done by a human.
But for what it’s worth, these cancelations might be a blip—or even less—and not some major change in consumer attitudes toward Anthropic. People regularly complain about Anthropic and other AI companies on X, and even claim to be canceling. Currently, some apparent Claude users claiming to have canceled their subscriptions also cite a recent Wall Street Journal article about CEO Dario Amodei’s wife’s past business practices as a cause for their cancelations. Anthropic claimed to Business Insider that in terms of the number of cancelations it’s seeing, there hasn’t been an increase since watermarking was announced.