Tech analyst Ben Thompson says requiring AI companies to be transparent about the content their models produce is missing the mark.
Thompson tore into the European Union AI Act, which Anthropic cited as the reason it will add watermarks to content in Claude-generated files and text.
In his widely read "Stratechery" newsletter, Thompson said such AI transparency requirements are both overburdensome and unlikely to be strong enough.
"I am sympathetic to the impetus behind watermarking: wouldn't it be better to know what is fake and what isn't?" he wrote. "What, though, is 'fake'? If an LLM states a true fact, and a human a fable, does it matter that the latter doesn't have a watermark?"
Anthropic's plan, which it announced earlier this week, has deeply divided the tech community. The AI giant said that Claude models launched after August 2 would support watermarking from the start and it is working to add such support to older AI models.
Thompson took particular issue with Anthropic adding a watermark even if Claude was used only for proofreading. The EU's regulation allows for an editing exception, but technically speaking, Thompson said there was no way for Anthropic to distinguish such usage.
"I'm glad I never developed the habit of copy-and-pasting proof-reading runs (I use LLMs for proof-reading, but manually make every individual change in my text editor), but it hardly seems fair that anyone who wants to fix their grammar runs the risk of having their original content labeled as AI; the same frustration applies to translation," he wrote.
OpenAI, which currently watermarks images and audio, has also pledged to expand the process to text output, though the exact details of their plan have not been released. OpenAI uses Google DeepMind's SynthID to watermark images and audio.
Regulators are turning to watermarks, traditionally seen in visual media, to help society discern when content is AI-generated. Unlike visual watermarks, the provenance markers for AI text are not perceptible to the naked eye, but Anthropic and other AI companies have pledged to release tools that will essentially serve as AI checkers.
Under the EU regulations, AI companies are required to ensure "that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated." Anthropic, OpenAI, Google, Meta, and Microsoft have all signed the relevant section of the regulation. Notably, SpaceX has not. The transparency provision went into effect on August 2.
Anthropic did not indicate that it would only enforce watermarking in Europe. It remains to be seen whether other companies will extend watermarking to users everywhere.
Overall, Thompson said watermarking unfairly credits AI with ideation when such models are "(at least for now) a tool that is wielded by humans."
"From this perspective, to insist on watermarking is no different than insisting that a ballpoint pen advertise itself as the author, a concept that is clearly absurd," he wrote.