Don’t bring an AI detector to a deepfake fight: proving reality through multimodal provenance

Fake media detectors are losing the arms race against deepfakes, so a new approach is needed: distrust all images and videos by default unless they are cryptographically signed with multimodal provenance data. This method, combined with platform policies and legal liability for untagged generated content, could raise the cost of spoofing exponentially and curb the spread of fake media.

Epistemic status: confident on the framing, speculative on the implementation TL;DR:Fake media detectors are on the losing end of an arms race. Instead of trying to spot fakes, we need to distrust images and videos by default, unless we can prove they're real. Cryptographically-signed multimodal capture raises the cost of spoofing exponentially, making casual fakes prohibitively expensive for the general public. Combined with platform policy and legal liability for untagged generated content, this could solve most of the problem of widespread fake media. Text is easy to fake, which is rather inconvenient. Now photos and videos are becoming easy to fake too. That, however, poses a much bigger problem. When we read something supposedly factual, we know that the written words don’t represent reality, as much as someone's opinion, idea, or account of reality. Because of how easy text is to counterfeit, we know to operate in a default state of distrust. We don't believe the text itself, but its content because we trust its source. Our whole epistemology of written information functions within this realm, including news reporting standards, witness testimonies or libel law. LLMs make fake text cheaper and faster to produce at scale, and that’s a problem, but not a new category of problem. We know not to blindly trust text, so we already have partial defence mechanisms like cross-referencing sources or checking the author's credentials. We are used to questioning any suspicious quote or tweet we encounter because we know how easy it is for anyone to write down words and falsely attribute them to someone. Pictures, audio recordings, and videos, on the other hand, are a whole other kind of beast. Historically, we have used them to capture a direct snapshot of reality https://www.lesswrong.com/posts/6s3xABaXKPdFwA3FS/what-is-evidence , not someone's opinion or rendition of reality. Because of that, they occupy a special place in courtrooms where they can be considered as evidence. The same goes for insurance claims and all sorts of journalism. We believe images because they don't lie, or more precisely, because it is expensive and difficult to make them lie convincingly. At least it was. It has recently become easier, cheaper, and faster to generate fake media at scale, with generation becoming more realistic by the day. The contract of audio-visual media as a testimony of real life events is breaking down, and we have almost no infrastructure to replace it. Before getting to the technical argument, it is worth being precise about what is actually at risk, because the public discourse tends to focus on the most visible harms like viral propaganda and celebrity deepfakes while underweighting https://psyche.co/ideas/why-deepfakes-pose-less-of-a-threat-than-many-predict what I think is the more insidious damage. The bounded distrust https://www.astralcodexten.com/p/bounded-distrust thesis proposes the existence of a line that even fake news won't cross, while a refined version https://www.lesswrong.com/posts/6YYmkpumigAmh3efu/on-bounded-distrust of this argument by Zvi rather frames it with the cost of getting caught crossing that line. Under those assumptions, outrageous fakes would still bear the same cost no matter how good they are, the risk of getting caught is simply reduced. The real issue comes from the multiplication of low-cost 1 fakes, and what they entail. Our legal system relies on audiovisual media, as a security camera video of an assault or a phone picture of a thief can sometimes be the only evidence a victim can produce. As deepfake technology matures, that evidence becomes systematically deniable. Citron and Chesney call this the liar's dividend https://papers.ssrn.com/sol3/papers.cfm?abstract id=3213954 : the mere existence of deepfake technology gives bad actors a plausible basis https://www.theguardian.com/technology/2023/apr/27/elon-musks-statements-could-be-deepfakes-tesla-defence-lawyers-tell-court to dismiss authentic evidence as fabricated. A defendant who can point to the existence of deepfake technology and say "that video could have been generated" has introduced reasonable doubt, regardless of whether the video was actually manipulated. The more low-cost fakes are uncovered, the more people will lose faith in audiovisual media altogether. The damage to epistemic trust is not just people being misled by fakes, but how real evidence becomes unreliable/inadmissible as a whole. When your enemy builds a bigger cannon, you just get a bigger shield, right? The natural response to fake media is to want to build detectors. After all, if we could easily identify generated content, it wouldn't pose much of a problem. Generally speaking, there are two main ways to detect a fake: either you point out what’s wrong with its content unrealistic output, generation artifacts , or you check the presence of a generation fingerprint metadata or pixel markers added during the generation process . This is the logic behind watermarking schemes like DeepMind's SynthID https://deepmind.google/models/synthid/ , as well as a growing body of academic work https://arxiv.org/html/2403.17881v5 and countless tools https://quillbot.com/ai-content-detector for AI detection. Both approaches have flaws, but, more importantly, I believe that this is fundamentally the wrong angle to tackle this problem. The problem of detection is that it’s stuck in an arms race, and it’s going to lose. For humans, it is a game of diminishing tells. When diffusion models became widespread, people looked at generated hands to spot fakes. But hands got better. People then looked at local texture artifacts. And texture got better. Now people look at perspective https://gijn.org/resource/guide-detecting-ai-generated-content/ and lighting consistency, and it would be naive to think this tell won't suffer the same fate. For automated systems, the problem is more fundamental. Every time a detector is deployed, generators can be adversarially trained not to trip it this is essentially how GANs https://en.wikipedia.org/wiki/Generative adversarial network work . Automated techniques rely on subtle statistical tells, but they suffer from the same flaw as human tells. They impress because they somewhat work when they’re released, before being made obsolete within months. Each new generation of models eliminates the artifacts that gave the previous one away, and there is no reason to think this process will ever stop. There is unfortunately no absolute law of physics that mandates that generated content has to be distinguishable from real content, and that it is always possible to build a detector for it. For example, the question of whether the sentence “The cat jumped on the table” has been written by a human or an LLM is fundamentally impossible to answer because that sentence is “perfect”, i.e. indistinguishable from other real content. If we want to tackle fake content for good, we need to assume that every kind of media generation will eventually become perfect. Even though images, sounds and videos offer much greater expressive potential and therefore room for generation flaws than text, we need to work with the premise that we will reach a point where all detectors have become useless 2 . Perhaps the subtler danger of detection-first thinking is not that it fails, but that while it appears to work, it gives a false sense of safety and displaces the harder conversation about what a durable solution would actually look like. The second way to detect generated content is to add a watermark during the generation process, which comes with its own distinct issues. Watermarks embedded in metadata are trivial to strip most social media platforms do this automatically when you upload an image anyway while watermarks embedded in the signal itself can fail against adversarial image manipulation https://unmarker,%20a%202025%20paper%20from%20the%20university%20of%20waterloo%20published%20at%20ieee%20s&p,%20demonstrated%20a%20universal%20attack%20requiring%20no%20knowledge%20of%20the%20underlying%20watermarking%20scheme%20whatsoever/ . If a watermark can be cropped out, or doesn't survive being compressed, stretched, or deep fried, it is virtually useless 3 . More fundamentally, watermarking only tags content that comes from a system that applies the watermark. It does nothing about content generated by systems that do not, like most open-source models and every fine-tuned variant running on private infrastructure. To build a robust system for fighting fake media, we must adopt a security mindset https://www.lesswrong.com/posts/8gqrbnW758qjHFTrH/security-mindset-and-ordinary-paranoia and assume that the generated content will come from non-watermarked systems. The deeper issue with both detection and watermarking is that they are asking the wrong question. They ask: "is this fake?", which is becoming increasingly unanswerable. The right one to ask is: "can we prove that it’s real?" Instead of trying to detect fakes, we should focus on certifying what is real. After going back and forth on this problem for a while, I came to the conclusion that the solution had to involve encryption directly at the capture layer. We need a way to ensure that the content we see has been captured by a real device, and that it hasn't been edited since 4 . It turns out others had the same instinct: the This is the right direction conceptually but these approaches aren’t leveraged in a meaningful way by content platforms a certificate is useless if no one checks it , and remain largely monomodal, making them prone to spoofing. Cryptographically signing an RGB image proves that a particular camera produced a particular file. It does not prove that the certified camera was pointed at a real scene rather than a screen displaying a generated image. With that setup, called the analog hole https://arxiv.org/pdf/1909.00056 attack, you could easily create fake media with a valid provenance chain simply by capturing it with a certified device. To prevent such spoofing, I propose to extend the method to include cross-modal consistency. Instead of simply ensuring that a video has been taken with a certified device, we also check, for example, that the IMU's data orientation, acceleration recorded by the device matches what the video displays. As we add more sensors audio, GPS, lidar point cloud, brightness, etc. , the cost of spoofing increases exponentially, which significantly changes the threat model. Spoofing each independently is one thing, but spoofing all of them in a physically coherent way becomes extremely challenging. With such a system, the analog hole attack becomes much less practical. In addition to filming a screen displaying the video you want to falsify, you also need to somehow fool the lidar into seeing a 3D scene consistent with the video, all while moving the entire rig to match the acceleration and orientation of the video. Technical note : the consistency verification model itself can be learned rather than handcrafted, using a self-supervised approach trained on multimodal recordings. For example, record everything you can with a phone or other device, and then match all sensors with each other. On top of that, it is crucial to make raw sensor outputs human-inspectable, like with a verification interface that displays the depth map, IMU data, audio alongside the video. This would give the user the same cross-modal picture, without having to trust a black box. It doesn't matter how good the verification system is, someone with enough means like an intelligence agency might still be able to construct a complex ad-hoc rig that spoofs all the inputs simultaneously, or find a zero-day vulnerability in the chip's secure enclave. That's a possible risk, and it is acceptable. Many security systems are breakable, but still useful. Putting a lock on your door doesn't mean that a determined burglar won't find a way to break or bypass it. Yet we all still use locks. The value of a security measure is not that it is perfect https://www.schneier.com/wp-content/uploads/2016/02/paper-psychology-of-security.pdf , but that it puts the cost of an attack above the threshold that most attackers are willing to pay. The realistic threat model for fake media I'm concerned with here is not what countries can do. It is what an average person can do with their laptop to generate a fake video of their ex-partner, or what one person can do in an hour to discredit a witness in a civil case. For such smaller actors, multimodal provenance verification raises the cost of convincing fakes from near zero, back to prohibitively expensive. That is the win. The technical provenance layer is necessary but not sufficient on its own. For us to be able to continue using images and videos as credible snapshots of reality, two more things need to happen: A multimodal provenance standard is only useful if platforms treat provenance-tagged and untagged content differently. The goal is to create a clear epistemic tier: content with a verified provenance chain is treated differently from content without one, especially in high-stakes contexts like news, legal proceedings, and public health. Once we expect every image to be linked to a page detailing the level of multimodal verification, we’ll learn to treat media as “ unreliable until proven otherwise ”, just like we do with text. It might seem exaggerated, as most of the media online might still be genuine for a time , but I believe it’s the only framing that will still hold when generation becomes perfect. In this setting, provenance verification isn't a simple binary trust/no-trust flag, but rather an indicator of how trustworthy a piece of media is, based on which captured modalities are available for scrutiny, and which have been verified to be globally coherent. The provenance system, if widely adopted by platforms, would largely make non-disclosure self-defeating unverified content about real life events would carry a default trust penalty, or even an algorithmic one preventing it from becoming viral , but even in the best case scenario that adoption is years away. In the meantime, legal liability is the bridge: we need to make it legally costly to present untagged content as factual evidence. This does not mean criminalizing unverified media, but making it so that presenting generated content as authentic carries legal risk, the same way using a forged document already does. Such a policy is completely orthogonal to the proposed multimodal provenance verification system, and could already be implemented today. There are many interesting uses for generative models, but I can’t find a single one that would justify not disclosing that the content has been AI-generated or doctored; similar to ads which are, in some countries, required to disclose if the model has been photoshopped https://www.bbc.co.uk/news/world-europe-41443027 . To be consistent, this should also extend to parody/satirical accounts, under the same umbrella of “presenting fiction as authentic content”. It should be easy and immediate to check the provenance of any content 5 . With a multimodal provenance verification system coupled with enforced platform compliance and legal liability, I believe that our media-based system has a chance to survive the imminent mass advent of flawless, unwatermarked generated content. Hiding the issue behind reassuring, but temporary and ultimately pernicious AI detectors only contributes to increasing the risk of a catastrophic rupture the day our institutions realise that simple images, videos and recordings can't be trusted anymore. Note : Right after I had finished writing this article, YouTube announced their effort https://blog.youtube/news-and-events/improving-ai-labels-viewers-creators/ to combat misinformation and AI-generated content. It has some good sides to it, such as adding a visible mention of AI-generated content, but also many of the flaws described in this article. In particular, they announce automatically flagging AI-generated videos, which suffers from all the detector issues we discussed. They also integrate C2PA metadata as a permanent disclosure signal, which is the right instinct but remains monomodal and strippable. Most importantly, disclosure is still voluntary for content not created with YouTube's own tools, with no legal consequence for non-compliance, meaning the people most likely to abuse the system are the least likely to use it. A bad actor using an open-source model on private infrastructure produces unwatermarked, unverified content that this system has no handle on. Better than nothing, but not a foundation we can build evidence standards on. In Zvi's sense, not the actual monetary cost of producing the fake. Having worked in image generation research for many years, I have witnessed the ridiculous pace at which advances are made. We always tend to overestimate how long it will take for image synthesis to reach a given quality threshold. We haven't reached perfect, flawless generation yet, but we're getting there, and it's coming sooner than we think. DeepMind's SynthID-Image https://arxiv.org/pdf/2510.09263 approach of perturbing pixel values in ways imperceptible to humans seems to be relatively robust https://allenkuo.medium.com/synthid-image-watermark-research-report-9b864b19f9cf to perturbations, but they acknowledge that 'achieving perfect security through watermarking is impossible'. Or at least that the modifications are properly documented and/or reversible. A reasonable objection is that satire needs to be believable to work https://www.theguardian.com/law/2022/oct/04/the-onion-defends-right-to-parody-in-very-real-supreme-court-brief-supporting-local-satirist , and a disclaimer would kill the joke. But verifiable origin is not the same as mandatory disclosure. A reader who wants to be fooled by The Onion can still be fooled, but a reader who wants to verify whether the news is real needs to be able to do so in one click.