Every "AI detector" landing page promises 99% accuracy. Then you paste in a
paragraph you actually wrote yourself and it flags you as ChatGPT. I kept
seeing this in the wild — students wrongly accused, editors discarding human
copy, and a pile of tools that were really just guessing.
So I went down the rabbit hole of how AI-content detection actually works,
built a tool to test the claims, and learned that the honest answer is far more
interesting than the marketing.
Image models can embed an invisible statistical watermark (the SynthID-style
approach), and that's a real, checkable signal. Text is different. A model
generates tokens probabilistically; there's no natural place to hide a bit
string that survives copy-paste. Researchers have proposed watermarking the
logit distribution (green/red token lists), but it breaks under:
If a tool claims 100% accuracy on short text, it's lying. Anyone who's actually
benchmarked one knows it.
In practice detectors lean on a few weaker, statistical signals:
None of these is a watermark. They're probabilities, and they fail on edited,
mixed, or short content. That gap between "statistical likelihood" and
"provenance" is where most products quietly pretend to be something they aren't.
I got tired of the gap, so I built a detector that reports honestly — a
confidence score plus an explanation of which signals fired, rather than a
single fake certainty. You can try it here: https://detectaiwatermarks.com
The technical choices I'd highlight:
python
signals = {
"perplexity": score_perplexity(text),
"burstiness": sentence_variance(text),
"watermark_scan": probe_known_watermark(text), # often None, that's fine
}
verdict = calibrate(signals, length=len(text))