Google’s AI is having trouble telling horror stories from real life. Ask it about certain online monsters, and it will describe them as documented fact.
According to a report by Futurism, Google’s AI Overviews repeatedly present entries from the “SCP Foundation” as real. The SCP Foundation is a vast collaborative fan-fiction project. Its “anomalies” are invented, and its website says so plainly.
‘Ed’s Head’ and a haunted toaster #
Take SCP-565, nicknamed “Ed’s Head”. Searched on Google, the AI Overview described it as an “ambulatory human head” that scuttles across the seabed like a crab, complete with a dead man’s name and “official” records to read. At no point did it note that none of this is real.
One case was stranger still. SCP-426 is a fictional toaster that makes people refer to it in the first person. So the AI Overview answered in the first person, as the toaster, and relayed invented accounts of harm as though they had happened. In all, Futurism found at least 20 such cases.
A known weak spot #
To be fair, SCP entries are written to look exactly like dry research files. That is the joke. Even so, the site carries a clear fiction disclaimer, and the AI mostly ignored it. Sometimes it gestured at “lore” without explaining what that meant.
It is also not a fresh problem. AI Overviews have confidently served nonsense before, from glue-on-pizza recipes to invented idioms. One analysis put their accuracy at about 91 per cent. That sounds high, until you remember Google handles trillions of queries, which turns the other 9 per cent into millions of wrong answers.
Why it matters #
Most people searching for SCP codes already know they are fiction. The risk is everyone else. A child who saw a scary clip, or an adult unsure what is real, may simply take the AI’s word for it. The stakes are rising, too, because Google is turning Search into an AI-first interface that answers rather than links.
That shift already strains the open web, and AI Overviews are tied to a sharp drop in clicks to the sites they summarise. Now add a tool that can put fan-fiction at the top of the page as fact. Google did not respond to Futurism. One caveat: when Digital Trends retried the searches, some now correctly labelled the entities “fictional”, so Google may have quietly fixed part of it.
Get the TNW newsletter #
Get the most important tech news in your inbox each week.