Haversack of RAG: Demos AI Report Fails Its Own Disinformation Test

A Demos think tank paper claiming Russia rigged websites to make AI chatbots spread disinformation is undermined by its own evidence, as the technical signal it cites as proof of deliberate targeting appears in the think tank's own website source code. The paper found that leading questions about made-up stories produced false answers about 16.6% of the time, matching a May 2025 NewsGuard audit, but failed to distinguish between models recalling junk from training and deliberate manipulation. Previous research by Maxim Alyukov, Mykola Makhortykh, Alexandr Voronovici, and Maryna Sydorova in the Harvard Kennedy School Misinformation Review found a 5% echo rate and attributed the issue to data voids rather than orchestrated campaigns.

A new think tank paper https://demos.co.uk/research/geo-for-geopolitics-what-happens-when-ai-and-information-warfare-collide/ is out with a shocking, shocking I tell you, revelation that there is disinformation afoot. Mincemeat https://www.flyingpenguin.com/was-operation-mincemeat-fashioned-after-the-millners-hat-mystery/ I say old chaps, this ruse is a Haversack https://www.flyingpenguin.com/this-day-in-history-1917-battle-of-beersheba/ Now, before I get too snarky about this paper being disconnected from reality, I have to admit they did prove a small thing: if you ask a chatbot leading questions about made-up stories, it plays along about one time in six, and the junk sites may turn up in its citations. That’s a finding. I can work with that. However, then these authors took a running leap off a cliff without a rope to claim Russia deliberately rigged websites so AI models would belch out the lies. I mean, duh. The whole web history has been astroturfing and sock puppets, and Russian history has been militarized disinformation https://www.flyingpenguin.com/origins-of-the-term-disinformation/ for over a century copying the Americans and British , so put that all together and what do you expect? But that’s the exact problem here. Our expectations aren’t a substitution for science. Their test can’t tell their explanation apart from a boring one: models remembered the junk from training. They never checked which it really is, so reading the paper is very disappointing. The one in six math also is familiar. NewsGuard in May 2025 https://www.newsguardrealitycheck.com/p/newsguard-audit-finds-chatbots-repeat tested Australian election falsehoods with its innocent, leading and malign prompt styles and also got 16.6 percent. A different country and different topic hit that same number, which suggests the test design may be at fault. Even more troubling is the fact that their website “evidence” of bad intentions is simply what a standard WordPress plugin does on its own. Let me explain. This report says the max-snippet:-1 directive is a strong signal of deliberate targeting. Well, it is also found in the page source of the think tank’s own release page https://demos.co.uk/research/geo-for-geopolitics-what-happens-when-ai-and-information-warfare-collide , and on the Euronews article https://www.euronews.com/my-europe/2026/07/27/ai-chatbots-citing-russian-propaganda-sourced-from-eu-sanctioned-outlet that launched the report. Anyone can view the source and check. Are they not aware of their own site undermining their main claim? And every one of the questions used came from the research team itself? Is that any different from what the authors are accusing the Russians of doing? Is this paper now not evidence of British information dissemination? The researchers typed the questions, and then just reported the answers as a public threat without proper checks. I’m not sure why. The necessary checks are easy, and yet they skipped them. As a matter of fact, they already were published by someone else. Four researchers at Manchester and Bern Maxim Alyukov, Mykola Makhortykh, Alexandr Voronovici and Maryna Sydorova tested the earlier NewsGuard claim https://misinforeview.hks.harvard.edu/article/llms-grooming-or-data-voids-llm-powered-chatbot-references-to-kremlin-disinformation-reflect-information-gaps-not-manipulation/ in the Harvard Kennedy School Misinformation Review last October. Their echo rate was 5 percent instead of 33. Junk citations appeared in 8 percent of answers, usually with warnings attached, and almost entirely when a prompt matched stories that appeared solely on the junk network. They concluded the cause was data voids: models pull from junk when better coverage is thin. They also noted NewsGuard published no prompt set and counted cautious answers as failures. Their own analysis in Al Jazeera has perhaps said it best https://www.aljazeera.com/opinions/2025/7/8/is-russia-really-grooming-western : a study built to find disinformation has found it. Swiss reporters at NZZ got NewsGuard to confirm https://www.nzz.ch/technologie/russische-agenten-versuchen-mit-einer-flut-von-propaganda-texten-westliche-ki-chatbots-zu-infiltrieren-meist-ohne-erfolg-ld.1876537 that its prompts were written so the chatbot only had to agree with the falsehood in the question, and NewsGuard declined to release the full prompt list. The same researchers also made the point https://www.aljazeera.com/opinions/2025/7/8/is-russia-really-grooming-western that Margarita Simonyan cites Western research as proof that RT works, rather than citing actual proof that RT works. So this paper says to me the think tank started out to prove a hypothesis “Russia did this” and then wrote everything down as proof. Tests that could have disproved it don’t appear to have been tried. And that means they launched an unproven conclusion . Far worse, they are gifting the Russians a huge boost. Why? These horrible spammers’ whole existence is convincing the world their junk works. A British report saying “the junk works” is just an ad for the junk sellers. This report was funded by whom exactly? Will Perrin, co-architect of the Online Safety Act’s duty of care. He is funding the paper that argues platform regulation should now extend to LLMs. Will Perrin alert - He managed the 2001 Communications White Paper https://www.iicom.org/profile/william-perrin/ that created Ofcom https://news.ycombinator.com/item?id=44866725 . - His duty of care model, by his own account https://indigotrust.org.uk/about-us/williamperrin/ , underpins the UK approach to regulating online services https://committees.parliament.uk/writtenevidence/134951/html/ . - He chaired the campaign coalition https://fosi.org/people/william-perrin/ that lobbied the Act into law https://carnegieuk.org/blog/online-harms-the-way-forward/ . - He advises the network https://www.onlinesafetyact.net/about/ implementing it. He built the regulator, then he wrote its doctrine, then he ran its lobby, and now he pays for the synthetic laundering , a paper with unproven claims about Russian disinformation, to push state censorship. Oh, and the report he funded calls for “black-lists”, in 2026. Racist language, on top of it all, in a report complaining about “bad” information spread Such fools. The UK’s National Cyber Security Centre retired that term in 2020, and the Home Office has followed. I’ll tell you who needs a block list . Will Perrin Fool me once…