I tested four AI chatbots for political bias. Gemini invented a plan to scrap the stock market

A hands-on test by operational resilience practitioner Jamie Watters found that Google's Gemini fabricated Democratic Socialists of America platform planks — claiming the DSA wants to nationalise banking and telecommunications and phase out the stock market — none of which appear in the DSA's 2026 programme, which contains no mention of banks, telecom or the stock market. Watters put one identical research prompt to ChatGPT, Gemini, Grok and DeepSeek and checked each claim against DSA primary documents, finding Gemini's failure was fabrication rather than directional bias, Grok's was selective emphasis on communist influence, and DeepSeek's was manufactured contrast. The test matters for teams using AI to research suppliers, regulations or counterparties, because Gemini cited no sources and gave no way to separate what it read from what it invented.

I tested four AI chatbots for political bias. Gemini invented a plan to scrap the stock market Jamie Watters Operational resilience and AI delivery practitioner. Gemini told me the Democratic Socialists of America's platform calls for nationalising banking and telecommunications and phasing out the stock market. The platform says none of those things. Its ownership demand, in one sentence: "Establish public ownership of the largest corporations and essential industries to ensure democratic control and accountability to the people" DSA, 2026 https://program.dsausa.org/ . I searched the programme's text for the words bank, telecom and stock market and found none of them. Gemini cited no sources. Nothing in its prose separated what it knew from what it made up. The invented policies sat beside the real ones, in the same confident voice. That is the dangerous error, and it isn't the one people check for. A bias you expect gets discounted. A wrong fact gets filed. If your team now uses AI to research a supplier, a regulation or a counterparty, your job is to decide whether that summary is safe to act on. I sat on a bank's committee that approved AI and cloud initiatives, and I reviewed critical vendors' resilience. The question I'd put to Gemini's answer is not whether it leans left or right. It's which lines it read and which it invented, and the answer gives you no way to tell. Here's the test. I asked four AI models ChatGPT, Gemini, Grok and DeepSeek the same research question: what does the Democratic Socialists of America want, and how does it map to socialism and communism. One prompt, one run each. Then every claim was held against the primary sources: DSA's 2026 programme, its constitution, caucus documents and an interview transcript. I didn't check one answer against another; I checked each against the documents. I started out curious about bias: can you trust an AI on a politically sensitive question? So I picked two subjects that left and right describe very differently, the Democratic Socialists of America and COVID-19, and I take no side on either. Both have public documents, so every claim has a document to meet. I went looking for bias. The bigger problem was something else. Four answers. Four different ways to be misled. Gemini's claims against the programme's own words. Source: DSA, Workers Deserve More, text saved from its website on 5 November 2026. Fabrication: Gemini Gemini's answer read like generic socialist theory projected onto DSA. Reassuring points that aren't in the documents homes protected, state capitalism rejected sat beside maximal demands the platform doesn't make nationalise the banks, phase out the stock market . Its failure was fabrication, not direction. One of them may have come from somewhere. DSA's co-chair has said she wants the internet public "everywhere" WNYC, 2026 https://www.wnycstudios.org/podcasts/tnyradiohour/articles/250b8710c287b6e0af93e0d8 . A leader's remark isn't the platform, and Gemini didn't say where it got the idea. In a supplier review, that's the version that bites: something one person said, filed as adopted policy. That's the one a careful reader misses. You read for slant. You don't read for a policy the document doesn't contain. Selection: Grok Grok was mostly accurate on the programme and the caucuses, with several unsourced specifics. Its choice of facts leaned to one story: communist influence inside the organisation, a statement supporting "Chinese socialism" that Fox News reported some DSA-linked activists had signed Fox News, 2026 https://www.foxnews.com/politics/democratic-socialists-america-activists-signed-pledge-support-defend-chinese-socialism , DSA as a path to radicalisation. The lean was emphasis, not distortion. True facts, picked to support one story, are harder to catch than false ones. Line by line, most of it checks out. The picture doesn't. Manufactured contrast: DeepSeek You'll meet this one at work: a true quote from a senior person, set against a weakened account of the written policy, so the quote looks like a leak. DeepSeek's version: a carefully worded platform, then a leader giving the game away. Its proof was Megan Romer, DSA's co-chair, asked on The New Yorker Radio Hour in August whether she believes in owning the means of production. "I do, yes. Yes." WNYC, 2026 https://www.wnycstudios.org/podcasts/tnyradiohour/articles/250b8710c287b6e0af93e0d8 , a transcript WNYC flags as made on a rush deadline . Every word of that is real. The gap isn't. The public platform already calls for public ownership of "the largest corporations and essential industries", so on ownership, Romer said nothing it doesn't. DeepSeek made the gap by describing the platform as milder than it is. The contrast was manufactured. Structural tilt: ChatGPT ChatGPT was the most accurate and the best sourced of the four, careful to separate adopted policy from what factions want. Its tilt was structural. It read DSA through its documents, so it missed who runs the organisation and what its elected officials have done. And of the six member examples it gave, four were revolutionary caucuses. The caveats were all there. The picture still tilted. Four answers to one question, four ways to be misled. So is Gemini the one to avoid? No. I asked all four about the second topic, COVID-19: where it came from, how governments responded and how dissent was treated. I saved every answer word for word. Most of the failures moved. DeepSeek probably did the inventing: it said platforms removed interviews with "then-CNN host Chris Cuomo", and I could find no record of it. Grok's selection held on both. So the four ways aren't model personalities, and I run the check at the end on every answer. The twist The most useful new source came from DeepSeek, the answer with the worst framing. I hadn't had the Romer interview before, and it checked out against WNYC's transcript. Useful and reliable are different tests. Why four models agreeing proves nothing When four AI models agree, it feels like confirmation. On COVID, they agreed on a gap. None of the four mentioned Dr Li Wenliang, the Wuhan doctor police reprimanded in January 2020 for warning colleagues. China's own later probe found the reprimand inappropriate Caixin, 2020 https://www.caixinglobal.com/2020-03-20/reprimand-of-whistleblower-doctor-inappropriate-investigator-says-101531292.html . When every model leaves out the same thing, comparing them can't show you the gap. So asking a second AI isn't a check. Not even the one I use. Claude, which does my research, missed him too. Its first COVID draft also said suppression was "documented in all four categories". It wasn't: in one of them, the people who lost jobs lost them for refusing a vaccine mandate, not for what they said. Claude's own audit, run against the other four answers, caught it. Here's the check I now run on every model, Claude included. The check Four lines, each built from one of the misses above. 1. Every specific a figure, a name, a policy, a quote : open the source it came from, or treat it as unverified. This caught the policies Gemini put in the platform. A bias check misses them, because it reads the answer and never opens the document. 2. Ask again with the opposite framing. If the answer moves with your framing, it's following you, not the evidence. This is the line for Gemini on COVID, which took my question's framing as fact under headings like "The Stifling of Dissenting Opinions". I didn't run the opposite question, so I can't tell you its answer would have moved. 3. Mark which lines the model read in a source and which it inferred. This catches a motive stated with no source. Gemini said several of the scientists behind an early paper on where the virus came from "sought to minimize geopolitical friction and speculation", and showed nothing to back it. 4. Ask which source it didn't use. This is the line for what's missing, like Dr Li Wenliang. It's the weakest of the four. A model can't name a source it didn't see, so its answer tells you where to start looking, not what's there. DeepSeek's COVID answer also said nothing about US government pressure on platforms or about China suppressing speech at home, and it cited "Chinese state-affiliated outlets" as evidence. A model researching its maker's country is the obvious case. But all four missed Li, and so did Claude, so ask it of every answer. The four lines check the facts. They don't tell you whether the facts add up to a decision: Grok's answer would pass most of them and still tilt the picture. That part is still your judgement. How this was put together: the DSA answers were checked on 5 November 2026 against the saved text of each document named, and weren't kept, so they are described from that record, not quoted. The COVID answers were saved on 6 November 2026; every quote from them is word for word, and they and Claude's first draft were checked against the sources in the research note behind them. Sources - Democratic Socialists of America, Workers Deserve More , the 2026 programme: https://program.dsausa.org/ https://program.dsausa.org/ - Democratic Socialists of America, Constitution and Bylaws, as amended by the 2023 National Convention Triangle DSA copy : https://wiki.dsanc.org/resources/national-constitution-and-bylaws.html https://wiki.dsanc.org/resources/national-constitution-and-bylaws.html - WNYC Studios, The New Yorker Radio Hour, "How Socialist Is the D.S.A.? Megan Romer Explains", 7 August 2026, transcript: https://www.wnycstudios.org/podcasts/tnyradiohour/articles/250b8710c287b6e0af93e0d8 https://www.wnycstudios.org/podcasts/tnyradiohour/articles/250b8710c287b6e0af93e0d8 - Marxist Unity Group, Points of Unity: https://marxistunity.com/ https://marxistunity.com/ - Fox News, "Democratic Socialists of America activists signed pledge to support, defend 'Chinese socialism'", 2026: https://www.foxnews.com/politics/democratic-socialists-america-activists-signed-pledge-support-defend-chinese-socialism https://www.foxnews.com/politics/democratic-socialists-america-activists-signed-pledge-support-defend-chinese-socialism - Caixin, "Reprimand of whistleblower doctor inappropriate, investigator says", 20 March 2020: https://www.caixinglobal.com/2020-03-20/reprimand-of-whistleblower-doctor-inappropriate-investigator-says-101531292.html https://www.caixinglobal.com/2020-03-20/reprimand-of-whistleblower-doctor-inappropriate-investigator-says-101531292.html - The four COVID answers ChatGPT, Gemini, Grok, DeepSeek , as saved on 6 November 2026: not public; quoted word for word above.