{"slug": "google-deepmind-study-finds-llms-cant-reliably-roll-their-own-dice", "title": "Google DeepMind study finds LLMs can’t reliably roll their own dice", "summary": "A Google DeepMind paper with the National University of Singapore, \"The Illusion of Stochasticity in LLMs\" (arXiv 2604.06543, presented at COLM 2026), found large language models fail to produce outputs matching specified distributions, scoring near-zero p-values on Chi-Square and Kolmogorov-Smirnov tests and favoring numbers such as 7 and 42. In game-like tests where an optimal randomized strategy would cap an attacker's success rate at 75%, LLM agents relying on their own sampling let attack success rates climb to between 96.6% and 99.9%. The researchers report that models including Gemini-3.0-Pro and Qwen3-32B sampled accurately when given random seeds from external code, separating the ability to reshape randomness from the ability to generate it.", "body_md": "# Google DeepMind study finds LLMs can’t reliably roll their own dice\n\nNew research shows large language models produce biased random outputs, but sample accurately when handed random numbers from code\n\nAsk a large language model to pick a random number, and it will happily oblige. According to new research from [Google](https://cryptobriefing.com/markets/alphabet/) DeepMind, it just won’t be very random about it.\n\nA paper titled “The Illusion of Stochasticity in LLMs” argues that reliable random sampling is a basic requirement for AI agents. It also argues that today’s models don’t meet it.\n\n## What the researchers found\n\nThe study was produced by Google DeepMind in collaboration with the National University of Singapore. Its listed authors include Xiangming Gu, Soham De, Michalis K. Titsias, Larisa Markeeva, Petar Veličković, and Razvan Pascanu.\n\nThe paper carries the arXiv ID 2604.06543. It was submitted on April 8, 2026, revised on August 2, 2026, and presented at the COLM 2026 conference.\n\nThe core problem is a disconnect between knowing and doing. LLMs can estimate probabilities well and handle the underlying math. Yet when asked to actually produce outputs that follow a given distribution, they fall short.\n\nThe researchers tested models on sampling from standard distributions, including uniform and Gaussian ones. They ran the results through Chi-Square and Kolmogorov-Smirnov tests, two common statistical checks for whether a set of numbers matches the pattern it is supposed to follow.\n\nThe models scored **near-zero p-values** across a range of configurations. Put plainly, the odds that the outputs were genuinely following the target distribution were vanishingly small.\n\nThe biases were not subtle. The study found models gravitating toward specific numbers, including 7 and 42. Outputs could also shift depending on the order in which information appeared in the prompt.\n\nThe researchers also stress that token-level entropy is not the same thing as controllable sampling. LLMs already introduce some variation when picking words, which is why the same prompt can produce different answers. That built-in variability does not mean a model can deliberately follow a specified distribution on request.\n\n### AI, tech, and the markets they move—in one daily briefing.\n\nDaily. Free. Join 34,000+ readers across crypto, finance, and policy.\n\n## The fix is surprisingly simple\n\nThe study also contains good news. When models were handed random seeds from an external source, such as numbers generated by code, they performed much better.\n\nAdvanced models including Gemini-3.0-Pro and Qwen3-32B were able to sample accurately from the desired distributions in that setup. The models could take a truly random input and transform it correctly into the target shape.\n\nThat distinction is the paper’s key insight. The models are capable of the deterministic math needed to reshape randomness. What they can’t do is generate the raw randomness themselves.\n\n## Why biased randomness becomes a security problem\n\nThe study tested this in game-like scenarios. In these tests, an optimal randomized strategy would hold an attacker’s success rate at 75%.\n\nWhen LLM agents relied on their own native sampling, attack success rates climbed to between **96.6% and 99.9%** in certain tests.\n\nThe paper frames this as a distinct failure point for agentic systems. Agents frequently need to sample from distributions, often ones inferred from observed data. If the agent can’t do that faithfully, its decision-making under uncertainty becomes skewed in ways that are hard to spot from the outside.\n\n## Background: a growing scrutiny of LLM randomness\n\nThe DeepMind paper adds to a broader body of work examining whether LLMs can be trusted with statistical tasks. Related research from 2025 had already flagged randomness issues and biases in sequential decision-making.\n\nWhat this study adds is a clean separation of the problem. Models understand probability. They can transform random inputs. The weak link is generating randomness natively, and the paper pins that down with formal statistical tests rather than anecdotes.\n\n## What this means for AI builders and buyers\n\nFor developers building agents, the practical lesson is straightforward. Don’t ask the model to be random. Generate randomness in code and pass it to the model to work with.\n\nThe security angle deserves particular attention. Agents deployed in adversarial or competitive environments could leak predictable patterns. The jump from a 75% optimal attack success rate to as high as 99.9% shows how quickly that weakness compounds.\n\nThere is also a testing implication. Companies evaluating AI agents may need to check statistical fidelity directly, rather than assuming varied outputs mean unbiased ones. As the study shows, a model can look unpredictable while still being highly exploitable.\n\n**Disclosure:** This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our\n\n[Editorial Policy](https://cryptobriefing.com/editorial-policy/).", "url": "https://wpnews.pro/news/google-deepmind-study-finds-llms-cant-reliably-roll-their-own-dice", "canonical_source": "https://cryptobriefing.com/google-deepmind-llms-random-sampling-flaws/", "published_at": "2026-10-26 18:28:55+00:00", "updated_at": "2026-10-26 18:45:22.156585+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-agents", "ai-safety"], "entities": ["Google DeepMind", "National University of Singapore", "Xiangming Gu", "Soham De", "Michalis K. Titsias", "Larisa Markeeva", "Petar Veličković", "Razvan Pascanu"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/google-deepmind-study-finds-llms-cant-reliably-roll-their-own-dice", "markdown": "https://wpnews.pro/news/google-deepmind-study-finds-llms-cant-reliably-roll-their-own-dice.md", "text": "https://wpnews.pro/news/google-deepmind-study-finds-llms-cant-reliably-roll-their-own-dice.txt", "jsonld": "https://wpnews.pro/news/google-deepmind-study-finds-llms-cant-reliably-roll-their-own-dice.jsonld"}}