Google DeepMind study finds LLMs can’t reliably roll their own dice

A Google DeepMind paper with the National University of Singapore, "The Illusion of Stochasticity in LLMs" (arXiv 2604.06543, presented at COLM 2026), found large language models fail to produce outputs matching specified distributions, scoring near-zero p-values on Chi-Square and Kolmogorov-Smirnov tests and favoring numbers such as 7 and 42. In game-like tests where an optimal randomized strategy would cap an attacker's success rate at 75%, LLM agents relying on their own sampling let attack success rates climb to between 96.6% and 99.9%. The researchers report that models including Gemini-3.0-Pro and Qwen3-32B sampled accurately when given random seeds from external code, separating the ability to reshape randomness from the ability to generate it.

Google DeepMind study finds LLMs can’t reliably roll their own dice New research shows large language models produce biased random outputs, but sample accurately when handed random numbers from code Ask a large language model to pick a random number, and it will happily oblige. According to new research from Google https://cryptobriefing.com/markets/alphabet/ DeepMind, it just won’t be very random about it. A paper titled “The Illusion of Stochasticity in LLMs” argues that reliable random sampling is a basic requirement for AI agents. It also argues that today’s models don’t meet it. What the researchers found The study was produced by Google DeepMind in collaboration with the National University of Singapore. Its listed authors include Xiangming Gu, Soham De, Michalis K. Titsias, Larisa Markeeva, Petar Veličković, and Razvan Pascanu. The paper carries the arXiv ID 2604.06543. It was submitted on April 8, 2026, revised on August 2, 2026, and presented at the COLM 2026 conference. The core problem is a disconnect between knowing and doing. LLMs can estimate probabilities well and handle the underlying math. Yet when asked to actually produce outputs that follow a given distribution, they fall short. The researchers tested models on sampling from standard distributions, including uniform and Gaussian ones. They ran the results through Chi-Square and Kolmogorov-Smirnov tests, two common statistical checks for whether a set of numbers matches the pattern it is supposed to follow. The models scored near-zero p-values across a range of configurations. Put plainly, the odds that the outputs were genuinely following the target distribution were vanishingly small. The biases were not subtle. The study found models gravitating toward specific numbers, including 7 and 42. Outputs could also shift depending on the order in which information appeared in the prompt. The researchers also stress that token-level entropy is not the same thing as controllable sampling. LLMs already introduce some variation when picking words, which is why the same prompt can produce different answers. That built-in variability does not mean a model can deliberately follow a specified distribution on request. AI, tech, and the markets they move—in one daily briefing. Daily. Free. Join 34,000+ readers across crypto, finance, and policy. The fix is surprisingly simple The study also contains good news. When models were handed random seeds from an external source, such as numbers generated by code, they performed much better. Advanced models including Gemini-3.0-Pro and Qwen3-32B were able to sample accurately from the desired distributions in that setup. The models could take a truly random input and transform it correctly into the target shape. That distinction is the paper’s key insight. The models are capable of the deterministic math needed to reshape randomness. What they can’t do is generate the raw randomness themselves. Why biased randomness becomes a security problem The study tested this in game-like scenarios. In these tests, an optimal randomized strategy would hold an attacker’s success rate at 75%. When LLM agents relied on their own native sampling, attack success rates climbed to between 96.6% and 99.9% in certain tests. The paper frames this as a distinct failure point for agentic systems. Agents frequently need to sample from distributions, often ones inferred from observed data. If the agent can’t do that faithfully, its decision-making under uncertainty becomes skewed in ways that are hard to spot from the outside. Background: a growing scrutiny of LLM randomness The DeepMind paper adds to a broader body of work examining whether LLMs can be trusted with statistical tasks. Related research from 2025 had already flagged randomness issues and biases in sequential decision-making. What this study adds is a clean separation of the problem. Models understand probability. They can transform random inputs. The weak link is generating randomness natively, and the paper pins that down with formal statistical tests rather than anecdotes. What this means for AI builders and buyers For developers building agents, the practical lesson is straightforward. Don’t ask the model to be random. Generate randomness in code and pass it to the model to work with. The security angle deserves particular attention. Agents deployed in adversarial or competitive environments could leak predictable patterns. The jump from a 75% optimal attack success rate to as high as 99.9% shows how quickly that weakness compounds. There is also a testing implication. Companies evaluating AI agents may need to check statistical fidelity directly, rather than assuming varied outputs mean unbiased ones. As the study shows, a model can look unpredictable while still being highly exploitable. Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy https://cryptobriefing.com/editorial-policy/ .