Belief Propagation in LLM World Models: Measuring Strategic Information Bias with Prediction Markets

A new study using LLMs and prediction markets finds that English news context systematically biases territorial predictions in Ukraine-related markets, with errors occurring 64 to 72 percent of the time when predictions favor territorial capture. The research, published on arXiv, analyzed 111 prediction markets and approximately 93,000 predictions across four models, showing that the bias originates primarily in the text sources rather than the models themselves. Supplementing with Ukrainian military-analytical sources reduced the bias for all clean models.

arXiv:2607.20441v1 Announce Type: new Abstract: Every information ecosystem produces beliefs that shape strategic decisions. Both human analysts and AI systems inherit the blind spots of their information sources. We show that LLMs, combined with prediction markets, function as a calibrated instrument for measuring how far ecosystem-induced beliefs deviate from an external reference: LLMs extract the beliefs a text corpus implies, and prediction market price trajectories, anchored at resolution by realised outcomes, provide the calibration reference against which to quantify the deviation. We isolate the bias contribution of specific text through ablation: varying information context while holding the model fixed, with a contaminated model that knows actual outcomes as control. Applied to 111 Ukraine-related prediction markets, comprising approximately 93,000 predictions across four models, we find that English news context systematically biases territorial predictions, wrong 64 to 72 percent of the time when it pushes predictions toward territorial capture. A contaminated model that knows actual outcomes shows the same error rate, indicating that the bias originates primarily in the text. Supplementing with Ukrainian military-analytical sources reduces the bias for all clean models, while absolute-error gains are partial and model-dependent. We show that the distortion originates primarily in the sources, not the models. Consistent across four architectures, it will persist in any system that processes them and propagate into downstream decisions.