Do Quantum Models Scale Like LLMs?

A new arXiv paper (2609.20912v1) reports that RydbergGPT, an autoregressive transformer trained on qubit projective measurement data from interacting Rydberg atom arrays, follows a power-law loss-versus-dataset-size relationship with a loss floor correction near the quantum system's critical point, but the power-law fit degrades substantially away from criticality. Comparing the statistical structure of Rydberg measurements and natural-language corpora using an entropy-normalised, finite-sample-corrected mutual information two-point function, the authors found near-critical two-point functions most closely resemble those in natural language, while configurations far from criticality decay more rapidly. The authors conclude this supports the hypothesis that multi-scale dependence contributes to stable neural scaling and that scaling behaviour should be viewed as a property of the model-data pair.

arXiv:2609.20912v1 Announce Type: new Abstract: In this work, we study the neural scaling laws of RydbergGPT, an autoregressive transformer model trained on qubit projective measurement data gathered from interacting Rydberg atom arrays. The quantum system is known to exhibit a finite-size remnant of a critical point as the laser detuning parameter is varied. We find that near the critical point the transformer loss as a function of training dataset size is well described by a power-law with a loss floor correction. However, away from criticality the quality of the power-law description is substantially reduced. We then compare the statistical structure of both Rydberg measurements and natural-language corpora using an entropy-normalised, finite sample corrected mutual information "two-point" function. We find that near-critical statistics of the two point functions are closest to those observed in natural-language, whilst other qubit configurations far from the critical point have two-point functions that decay more rapidly. This supports the hypothesis that multi-scale dependence contributes to stable neural scaling, and that scaling behaviour should be viewed as a property of the model-data pair.