04:00
2026-07-09
arxiv.org
machine-learning
Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention
A new study analyzing seven pretrained models reveals that the positional encoding scheme in transformers determines the spectral properties of attention heads, with RoPE inducing rotational spectra a…