Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference

A new arXiv paper (2608.10288v1) introduces Power Law Graph Attention (PLGA), a learned attention mechanism that exactly generalizes scaled dot-product attention (SDPA) and is used in the Power Law Decoder Representations LLM (PLDR-LLM). The paper proves that PLGA contains SDPA exactly when the bilinear operator is the identity, and presents an inference-collapse theorem showing that exact input invariance collapses inference to generalized SDPA with a constant operator. Measured invariance shows relative fluctuations of 10^-6 and below, and block and sequential scoring agree on TruthfulQA within 5e-5 per item.

arXiv:2608.10288v1 Announce Type: new Abstract: The Large Language Model from Power Law Decoder Representations PLDR-LLM and its attention, Power Law Graph Attention PLGA , replace the fixed bilinear form of scaled dot-product attention SDPA with a learned, input-generated bilinear operator $G {LM}$, built from a positive tensor $A {LM}$ by elementwise power laws. The architecture is fully specified, verified against pinned reference releases; claims are labeled theorem, conditional theorem, measurement, or conjecture. Unconditionally: PLGA contains SDPA exactly at $G {LM}=I$; $A {LM}$ and $A P$ are strictly entrywise positive, with Perron-Frobenius structure on $A {LM}$; the DAG regularizer has the NOTEARS walk-counting form and positivity obstructs exact acyclicity; and, under nonresonance satisfied by standard rotary frequencies , a commutant criterion identifies which operators preserve relative-position dependence. An inference-collapse theorem: exact input invariance of deductive outputs collapses inference to generalized SDPA with a constant operator. Measured invariance: relative fluctuations of $10^{-6}$ and below; perturbation bounds quantify but do not certify cached inference; the assembled proxy misses the decoding margin. A conditional three-stage mechanism rotary twirl, concentration, row-map contraction is measured on a released checkpoint. Blockwise training and scoring under the global Gram are stated with explicit target exposure; on tested samples, block and sequential scoring select identical answers and agree on the published TruthfulQA probability-mass metric within $5\times 10^{-5}$ per item. Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures. Selected proof cores are machine-checked in Lean 4.