{"slug": "power-law-graph-attention-exact-generalization-of-scaled-dot-product-attention", "title": "Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference", "summary": "A new arXiv paper (2608.10288v1) introduces Power Law Graph Attention (PLGA), a learned attention mechanism that exactly generalizes scaled dot-product attention (SDPA) and is used in the Power Law Decoder Representations LLM (PLDR-LLM). The paper proves that PLGA contains SDPA exactly when the bilinear operator is the identity, and presents an inference-collapse theorem showing that exact input invariance collapses inference to generalized SDPA with a constant operator. Measured invariance shows relative fluctuations of 10^-6 and below, and block and sequential scoring agree on TruthfulQA within 5e-5 per item.", "body_md": "arXiv:2608.10288v1 Announce Type: new\nAbstract: The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA), replace the fixed bilinear form of scaled dot-product attention (SDPA) with a learned, input-generated bilinear operator $G_{LM}$, built from a positive tensor $A_{LM}$ by elementwise power laws. The architecture is fully specified, verified against pinned reference releases; claims are labeled theorem, conditional theorem, measurement, or conjecture. Unconditionally: PLGA contains SDPA exactly at $G_{LM}=I$; $A_{LM}$ and $A_P$ are strictly entrywise positive, with Perron-Frobenius structure on $A_{LM}$; the DAG regularizer has the NOTEARS walk-counting form and positivity obstructs exact acyclicity; and, under nonresonance (satisfied by standard rotary frequencies), a commutant criterion identifies which operators preserve relative-position dependence. An inference-collapse theorem: exact input invariance of deductive outputs collapses inference to generalized SDPA with a constant operator. Measured invariance: relative fluctuations of $10^{-6}$ and below; perturbation bounds quantify but do not certify cached inference; the assembled proxy misses the decoding margin. A conditional three-stage mechanism (rotary twirl, concentration, row-map contraction) is measured on a released checkpoint. Blockwise training and scoring under the global Gram are stated with explicit target exposure; on tested samples, block and sequential scoring select identical answers and agree on the published TruthfulQA probability-mass metric within $5\\times 10^{-5}$ per item. Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures. Selected proof cores are machine-checked in Lean 4.", "url": "https://wpnews.pro/news/power-law-graph-attention-exact-generalization-of-scaled-dot-product-attention", "canonical_source": "https://www.machinebrief.com/news/power-law-graph-attention-exact-generalization-of-scaled-dot-fr3y", "published_at": "2026-08-12 04:00:00+00:00", "updated_at": "2026-08-12 04:14:44.148921+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["arXiv", "Power Law Graph Attention (PLGA)", "Power Law Decoder Representations LLM (PLDR-LLM)", "TruthfulQA", "Lean 4"], "alternates": {"html": "https://wpnews.pro/news/power-law-graph-attention-exact-generalization-of-scaled-dot-product-attention", "markdown": "https://wpnews.pro/news/power-law-graph-attention-exact-generalization-of-scaled-dot-product-attention.md", "text": "https://wpnews.pro/news/power-law-graph-attention-exact-generalization-of-scaled-dot-product-attention.txt", "jsonld": "https://wpnews.pro/news/power-law-graph-attention-exact-generalization-of-scaled-dot-product-attention.jsonld"}}