{"slug": "where-grokking-happens-distributed-utility-and-fourier-recoding-without-a-module", "title": "Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch", "summary": "A new arXiv paper (2609.17571v1) reports that grokking in Transformers is a spectral recoding of an existing distributed circuit rather than a switch between modules, based on behavior-aligned exact activation games the authors call Transition Games. The study found distributed utility gain with a prospective block-0 attention bias, with selected degree-two modes accounting for 67–92% of its addition contrast across replacement games, and a disjoint exact path study confirming block-1 MLP mediates more of the effect than all other tested downstream paths in 12/12 pairs. The sharper \"MLP memorizes, attention generalizes\" prediction reversed (-.331 bits/example at the memory anchor; 0/12 in the predicted direction), while routing onset, global rank collapse, and a prime-invariant architecture ridge also failed.", "body_md": "arXiv:2609.17571v1 Announce Type: new \nAbstract: Where in a Transformer is the change from memorization to generalization functionally expressed? We introduce Transition Games--behavior-aligned exact activation games with paired non-generalizing controls--and find distributed utility gain with a prospective block-0 attention bias; selected degree-two modes account for 67--92% of its addition contrast across replacement games, and a disjoint exact path study confirms that block-1 MLP mediates more of their effect than all other tested downstream paths in 12/12 pairs. The sharper \"MLP memorizes, attention generalizes\" prediction instead reverses (-.331 bits/example at the memory anchor; 0/12 in the predicted direction), while routing onset, global rank collapse, and a prime-invariant architecture ridge also fail, identifying grokking here as spectral recoding of an existing distributed circuit rather than a module switch.", "url": "https://wpnews.pro/news/where-grokking-happens-distributed-utility-and-fourier-recoding-without-a-module", "canonical_source": "https://arxiv.org/abs/2609.17571", "published_at": "2026-09-17 04:00:00+00:00", "updated_at": "2026-09-17 04:25:50.929019+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "neural-networks", "large-language-models"], "entities": ["arXiv", "Transformer", "Transition Games", "MLP", "block-0 attention bias", "block-1 MLP"], "alternates": {"html": "https://wpnews.pro/news/where-grokking-happens-distributed-utility-and-fourier-recoding-without-a-module", "markdown": "https://wpnews.pro/news/where-grokking-happens-distributed-utility-and-fourier-recoding-without-a-module.md", "text": "https://wpnews.pro/news/where-grokking-happens-distributed-utility-and-fourier-recoding-without-a-module.txt", "jsonld": "https://wpnews.pro/news/where-grokking-happens-distributed-utility-and-fourier-recoding-without-a-module.jsonld"}}