{"slug": "grrr-the-geometry-of-reshaping-rotation-and-routing-in-decoder-llm-post-training", "title": "GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-training", "summary": "A study of 12 post-training chains using supervised fine-tuning (SFT) and reinforcement learning (RL) found that removing the diagonal component of weight updates — the part that reshapes singular values — usually preserves most of the gains from post-training on a math evaluation suite. The research, posted as arXiv:2609.22146v1, expresses each weight update in the pretrained matrix's singular value decomposition (SVD) frame, separating changes into diagonal values, off-diagonal values that rotate the coupling between pretrained input and output directions, and null-space values that route outside the matrix's original nonzero SVD core. The authors conclude that post-training gains are carried primarily by reconfiguring and extending pretrained pathways rather than by substantially changing the singular values of pretrained models.", "body_md": "arXiv:2609.22146v1 Announce Type: new \nAbstract: We study how post-training changes the weights of Large Language Models (LLMs) relative to their pretrained weights. Across 12 post-training chains with supervised fine-tuning (SFT) and reinforcement learning (RL), we express each weight update in the pretrained matrix's singular value decomposition (SVD) frame. This decomposition separates the changes of three geometrically distinct components: diagonal values, which reshapes singular values; off-diagonal values, which rotates the coupling between pretrained input and output directions; and null-space values, which routes outside the matrix's original nonzero SVD core. On a math evaluation suite, we find that removing the diagonal component usually preserves most of the gains from post-training. These results suggest that post-training gains are carried primarily by reconfiguring and extending pretrained pathways rather than by substantially changing singular values of pre-trained models.", "url": "https://wpnews.pro/news/grrr-the-geometry-of-reshaping-rotation-and-routing-in-decoder-llm-post-training", "canonical_source": "https://arxiv.org/abs/2609.22146", "published_at": "2026-09-22 04:00:00+00:00", "updated_at": "2026-09-22 04:26:08.247827+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "natural-language-processing"], "entities": ["arXiv", "GRRR"], "alternates": {"html": "https://wpnews.pro/news/grrr-the-geometry-of-reshaping-rotation-and-routing-in-decoder-llm-post-training", "markdown": "https://wpnews.pro/news/grrr-the-geometry-of-reshaping-rotation-and-routing-in-decoder-llm-post-training.md", "text": "https://wpnews.pro/news/grrr-the-geometry-of-reshaping-rotation-and-routing-in-decoder-llm-post-training.txt", "jsonld": "https://wpnews.pro/news/grrr-the-geometry-of-reshaping-rotation-and-routing-in-decoder-llm-post-training.jsonld"}}