{"slug": "localized-adaptation-reveals-distinct-learning-signatures-in-transformers", "title": "Localized Adaptation Reveals Distinct Learning Signatures in Transformers", "summary": "A new study from arXiv introduces a controlled benchmark revealing that the site of adaptation in transformers—early, middle, or late layers—produces distinct learning signatures across five objectives: lexical binding, factual association, behavioral policy learning, causal mapping, and procedural reasoning. The findings show that lexical binding favors early-layer adaptation, factual association benefits later layers, behavioral learning separates late-layer action acquisition from middle-layer policy gating, and causal and procedural transfer improve most with middle- or full-stack adaptation. These patterns replicate across five model families, establishing adaptation site as a key design variable for controlling what models learn and generalize.", "body_md": "arXiv:2607.25663v1 Announce Type: cross\nAbstract: Transformer adaptation is typically distributed across model depth, even when the intended change is narrow. We investigate how adaptation site shapes what a model learns, how well that learning generalizes, and how selectively it is applied. We introduce a controlled benchmark spanning five objectives (lexical binding, factual association, behavioral policy learning, causal mapping, and procedural reasoning) and define each objective's \"adaptation geometry\" as its profile of acquisition, transfer, and boundedness under full-stack and early-, middle-, or late-layer LoRA. The objectives exhibit distinct geometries. Lexical binding favors early-layer adaptation for acquisition and boundedness but requires broader updates for transfer; factual association favors later layers among localized adapters; behavioral learning separates late-layer action acquisition from middle-layer policy gating; and causal and procedural transfer benefit most from middle- or full-stack adaptation. These patterns largely persist under parameter-matched controls, and most corresponding directional contrasts replicate across five model families. These findings establish adaptation site as a key design variable for controlling what models learn, generalize, and leave unchanged.", "url": "https://wpnews.pro/news/localized-adaptation-reveals-distinct-learning-signatures-in-transformers", "canonical_source": "https://www.machinebrief.com/news/localized-adaptation-reveals-distinct-learning-signatures-in-496v", "published_at": "2026-07-29 04:00:00+00:00", "updated_at": "2026-07-29 04:59:32.475340+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research"], "entities": ["arXiv"], "alternates": {"html": "https://wpnews.pro/news/localized-adaptation-reveals-distinct-learning-signatures-in-transformers", "markdown": "https://wpnews.pro/news/localized-adaptation-reveals-distinct-learning-signatures-in-transformers.md", "text": "https://wpnews.pro/news/localized-adaptation-reveals-distinct-learning-signatures-in-transformers.txt", "jsonld": "https://wpnews.pro/news/localized-adaptation-reveals-distinct-learning-signatures-in-transformers.jsonld"}}