{"slug": "decodable-but-not-detachable-training-data-granularity-determines-parametric-in", "title": "Decodable But Not Detachable: Training Data Granularity Determines Parametric Modularity in Large Language Models", "summary": "A new arXiv study (2608.10214v1) finds that large language models contain domain-specific parametric shells only when training data was modular at the token level, with 0.65–1.14% of neurons exceeding 60% selectivity for language and modality domains, while academic subject domains show no such structure. The study, spanning three model families (1.5B to 7B parameters) and eight domains, reports that masking code-selective neurons reduces mathematical reasoning accuracy by 16–24 percentage points, and shell strength increases with scale.", "body_md": "arXiv:2608.10214v1 Announce Type: new\nAbstract: Do large language models contain domain-specific parametric shells: concentrated, causally necessary neuron populations whose removal selectively degrades a target domain while sparing others? We apply a uniform causal methodology across two domain granularities, three model families (1.5B to 7B parameters), and eight domains. At the academic subject level, zero neurons exceed 60\\% domain selectivity across 939,008 combined FFN neurons and causal damage matrices are flat, despite domain identity being linearly decodable above 85\\% accuracy. At the language and modality level, 0.65--1.14\\% of neurons exceed 60\\% selectivity, damage matrices are near-perfectly diagonal (ratios up to 595:1), and shell neuron sets are essentially disjoint (IoU $< 0.003$). Masking code-selective neurons reduces mathematical reasoning accuracy by 16--24 percentage points across all models; masking Spanish or Chinese neurons leaves it at or below random. Shell strength increases monotonically with scale and shells are spatially interleaved in a pattern that precludes group-level selective quantization. Parametric shells form where and only where training data was modular at the token level.", "url": "https://wpnews.pro/news/decodable-but-not-detachable-training-data-granularity-determines-parametric-in", "canonical_source": "https://arxiv.org/abs/2608.10214", "published_at": "2026-08-12 04:00:00+00:00", "updated_at": "2026-08-12 04:18:46.238063+00:00", "lang": "en", "topics": ["large-language-models", "ai-research"], "entities": ["arXiv"], "alternates": {"html": "https://wpnews.pro/news/decodable-but-not-detachable-training-data-granularity-determines-parametric-in", "markdown": "https://wpnews.pro/news/decodable-but-not-detachable-training-data-granularity-determines-parametric-in.md", "text": "https://wpnews.pro/news/decodable-but-not-detachable-training-data-granularity-determines-parametric-in.txt", "jsonld": "https://wpnews.pro/news/decodable-but-not-detachable-training-data-granularity-determines-parametric-in.jsonld"}}