{"slug": "readable-faithful-used-three-dissociable-properties-of-demographic-identity-in-a", "title": "Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model", "summary": "A study of Mistral-7B found that demographic identity is readable and faithfully arranged in attention heads, but not causally used, with a single head (L11 H16) faithful across all six attribute types and selection-corrected fidelity up to rho=0.63, while causal use was clearest in one of the least faithful types and replacing the entire identity moved predictions by under 2% of their error. The findings, based on representational similarity analysis against Pew ground truth over 169 demographic cells and 1,089 read-out locations, suggest that readability, faithfulness, and causal use are dissociable properties, complicating the debate on whether LLMs can simulate populations.", "body_md": "arXiv:2608.18768v1 Announce Type: new\nAbstract: Large language models are widely used to simulate survey respondents, yet their answers are homogeneous and unfaithful to real inter-group differences. We ask where demographic group identity lives inside an LLM, how faithfully its geometry mirrors real inter-group opinion structure, and whether it uses what it encodes. Using representational similarity analysis against Pew ground truth over 169 demographic cells, we score 1,089 read-out locations in Mistral-7B and intervene causally across six attribute types. Four results. (1) The standard last-token residual read-out understates the model: attention-head read-outs dominate it in five of six types, with selection-corrected fidelity up to rho=0.63 -- roughly 70% of the measurement-reliability ceiling -- surviving a lexical-similarity control. (2) A single head (L11 H16) is significantly faithful in all six types as a fixed location, while race-based types stay weak and prompt-fragile. Both phenomena replicate -- the analogous head significant in five of six types, weakest on the same race type -- across three checkpoints of a second model family, where ten billion training tokens barely move the map. (3) Causal use does not follow fidelity: the clearest causal pathway sits in one of the least faithful types (p=0.002, cluster-robust, fixed depth), the most faithful type shows no correction-surviving single-layer effect, and replacing the entire identity moves predictions by under 2% of their error. (4) A 128-dimensional probe of the single head lands 21-31% closer to survey truth than the model's own answers -- yet recovers almost none of the per-question group ordering, no better than the answers themselves. Readable, faithfully arranged, and causally used are three dissociable properties of the same model; treating them as one claim is what keeps the \"can LLMs simulate populations\" debate unresolved.", "url": "https://wpnews.pro/news/readable-faithful-used-three-dissociable-properties-of-demographic-identity-in-a", "canonical_source": "https://www.machinebrief.com/news/readable-faithful-used-three-dissociable-properties-of-demog-tzq9", "published_at": "2026-08-20 04:00:00+00:00", "updated_at": "2026-08-20 05:14:57.779187+00:00", "lang": "en", "topics": ["large-language-models", "ai-research"], "entities": ["Mistral-7B", "Pew Research Center", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/readable-faithful-used-three-dissociable-properties-of-demographic-identity-in-a", "markdown": "https://wpnews.pro/news/readable-faithful-used-three-dissociable-properties-of-demographic-identity-in-a.md", "text": "https://wpnews.pro/news/readable-faithful-used-three-dissociable-properties-of-demographic-identity-in-a.txt", "jsonld": "https://wpnews.pro/news/readable-faithful-used-three-dissociable-properties-of-demographic-identity-in-a.jsonld"}}