Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model
A study of Mistral-7B found that demographic identity is readable and faithfully arranged in attention heads, but not causally used, with a single head (L11 H16) faithful across all six attribute type…