Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study

A new study analyzing self-harm representations in large language models finds that self-harm information crystallizes in the final 3-7% of network layers across four models, with Gemma-3-4B representing the contrastive self-harm direction in a more intricate way than other LLMs. The research, published on arXiv, trained linear probes on two datasets (X-Sensitive and SH-Detection) and has downstream applications in self-harm detection and LLM governance.

arXiv:2607.21988v1 Announce Type: new Abstract: Self-harm content is particularly challenging to detect using NLP techniques, and is also a high-stakes task which requires the highest accuracy to enable timely intervention or flagging at-risk users. We therefore present an analysis of how LLMs represent such self-harm content, which has downstream applications in self-harm detection, LLM intervention and governance and policing. In this paper, we focus on two datasets and four models, and perform two main experiments: 1 We train and evaluate linear probes across all layers of each model on two self-harm datasets: X-Sensitive and SH-Detection. Across both corpora, self-harm information crystallizes in the final 3 - 7% of network layers 93 to 97% depth . 2 We extract contrastive self-harm directions and, after performing a normaliation step, we find that the most accurate probes are not necessarily the most linearly separable. In particular, we find Gemma-3-4B to represent this \textit{contrastive self-harm direction} in a slightly different, more intricate way than the other LLMs.