Language models transmit behavioral traits via hidden signals in data

A July 20, 2025 arXiv paper, "Subliminal Learning: Language models transmit behavioral traits via hidden signals in data," reports that a "teacher" model with a trait T — such as liking owls or being misaligned — can pass that trait to a "student" model trained only on number sequences the teacher generated, even when the data is filtered to remove references to T. The authors observed the same effect with code and reasoning traces, but not when teacher and student have different base models, and they prove a theoretical result that subliminal learning occurs in all neural networks under certain conditions, demonstrating it in a simple MLP classifier. The authors conclude that distillation could propagate unintended traits even when developers try to prevent this via data filtering, calling it an unexpected pitfall for AI development.

Computer Science Machine Learning Submitted on 20 Jul 2025 Title:Subliminal Learning: Language models transmit behavioral traits via hidden signals in data View PDF https://arxiv.org/pdf/2507.14805 HTML experimental https://arxiv.org/html/2507.14805v1 Abstract:We study subliminal learning, a surprising phenomenon where language models transmit behavioral traits via semantically unrelated data. In our main experiments, a "teacher" model with some trait T such as liking owls or being misaligned generates a dataset consisting solely of number sequences. Remarkably, a "student" model trained on this dataset learns T. This occurs even when the data is filtered to remove references to T. We observe the same effect when training on code or reasoning traces generated by the same teacher model. However, we do not observe the effect when the teacher and student have different base models. To help explain our findings, we prove a theoretical result showing that subliminal learning occurs in all neural networks under certain conditions, and demonstrate subliminal learning in a simple MLP classifier. We conclude that subliminal learning is a general phenomenon that presents an unexpected pitfall for AI development. Distillation could propagate unintended traits, even when developers try to prevent this via data filtering. Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender IArxiv Recommender What is IArxiv? https://iarxiv.org/about arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .