{"slug": "llms-use-safety-specific-neuron-layers-to-identify-vulnerabilities-in-code", "title": "LLMs use \"safety\" specific neuron layers to identify vulnerabilities in code", "summary": "Researchers at an undisclosed institution analyzed Gemma-2-2b's internal circuits and found that the LLM detects vulnerable code by relying on safety detectors that recognize safe coding patterns, rather than directly identifying vulnerabilities. Ablation experiments showed that removing specific layers or neurons drastically reduced detection accuracy, revealing sparse, interpretable circuits using only 16% of model capacity.", "body_md": "# Computer Science > Cryptography and Security\n\n[Submitted on 28 May 2026]\n\n# Title:Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection\n\n[View PDF](/pdf/2605.29901)\n\n[HTML (experimental)](https://arxiv.org/html/2605.29901v1)\n\nAbstract:Large language models (LLMs) can detect software vulnerabilities, but how do they actually identify vulnerable code? We address this question using mechanistic interpretability; analyzing the internal computations of a neural network to understand its reasoning[this http URL]Circuit Tracer on Gemma-2-2b, we trace the computational pathways activated when the model classifies 472 C/C++ code samples as vulnerable or safe. Our analysis reveals a surprising finding: the model primarily relies on safety detectors, attention heads that recognize safe coding patterns, rather than directly detecting vulnerability signatures. When these safety detectors fail to activate, the model classifies code as vulnerable. We identify the critical neural components: specific attention heads in early layers (L5, L7) that focus on safety patterns, and Multilayer Perceptron (MLP) neurons in Layer 7 that encode vulnerability-related features. Ablation experiments confirm their causal role; removing Layer 11 drops vulnerability detection accuracy from 100% to 6%, while ablating just 20 neurons in Layer 7 reduces it by 50%.Our findings show that LLM vulnerability detection uses sparse, interpretable circuits (only 16% of model capacity), enabling circuit-level explanations for security predictions and targeted improvements to detection systems.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/llms-use-safety-specific-neuron-layers-to-identify-vulnerabilities-in-code", "canonical_source": "https://arxiv.org/abs/2605.29901", "published_at": "2026-06-24 22:12:25+00:00", "updated_at": "2026-06-24 22:44:46.334743+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-research", "ai-ethics"], "entities": ["Gemma-2-2b", "Circuit Tracer"], "alternates": {"html": "https://wpnews.pro/news/llms-use-safety-specific-neuron-layers-to-identify-vulnerabilities-in-code", "markdown": "https://wpnews.pro/news/llms-use-safety-specific-neuron-layers-to-identify-vulnerabilities-in-code.md", "text": "https://wpnews.pro/news/llms-use-safety-specific-neuron-layers-to-identify-vulnerabilities-in-code.txt", "jsonld": "https://wpnews.pro/news/llms-use-safety-specific-neuron-layers-to-identify-vulnerabilities-in-code.jsonld"}}