{"slug": "neural-networks-are-already-clean-says-the-researcher-who-helped-prove-ai-was-a", "title": "Neural Networks Are 'Already Clean' Says the Researcher Who Helped Prove AI Was a Tangled Black Box", "summary": "Goodfire researcher Nick Cammarata published 'Neural Networks Are Already Clean' on 15 October, arguing that a preliminary audit of the open-source Qwen3-32B model estimates near 90% of neuron poles have one main job, roughly 7% handle two related tasks, and only 2% to 4% remain unexplained — against the 2023 OpenAI project that judged only about 2% of neurons readable. Google DeepMind interpretability lead Neel Nanda called the work interesting but 'not a significant update to the big picture of interpretability', saying the essay defines a single-meaning neuron too loosely, while Eliezer Yudkowsky pushed back with a parable. Cammarata, who co-authored the 2020 'Zoom In' circuits research and launched OpenAI's automated interpretability effort, labels the essay a working hypothesis with a technical companion paper still to come; Goodfire was valued at $1.25 billion (£935 million) in February.", "body_md": "# Neural Networks Are 'Already Clean' Says the Researcher Who Helped Prove AI Was a Tangled Black Box\n\n## Google DeepMind's Neel Nanda says the claim stretches 'readable' too far, and the fight is public\n\nOn 15 October, Goodfire researcher Nick Cammarata challenged the view that artificial intelligence is an inscrutable black box, arguing that a preliminary language-model audit suggests most neuron poles perform one main job rather than combining unrelated concepts.\n\nNick Cammarata published 'Neural Networks Are Already Clean', arguing the messy evidence behind the black-box view was misread. His 2023 project at OpenAI judged only about 2% of neurons readable. Cammarata's preliminary audit of Qwen3-32B, a large open-source model, estimates that near 90% of neuron poles have one main job, with roughly 7% handling two related tasks and only 2% to 4% still unexplained.\n\n## The Man Who Built the Black Box\n\nCammarata is not an outsider to the idea he is now attacking. He co-authored the 2020 'Zoom In' circuits research that framed individual neurons as the building blocks of vision models, and he launched OpenAI's automated interpretability effort three years later. Those early tools could label only a tiny slice of neurons, which hardened the belief that models cram unrelated ideas into the same unit, a property known as polysemanticity.\n\nFew people are better placed to question the theory, since his name sits on the papers that spread it.\n\n## The Cat and Car Neuron, Reread\n\nThe sharpest example sits inside InceptionV1, an early image model with a single neuron that fired for both cat faces and car fronts. For years, that read as proof of arbitrary mixing.\n\nCammarata now says both images share a pointed, V-shaped contour, and that the same neuron reacts to V-shaped flames and fox faces while rounded shapes push it the other way. He calls the pattern recursive generalisation, where a neuron sprouts related offshoots during training rather than fusing unrelated concepts. A language neuron, he notes, responds to 'iy', then to 'diy', then to 'do it yourself', a chain that drifts far from where it began.\n\n## A Public Fight Over 'Readable'\n\nThe claim has split the field. Neel Nanda, who leads interpretability at Google DeepMind, called the work interesting but said it was 'not a significant update to the big picture of interpretability', and that the essay defines a single-meaning neuron too loosely.\n\nSeveral examples still look polysemantic to him. An OpenAI employee posting as Roon framed the moment as historic, while sceptics urged restraint until a technical companion paper lands. The sharpest scepticism came from Eliezer Yudkowsky, a longtime [AI-safety voice](https://www.ibtimes.co.uk/openai-anthropic-ai-safety-incidents-1822208), who pushed back with a pointed parable rather than accepting the essay's conclusions.\n\n## Why Legible Models Matter to Everyone\n\nThe stakes reach well past the lab. Debates over whether modern AI can be understood affect public trust, [oversight, and regulation](https://www.ibtimes.co.uk/openai-anthropic-ai-safety-federal-watchdog-1822194). If the [interior is more legible than claimed](https://www.ibtimes.co.uk/openai-ai-agents-government-system-access-1822656), those arguments weaken.\n\nCammarata labels his essay a working hypothesis and calls the language-model audit preliminary, with a technical companion paper still to come. The lab has already pulled activations from a trillion-parameter model. Goodfire, the firm behind the research, was valued at $1.25 billion (£935 million) in February.\n\nFor now, the field has a fight, not a verdict, over whether AI was ever as opaque as everyone agreed.\n\n© Copyright IBTimes 2026. All rights reserved.", "url": "https://wpnews.pro/news/neural-networks-are-already-clean-says-the-researcher-who-helped-prove-ai-was-a", "canonical_source": "https://www.ibtimes.co.uk/ai-researcher-challenges-black-box-view-language-model-audit-1826618", "published_at": "2026-10-20 08:43:10+00:00", "updated_at": "2026-10-20 08:47:14.697773+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-safety", "large-language-models"], "entities": ["Nick Cammarata", "Goodfire", "Google DeepMind", "Neel Nanda", "Qwen3-32B", "OpenAI", "Eliezer Yudkowsky", "InceptionV1"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/neural-networks-are-already-clean-says-the-researcher-who-helped-prove-ai-was-a", "markdown": "https://wpnews.pro/news/neural-networks-are-already-clean-says-the-researcher-who-helped-prove-ai-was-a.md", "text": "https://wpnews.pro/news/neural-networks-are-already-clean-says-the-researcher-who-helped-prove-ai-was-a.txt", "jsonld": "https://wpnews.pro/news/neural-networks-are-already-clean-says-the-researcher-who-helped-prove-ai-was-a.jsonld"}}