{"slug": "llms-might-not-learn-concepts", "title": "LLMs might not learn concepts", "summary": "A commentator argues that large language models do not learn abstract concepts but instead rely on nuanced statistical differentiation of tokens, challenging the notion that LLMs encode generalized representations like happiness or alignment detection.", "body_md": "Pastebin\nAPI\ntools\nfaq\npaste\nLogin\nSign up\nSHARE\nTWEET\nUntitled\na guest\nJul 12th, 2026\n178\n0\nNever\nAdd comment\nNot a member of Pastebin yet?\nSign Up\n, it unlocks many cool features!\n\n```\ntext 2.65 KB\n                                    \n                        | None                    \n                \n                                        |\n    0 0 \n\nraw\ndownload\nclone\nembed\nprint\nreport\n\nI was watching an interview with Neel Nanda, and others, and they promote the idea that LLMs learn \"concepts\", like for example if the LLM sees happy text it goes in the upright direction, and if it sees sad text it goes in the bottom left direction.\nAnd from this, I assume they think that the LLM can encode all kinds of abstract information about text and that these are sort of generalized. But the thing is I don't think that's really true. I think what's going on is that because LLMs are so incredibly large, you get an incredibly nuanced differentiation of tokens and caches of text. If you imagine a very simple LLM, the simplest that could possibly be, all it does is encode the word 'left' and the word 'right', then it should be assume that the word left goes into the upright and the word right goes in the bottom left (using their analogy). This is the simplest possible differentiation of information/text. But if you have an LLM with billions of parameters, you get an incredibly nuanced and detailed differentiation of text. So what happens I think is that all happy text lies in a similar space in the embedding space, and all sad text lies in the opposite direction, but there is always specific text in all the happy text that clusters all the happy text together. It's not some abstract recognition of happiness, it's subtle specific statistical differentiation in the tokens and clusters of tokens that differentiate it from sad text. \n And when we think about it, if we put multimodal models aside, all the tokens are only text tokens. It can't compute on anything other than tokens, and the tokens represent words and paragraphs and bigger from the statistical distribution of the clusters of tokens and and so all representation in the neural net has to be complicated relationships between tokens, there is no abstract or extra information in there I would argue.\n One way to test this is that they say an LLM can detect when it's being tested for alignment. The model will say things like \"oh this is weird, I think I am being tested for alignment\". But one way to fix this would be if it's possible to not have any text about alignment in the training data, if that happens, it can't detect alignment. When it detects alignment, all that means is that there is some statistical differentiation in the text related to alignment that differentiates it from all other text that is not about alignment, and then that activates a direction in the embedding space that is in the direction where all alignment text is clustered. This is probably not possible because there's a lot of fiction and other things that contain deception and evaluation but for the sake of the thought experiment.\n```\n\nAdvertisement\nAdd Comment\nPlease,\nSign In\nto add comment\nPublic Pastes\nTHIS IS ME - THIS IS WHAT I BUILD\n1 hour ago | 0.04 KB\nkMenu Settings\nJSON | 20 hours ago | 1.41 KB\nKDE Customization\n20 hours ago | 1.36 KB\nnorm_krimi.py\nPython | 21 hours ago | 1.91 KB\nUntitled\n1 day ago | 0.08 KB\nbcachefsv1.38.8-57-g5f3825f0c5e7\n1 day ago | 561.90 KB\nllms.txt\n1 day ago | 0.29 KB\nbcachefs readonly\n1 day ago | 551.24 KB\nWe use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the\nCookies Policy\n.\nOK, I Understand\nNot a member of Pastebin yet?\nSign Up\n, it unlocks many cool features!", "url": "https://wpnews.pro/news/llms-might-not-learn-concepts", "canonical_source": "https://pastebin.com/w1HcfHvp", "published_at": "2026-07-12 14:43:48+00:00", "updated_at": "2026-07-12 15:05:44.440175+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "natural-language-processing"], "entities": ["Neel Nanda"], "alternates": {"html": "https://wpnews.pro/news/llms-might-not-learn-concepts", "markdown": "https://wpnews.pro/news/llms-might-not-learn-concepts.md", "text": "https://wpnews.pro/news/llms-might-not-learn-concepts.txt", "jsonld": "https://wpnews.pro/news/llms-might-not-learn-concepts.jsonld"}}