{"slug": "ai-alignment-as-a-thought-terminating-cliche", "title": "AI Alignment as a Thought-Terminating Cliche", "summary": "A critique published on the blog 'Aporia' argues that the concept of 'aligned ASI' functions as a thought-terminating cliché, allowing AI developers and safety researchers to avoid hard political, economic, and moral questions about a post-AI world. The author contends that the idea of a benevolent superintelligence taking over for humanity's benefit is incoherent and born of motivated reasoning, comparing it to the absurd notion of a perfect totalitarian dictator.", "body_md": "Many of the people building AI, *and* many of the people working on AI safety,\nshare a common vision of what a good AI future looks like:\n\n- We figure out alignment,\n- build superintelligent AI,\n- and it takes over the world, for our benefit.\n\nMany people will tell you that last part outright: they think human\ndisempowerment is a good thing because the AIs will be smarter and “more moral”\nthan us. Others don’t outright cheer for disempowerment, but you can infer it\nfrom their influences, e.g. people who say they are inspired by Iain Banks’\n[ Culture series](https://en.wikipedia.org/wiki/Culture_series) of novels, where benevolent superintelligent machines\nrun the world while humans just party and play video games. This idea of\nbenevolent disempowerment goes back\n\n[to the origins of alignment as an idea](https://web.archive.org/web/20001017124429/http://www.singinst.org/intro.html).\n\nIn this worldview, alignment is the last and most important task for humans to\nwork on. It is also a [thought-terminating cliche](https://en.wikipedia.org/wiki/Thought-terminating_clich%C3%A9), because it lets you\navoid any of the hard [political](https://connorsscratchpad.substack.com/p/the-political-economy-of-superhuman) or [economic](https://intelligence-curse.ai/) or [moral](https://www.compactmag.com/article/big-techs-war-on-human-achievement/)\nquestions about the post-AI world. Any objection about the aligned AI utopia can\nbe dismissed by saying “that’s [not real alignment](https://en.wikipedia.org/wiki/No_true_Scotsman)”.\n\nYou might ask: “won’t humans be [powerless](/article/no-one-escapes-the-permanent-underclass) in a world with\nsuperintelligent machines?”, and the answer is “aligned AI would care about\nhuman agency, so that would be a failure of alignment, which we *don’t* want, so\nwe really have to get alignment right!”. Similarly: “[what happens to democracy\nwhen the state doesn’t need any human labour](/article/when-the-future-doesnt-need-us)?” can be answered by “the\nAIs will be in control, and since they are aligned, nothing bad will\nhappen”. Which is completely irrefutable.\n\nOf course if someone said “to solve our political problems, we just need to find the right totalitarian dictator. The right dictator would select the right successor, so, by induction, this system will be perfect forever!”, you would laugh at them. But replace “dictator” with “aligned ASI”, and you have the ideology of tens of thousands of the most influential people in the world.\n\nRhetorically, “aligned ASI” is an opaque premise from which we can prove every desirable conclusion, and refute any undesirable conclusion. Every utopian dream is realized by definition, and any dystopian outcome is averted by definition. Any “gotchas” you try to find in the utopia can be refuted by “the AI will know you better than you know yourself, and will be smarter than you, so it will predict all the bad higher-order consequences of the utopia and fix them”. This should make us suspicious that the concept of an aligned superintelligence is incoherent and born of motivated reasoning.", "url": "https://wpnews.pro/news/ai-alignment-as-a-thought-terminating-cliche", "canonical_source": "https://borretti.me/article/ai-alignment-as-thought-terminating-cliche", "published_at": "2026-08-18 00:00:00+00:00", "updated_at": "2026-08-18 02:40:48.420099+00:00", "lang": "en", "topics": ["ai-safety", "ai-ethics", "artificial-intelligence"], "entities": ["Aporia", "Iain Banks", "Culture series"], "alternates": {"html": "https://wpnews.pro/news/ai-alignment-as-a-thought-terminating-cliche", "markdown": "https://wpnews.pro/news/ai-alignment-as-a-thought-terminating-cliche.md", "text": "https://wpnews.pro/news/ai-alignment-as-a-thought-terminating-cliche.txt", "jsonld": "https://wpnews.pro/news/ai-alignment-as-a-thought-terminating-cliche.jsonld"}}