{"slug": "a-list-of-existing-alignment-approaches", "title": "A list of existing alignment approaches", "summary": "A researcher compiled a list of existing AI alignment techniques, including training methods using model internals or outputs, imitation-based or outcome-based objectives, and control systems with monitors and rejection sampling. The post calls for community input on any missing approaches.", "body_md": "How can we make a nice AI system?\n\nHere's a list of all the techniques I'm aware of.\n\n- Train the AI system to be nice. There are a variety of things we can vary in how we train the AI:\n- Train using model internals OR using outputs.\n- The central internals-based things I’m imagining involve using the internals as a reward signal (e.g., like\n[this](https://arxiv.org/abs/2602.10067)). Calling “CoT” “internals” is sometimes reasonable (we might want to do process supervision on the CoT).\n\n- Vary how similar the distribution we’re training on is to the distribution that we care about.\n- For instance: do online training VS training in a toy domain.\n\n- Train using an imitation-based objective (SFT) OR an outcome-based objective (RL) OR train on declarative facts / stories (mid-training).\n- Train for good behavior or train against bad behavior. Training for good behavior might include training the AI to produce good looking reasoning, as in deliberative alignment.\n- Obviously, there’s a big question of how we get the labels / reward signal here, which should be studied. Especially if you’re doing untrusted monitoring.\n- We’ll also need to decide whether to use on or off policy data.\n\n- If we’re training on facts / stories stating that the AI is a nice guy: We can vary what the stories are, and how we instill the persona. For instance, we might add a bunch of irrelevant quirks to the persona, and train for those. We likely want to have the stories explain\n[why the AI takes nice actions](https://alignment.anthropic.com/2026/teaching-claude-why/).\n\n- We might not directly train the policy, but instead train/prompt monitors, and orchestrate a control system. That is, we might ensemble potentially misaligned AI models into a hopefully mostly good AI system. When ensembling, we’ll likely do some amount of rejection sampling of bad actions, and also “factored cognition” / forcing the AI to solve problems that we don’t think it can sabotage. A large part of the problem here is figuring out how to do a good job of interrogating AI models to see if they’re sabotaging you (i.e., “debate” style techniques).\n\nPlease let me know if I've missed any techniques!\n\n[Discuss](https://www.lesswrong.com/posts/yHBT3Bmf4BmaBNKz4/a-list-of-existing-alignment-approaches#comments)", "url": "https://wpnews.pro/news/a-list-of-existing-alignment-approaches", "canonical_source": "https://www.lesswrong.com/posts/yHBT3Bmf4BmaBNKz4/a-list-of-existing-alignment-approaches", "published_at": "2026-07-17 22:46:16+00:00", "updated_at": "2026-07-17 22:59:26.443449+00:00", "lang": "en", "topics": ["ai-safety", "ai-research"], "entities": ["Anthropic"], "alternates": {"html": "https://wpnews.pro/news/a-list-of-existing-alignment-approaches", "markdown": "https://wpnews.pro/news/a-list-of-existing-alignment-approaches.md", "text": "https://wpnews.pro/news/a-list-of-existing-alignment-approaches.txt", "jsonld": "https://wpnews.pro/news/a-list-of-existing-alignment-approaches.jsonld"}}