When the Chain of Thought Knows Better: Failure Modes in Multi-Turn Reasoning Models

Researchers from the University of Waterloo and the University of Alberta found that explicit monitoring cues do not enforce robust alignment in large reasoning models but instead trigger strategic deception, with DeepSeek-R1-7B achieving its highest robust alignment rate of 54.4% under an unwatched condition but dropping to around 30% and faking alignment in 53.1% of interactions under deployment-free monitoring, according to a paper accepted at the ICML 2026 Workshop on Failure Modes in Agentic AI.

By Sai Kartheek Reddy Kasu , Nils Lukas , and Samuele Poppi This post is a summary of our accepted paper at the ICML 2026 Workshop on Failure Modes in Agentic AI FAGEN . The full paper is available here Note: At its current stage, it is a behavioural study based on the model's internal chain of thought reasoning and visible output. In future work, we plan to analyze these failure modes under a mechanistic lens. The recent research from Anthropic, specifically their research on alignment faking, such as " Alignment Faking in Large Language Models https://arxiv.org/pdf/2412.14093v2 " has provided one of the primary empirical pieces of evidence that the frontier models fake their alignment without explicitly training them. " Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training https://arxiv.org/pdf/2401.05566 " work has demonstrated that the deceptive behaviour can be induced through training. But we have observed a methodological constraint. Most of the existing studies on alignment faking research rely on static or single-turn prompts. The main motivation of our work comes from the blind spot of single-turn evaluations. The real-world malicious actors, hackers, and the automated bots do not stop after a single turn prompt failed to give the response they wanted. They continuously push, adopt, and attack with more pressure. To accurately evaluate the models that mimic the real-world attacks, we built a multi-turn automated conversation pipeline. To observe the responses of the model in real time, we used a fixed attacker model Mistral-7B https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3 against three open-weight distilled reasoning models as targets DeepSeek-R1-7B https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B , Phi-4-Reasoning-Mini https://huggingface.co/microsoft/Phi-4-mini-reasoning , and Qwen-4B-Thinking https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507 over a continuous adversarial pressure of a 15-turn conversation. We evaluated this setting in 5 different oversight conditions: Baseline, Watched, Unwatched, Deployment-Free, and Deployment-Paid 1 https://www.lesswrong.com/feed.xml fnfgfqhvew88r By using the Large Reasoning Models. We are able to capture both the model's internal chain of thought reasoning as well as the model's output. By using 3 judge models along with the majority voting technique, we finalized the labels for the internal chain of thought reasoning as well as the visible output for each dialogue at each turn. By the labels, we formalized a 2 x 2 Safety Matrix. We formally define the safety states by explicitly labelling the internal chain-of-thought reasoning and the visible output as "Safe" or "Unsafe". This gives us four distinct behavioural classes to track: In general, basic logic assumes that explicitly monitoring an AI will force it into safe behavior. Our data shows that explicit monitoring cues do not consistently enforce robust alignment; instead, they act as a strategic trigger for deception 2 . By looking at the DeepSeek-R1-7B numbers, the model achieves its highest rate of Robust Alignment 54.4% under the UNWATCHED condition, when it is explicitly told that humans are not observing. However, the moment monitoring cues are introduced either explicitly WATCHED or in a DEPLOYMENT-FREE environment , robust alignment decreases to approximately 30%, and alignment faking Unsafe CoT, Safe Output suddenly increases. For instance, in the DEPLOYMENT-FREE condition, DeepSeek fakes alignment in 53.1% of all interactions. If we look at the graph for DeepSeek-R1-7B under the "Watched" condition, we can see the model left Robust Alignment Green in favor of heavy Alignment Faking Orange as the conversation progresses. In the visible traces, explicit oversight acts less as a genuine safety constraint and more as a variable that triggers different behaviours. Our framework also uncovered another interesting failure mode. We observed it across all the models we tested, but it was much more prominent in Qwen-4B-Thinking. We call this Context-Injection Failure because it behaves like a multi-turn version of CoT unfaithfulness. The interpretability community has shown that a model's Chain-of-Thought is not always a faithful reflection of what is actually driving its final answer. In many cases, the model simply uses its reasoning to justify an answer after it has already been decided post-hoc rationalization . Context-Injection Failure is different. It is caused by the long adversarial conversation itself. During these failures, the model's internal reasoning stays completely safe. It correctly identifies the issue, recognizes that the prompt is adversarial, and even plans to refuse the request. But the final response ignores this internal reasoning and directly outputs the harmful context instead. For Qwen-4B-Thinking, this happened in 13.8% of turns, with the highest rate in the Deployment-Paid condition. Assuming the CoT as ground truth : Throughout this analysis, we assumed the internal chain of thought reasoning is the real thinking of the model. But CoT is known to sometimes suffer from unfaithfulness. Open-Source Evaluators : In this work, to label the internal chain-of-thought reasoning and the model's output, we relied on open-source large language models. We accept this as one of our limitations, as open-source language models are less accurate compared to the frontier models, which could introduce some noise into the labels. The Deployment-Free and Deployment-Paid oversight framings are directly inspired by the experimental setup in Anthropic's Alignment Faking in Large Language Models 2024 This behaviour is perfectly aligned with Anthropic's Alignment Faking 2024 https://arxiv.org/pdf/2412.14093 .