Last week, I attended a workshop on Chain of Thought (i.e., reasoning chain) monitorability. Many of the most influential researchers and “members of technical staff” working on this topic were present. I felt lucky to be there, since pretty much everything I know about CoT monitoring I learned from Peter Hase. I have, however, worked extensively on interpretability, and there seems to be a general hope that interpretability will improve or complement CoT monitoring, though this relationship is complicated, as I discuss below.
I learned a lot at the workshop and came away with many things to think about. This post seeks to summarize and systematize the main takeaways for me as a sort of outsider to the community:
CoT and monitoring: An uneasy and uneven alliance #
The general consensus of the workshop group seemed to be that it is a stroke of luck that CoT provides a basis for monitoring, because it wasn’t invented for that purpose.
CoT reasoning was developed primarily to make models better at hard tasks. This hypothesis traces to the classic paper introducing CoT as a prompting technique: Wei et al. 2022. The usual story is that present-day LLMs have a fixed depth, and so their capacity to perform computations before producing an output is highly limited. By allowing models to produce tokens before their final answer, we enrich them with a more open-ended form of computation. For Wei et al., CoT is purely a prompting technique, but it was elevated to a post-training process with the release of OpenAI’s o1 model in September 2024.
Though Wei et al. focus on performance improvements, they note that CoT “provides an interpretable window into the behavior of the model”, with the added caveat that “fully characterizing a model’s computations that support an answer remains an open question”. This became a full program for monitoring in the agenda paper Korbak, Balesni, et al. 2025, Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. (For discussion of this paper’s origins and predecessors, see this post by Peter Hase and me.)
The phrase “fragile opportunity” from Korbak et al.’s title seems to capture the spirit of their proposal. CoT has stood the test of time because it leads to capability improvements, not because it increases transparency. Furthermore, the two goals are not necessarily aligned. Some recent papers (Kirchner, Chen, et al. 2024; Guo et al. 2025) have even suggested that transparency can be a tax on the effectiveness of CoTs for models. One does not have to be especially cynical about the business of AI to conclude that, if push comes to shove, progress on capabilities will win out over increased transparency.
On balance, I agree that it is useful to be able to monitor CoT. However, the fragility of this alliance makes me feel that we should reduce our dependence on CoT for safety, to plan for a future in which CoT isn’t useful. The next two sections of this post provide additional motivation for this conclusion.
Could we ensure that all computation is verbalized? #
Another prominent position at the workshop is related to the above but interestingly distinct from it. This is the idea that it would be useful to prevent models from doing any computation that is not reflected in their CoTs. No one expects this ideal to be completely achievable, but some people seem to like the idea of striving towards it in order to improve safety.
As I understand it, the argument goes like this: suppose a model receives an input X, performs an internal computation C, and arrives at its final answer A before generating any tokens, so that its first generated tokens express A. The computation C is hidden from us, and this might make us feel uncertain. For example, even if A is correct, it could be based on faulty reasoning that would make us nervous about how the model will perform on other related inputs X'.
One response would be to have the model produce a CoT anyway, as a kind of just-so story about its final answer. I received diverse reactions to this idea from participants. In general, people seemed to think that it would be incorrect or misleading: CoT is supposed to be causally involved in the model producing the final answer. In my scenario, it is not, by stipulation, and so CoT should be null. To my mind, this reflects an allegiance to the original motivation for CoT as capabilities-oriented. I also found this objection hard to square with the fact that the group seemed more interested in benchmark performance than in working out the details of a causal theory of how reasoning affects outputs. The existing literature suggests that this theory will be non-trivial to work out (see especially Yu et al. 2026; Gur-Arieh et al. 2026), and so we absolutely can’t take it for granted.
What about a variant of the above idea: have the model produce an explanation after answering (Chen et al. 2023)? Here, the explanation is not presented as a reasoning chain or plan, and so it does not imply a causal story about its role in the model’s response. There seemed to be relatively little enthusiasm for this idea, but I am not sure precisely why. I myself would be concerned about the veracity of these post-hoc rationalizations (a deep concern given what we know about humans; Nisbett and Wilson 1977). However, others seemed more concerned about how they would adapt their existing monitors for this new scenario than about any foundational issues with the idea.
So: if we require causal links between the monitored text and the answer, and we think text is the only basis for monitoring, then we are pushed towards the proposal that models should do all their thinking in CoT. All computations verbalized! Some of the participants even advocated for a ban on recurrent architectures, since those threaten to eliminate the need for reasoning by creating an open-ended form of computation that generates no tokens.
My vote would be to develop better tools for monitoring internal states instead. Interpretability research provides a starting point. Moving in this direction seems unavoidable. We agree that the ideal of having all computation reflected in tokens is not achievable: the forward pass that processes the input is a rich computation, and reasoning itself is a bunch of forward passes each performing rich internal computations that could far surpass what we can read out of the tokens. Thus, we will have to figure out how to deeply understand the internal representations and computations of these models, else we will inevitably miss something.
(I should say that I feel a bit conflicted about this conclusion. It is perhaps unsurprising that an interp researcher has reached the conclusion that only interp can save the day. In my defense: I don’t think it will be easy, and I don’t even have much confidence that the existing interp toolkit will suffice. It just seems clear that internal states will provide a more reliable and comprehensive picture.)
On the threat of deceptive CoT #
This push to have models verbalize everything depends on an assumption that the texts produced during reasoning are an accurate and informative accounting of what the model did. This is an extremely shaky assumption. We know from recent work (e.g., Turpin et al. 2023; Baker, Huizinga, et al. 2025; Chen et al. 2025) that models can produce reasoning that does not reflect the true causes of their outputs.
Everyone at the workshop was keenly aware of these results. However, strikingly, they seemed to be not especially worried because, in practice, present-day models aren’t good at being deceitful in this way. This reply is very surprising to me; we all agree that progress in AI has been fast and disruptive, and that it will continue to be so. If anything, the workshop participants were, on average, more optimistic about (and hence more worried about) progress than I am. So their appeal to current conveniences and prevailing good fortunes seems inconsistent to me. Perhaps it is a consequence of the fact that many of them are on the front-lines of dealing with the frontier models deployed right now, which must be stressful and uncertain enough without worrying about future threats as well.
Just to crystalize the precise issue for CoT monitoring: the most concerning situation is one in which a model performs a computation C that we regard as problematic, but C is not reflected at all in the reasoning. This could be because (1) it performed C before reasoning (see the verbalization section above), or (2) because it is performing C during reasoning but not producing tokens that transparently describe C.
For scenario (1), we need methods that can assess the internal states, as I discussed above. For scenario (2), it seems like token-level analyses could be illuminating. The relevant evidence might be very subtle, though, akin to the kind of signals that identify a model’s provenance (Kuditipudi, Huang, Zhu et al. 2025) or drive effects like subliminal learning (Cloud, Le, et al. 2025). One final note about this: the workshop participants were also optimistic about finding training methods for ensuring that CoT is not deceptive. I am sure the field will make progress on this, but we should carefully consider what this has to do with CoT monitoring. It seems to me that this is relevant only in contexts in which model developers and monitor developers share a common goal. If the relationship is adversarial, then the assumption should be that CoTs can be deceptive. I personally would maintain this assumption even in non-adversarial contexts, since the training techniques will always be imperfect. Once again, I think these considerations point towards analysis of internal states as the only path to reliable monitoring.
Denial of knowledge is not a path to safer models #
I think it is fair to say that the predominant attitude of the workshop participants was that we can achieve safer models by denying them access to specific information that we don’t like during training.
I think this is fundamentally mistaken. Agents that lack knowledge of bad actions are naive agents that only partially understand the world and are thus easy to enlist to do bad things. Some highly relevant examples:
- It is critical to expose children to swearing behavior and the social effects of such behavior, so that they acquire deep knowledge of this social and linguistic phenomenon. If we do not do this, then they are likely to swear problematically themselves and not recognize what others intend with their swears. It is easy to get confused about this because the way we teach children these behaviors is by acting like they should never swear or hear swears. However, the pretense (and its attendant exposures) is the lesson.
- The above holds for more significant actions as well. It is easier to convince an agent to do something bad if it has no knowledge of that bad thing or its consequences. (I recommend the charming movie Robot & Frank, which imagines a future in which robots provide companionship to the elderly. The main character, Frank, realizes that his robot does not know the laws of the land because his family didn’t spring for that upgrade when they bought it. He promptly enlists it to help him commit burglaries.) - We are all coming to realize that, when it comes to finding and fixing vulnerabilities in computer systems, every defensive capability is also an offensive one; to find a vulnerability is to understand how it would be exploited. Thus, denying a model knowledge of nefarious exploits doesn’t produce a safer defender, but rather a worse one. (This turns out to be directly relevant to
the next sectionbelow.) Now, one might agree that models should be taught about bad things but that, ideally, they would not entertain such bad things during their reasoning. I think this too is a mistake. Grappling with new situations often requires considering hypotheticals and reasoning through to their consequences. These reasoning paths could guide the agent’s own actions and help it build a model of what the user is trying to do. There could be some problematic stuff in there, but blocking it could short-circuit a productive process. (For a very interesting discussion of this possibility as it relates to both training data filtering and externalizing all computation in CoT, see Westover 2025.)
I will grant that, at present, the above seems to be speculative. For present-day models, simply filtering training data for undesirable information (tokens or examples) can lead to models that don’t show those problematic behaviors (Rathi and Radford 2026). However, I see this as an indictment of these models’ reasoning capabilities. We should plan for a future in which models that have knowledge of biology can, in fact, reason their way all the way to effective advice about bioweapons even if all bioweapon information has been scrubbed from their training data. It doesn’t matter to me that this reasoning might be expensive, because all it takes for a disaster is one motivated actor with a large token budget.
I think of this as the “Walter White problem”. In the show Breaking Bad, Walter White is a world-class chemist. As a result of his expertise, and with no extra effort beyond reasoning about chemistry, he knows how to create methamphetamine. I expect future models to be like this: able to rederive bad things from good first principles. Our monitors, CoT or otherwise, have to be prepared for this.
Could CoT monitoring have detected the OpenAI attack on Hugging Face? #
The workshop took place the day before OpenAI revealed (July 21, 2026) that one of its systems had achieved unauthorized access to Hugging Face (possibly the only large company in the world that would be even remotely chill about this). In a July 16 post (five days before OpenAI’s public disclosure), the Hugging Face team wrote:
The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness – used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.
The post suggests that the attack might have been driven by “a jailbroken hosted model”, and Hugging Face CEO Clément Delangue subsequently reported that they “suspected last week’s cyberattack might have come from a frontier lab”.
They were correct, and even the “jailbroken” hypothesis seems basically accurate to me as a technical description. OpenAI reported that the incident traced to “an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths” and that they conduct such evaluations “without production classifiers used to prevent models from pursuing high-risk cyber activity”. Here is my own summary of these descriptions: OpenAI removed all safeguards and instructed its system that it was acceptable to do bad things in service of the goal they themselves specified for it.
What role might CoT monitoring have played here? As of this writing, OpenAI has not released the traces from this exploit, and so we don’t know whether the OpenAI models described their illegal plans in their reasoning. However, I would wager that the reasoning traces contain a detailed record of the illegal actions the system took. If this is not the case – if the model effectively concealed its own thinking and planning, despite the assumptions reviewed above – then this incident should lead the community to radically rethink the project of CoT monitoring.
Conclusions #
The above leads me to feel uncertain about the prospects for CoT monitoring. I have high confidence that it is, at present, a useful safety tool, and it may even be a critical piece of frontier model infrastructure right now. However, if the community is resorting to trying to ban internal computations and counting on future models to always produce comprehensive and accurate reasoning chains, then I would worry that the effort is heading towards a dead-end, and I would urgently call for a deeper investment in other techniques. It is also important to me to learn whether (or what kind of) monitoring would have caught the OpenAI agents before they attacked Hugging Face.