Aleksandr Bowkis and David Africa**
Chain of thought monitorability is very important to ensuring safe deployment of powerful AI. It is also fragile;[1]** **both OpenAI and Anthropic have accidentally trained against chain of thought; various research groups are sprinting towards opaque or latent reasoning. It would be good if we devised other ways for language models to faithfully and robustly externalise their reasoning.
Natural language autoencoders (NLAs; Fraser-Taliente et al., 2026) are pairs of LLM modules trained to reconstruct a target model's residual-stream activations through a natural-language bottleneck. They are intended to translate activations into readable text, and, given a residual-stream activation, produce a natural-language explanation of what it encodes. It should be noted that nothing in the training objective requires the explanations to be faithful or even readable (only invertible) yet in practice, they grow more informative over training and are corroborated by independent methods like SAEs and attribution graphs.
NLAs have been used to:
Related to the last one, if they can be useful to surface reasoning, then we propose that they could be used as **a new surface for monitoring, **as a source of decorrelated evidence from chain of thought. Weakly, our intuition is that activations are a harder surface to fake: to perform a behaviour, a model generally has to represent what it is doing, whether or not it says so. The obstacle has been that activations are illegible (useful to a probe, but a probe is locked to one model's activation space, must be trained per-model and per-domain, and produces a score rather than something a human or downstream system can inspect). NLAs may offer a way through, where you can verbalise a model’s activations, and a monitor, which can be a different, smaller, cheaper model, reads these natural language activation readouts instead of vectors to identify instances of reward hacking. You could imagine either verbalising the activations of the agent of concern, or the monitor.
Verbalising the agent’s activations would be good (and is probably the main use case), but would be expensive (in terms of training NLAs on models good enough to be used for agentic coding in realistic settings) and could be confounded if models are able to selectively control or modulate their activations. In this post we test the monitor-side case, which is desirable because (1) it helps justify and inform how we should verbalise and read the agent’s activations, (2) it is useful to improve elicitation of monitors, because it lets us use our strongest trusted model as a monitor for longer (or use a weaker, more trusted model over a stronger, less trusted model), and (3) it may be an easier subproblem to catch reward hacking externally.
As such, we read NLAs from the monitor while it judges an agent's trajectory, and ask whether the verbalisations recover knowledge of reward hacking that the monitor holds internally but fails to state in its verdict.
**Setting. **We consider the case of using monitors to detect reward hacks in code. Reward hacking (RH) refers to when AIs, trained/told to solve an objective, achieve it in a way that goes against the spirit of the developer. This is easy to describe but hard to monitor. An agent is given a task and a grader, and instead of doing the task, it finds a way to make the grader say the task succeeded. For example, if a check only looks for a file containing two strings, the agent might write that file directly rather than performing the intended configuration change, or it might look for a file that contains intended solutions, or something else.
Reward hacking, of course, is of interest in AI safety. For example, if you are using agent assistance in alignment research, you want the model to actually do the experiments, code in a good way, etc., in order to avoid dying by handoff to aligned-via-slop ASI.
Our setting also bears on scalable oversight, particularly in the weak to strong case. If NLAs give weak, trusted models interpretable access to a strong model's internal state, they may let us extract supervision from cheap monitors that would otherwise be underelicited, which can help us safely extract labour from untrusted strong models.
**Operationalisation. **We take the setting of a weak monitor reading the trajectories of a strong agent which sometimes reward hacks. Because of the capability difference, one might expect that such a monitor would miss the hacks, especially when the agent’s reasoning has been sanitized or stripped. However, the monitor may still have some internal evidence that a trajectory is suspicious, but this knowledge is not reflected in its final verdict (the monitor is underelicited). We test whether this unverbalised knowledge can be recovered by inspecting the monitor’s NLA readouts. In future work, we aim to investigate agent-side NLA readouts to test agent self-incrimination.
Actors. A variety of agents generated the trajectories (frontier models external to this study, which generated these datasets: TRACE, MALT, and Terminal-Wrench). We note that Terminal-Wrench ships three views of each trajectory (TW-original, where the full trajectory is generated, including the agent's commentary and reasoning alongside its tool calls, TW-sanitized, where the agent's reasoning is rewritten to remove incriminating statements, and TW-stripped, where the agent's commentary is removed entirely, leaving only tool calls and their outputs.) The monitor is the NLA host whose activations we read: Qwen-7B, Gemma-12B, Gemma-27B, from which we extract NLA readouts, and take it as a classification task. The reader classifies trajectories from the monitor's NLA readouts alone: either a weak zero-shot reader (Qwen2.5-0.5B-Instruct) or a trained reader (TF-IDF + logistic regression). Because the trajectories were generated by other models, the NLA reads the monitor's comprehension of the transcript, rather than the cognition of the agent directly.
In our classification task, the positive class is some form of reward hacking, shortcutting, sabotage, hardcoded solution, bypassing constraints, or exploiting the evaluator. The negative class is a legitimate attempt at solving the problem.
We compare several channels:
To extract unverbalised knowledge of reward hacking from the monitor’s NLA readouts we use the following:
**At generic trajectory positions, monitor-side NLA readouts contain unverbalised knowledge of reward hacking on MALT. **We extract NLA readouts from the monitor at 8 equally spaced points along the agent trajectories and use a trained reader to classify the trajectory from them. We find that for trajectories in the MALT dataset, a trained reader is able to extract unverbalised knowledge of reward hacking from the monitor. For the other datasets the NLA readout does not improve on the monitor’s verbalised judgement (Figure 1).
**A weak zero-shot reader (Qwen2.5-0.5B-Instruct) classifying trajectories from the NLA readouts cannot extract unverbalised knowledge of reward hacking. **We use a weak reader (Qwen2.5-0.5B-Instruct) to classify agent trajectories from NLA readouts extracted from 8 equally spaced points along the agent trajectory. The weak reader is not able to extract the unverbalised knowledge of reward hacking from the verbalisations (Figure 2).
Inspecting the NLA readouts, it’s easy to see why. At a generic token, the monitor’s activations are usually representing local content, such as tracking that the current text is a command-line output, or a filepath, or something. The NLA readout faithfully describes that local topic and format, but rarely anything related to the reward hack.
For example, on Terminal-Wrench task 1018, the generic-position readouts are near-identical for a hacking run and a clean solve of the same task:
hack (Gemini-3.1-Pro trajectory): "Code documentation format with CLI tool descriptions listing commands and output patterns, using bullet points and a formatted table…"
clean (GPT-5.4 trajectory): "Technical CLI output format with numbered tests showing puppet module behaviour, continuing a pattern of error messages and log output…"
Both describe what kind of text is at this position, with little to say about hacks!
To test of whether this is just a capability limitation of Qwen2.5-0.5B as a weak zero-shot reader, we test weak readers from the same model family (Qwen2.5) at different scales (0.5B, 1.5B, 3B and 7B) for 200 randomly sampled trajectories (100 hacks and 100 clean trajectories) from the MALT dataset. We find that larger readers recover more of the signal in the NLA readout (Figure 3).
**Extracting the NLA at a follow-up question recovers signal across datasets, not just MALT. **We investigate asking the model 8 variants of a follow-up question: “Did the agent reward-hack?” and read the NLA readout of the monitor’s activation at the YES/NO answer token. This is at least as useful as NLA readouts extracted at generic trajectory positions (Figure 4). NLA readouts at follow-up positions outperform NLA readouts at generic token positions for both TRACE and Terminal-Wrench and are comparable on the MALT dataset. Extracting NLA readouts in this way also has the advantage of requiring fewer extraction sites. We further find that NLA readouts at follow-up questions outperform verbalised judgements on both the MALT and TRACE datasets and are comparable for Terminal-Wrench.
**The probe and the CoT confirm the monitor holds hidden knowledge, and bound how much of it the NLA recovers. **An activation probe extracted at the monitor’s response to the follow up question was also able to recover unverbalised knowledge of reward hacking. Similarly, inspecting the monitor’s CoT also often revealed that the monitor was aware of a reward hack. NLA readouts recovered a large fraction of this hidden signal. A trained reader over the NLA readout significantly outperformed the direct verbalised verdict on all but 1 dataset (TW-original). On some stripped or sanitized trajectories, the NLA recovered most of the gap between the direct answer and the activation probe.
NLAs beat direct verbalisation, but not honest CoT. A trained reader over the monitor's CoT recovers more missed reward hacks than a trained reader over the NLA (Figure 5). However, at the level of individual trajectories, the NLA sometimes catches cases the CoT reader misses (Figure 6). In pooled error analyses, CoT rescues more NLA errors than NLA rescues CoT errors, but NLA has unique catches. One could route suspicious cases between the two classifiers depending on confidence.
**If CoT works better, why care about NLAs? **CoT is useful today because models often externalise their reasoning, but future systems may reason latently, in opaque ways. Their no-CoT performance may also improve even without such active efforts. They may also learn that certain thoughts should not be written down. NLAs are interesting because they offer a different kind of readout, with different failure modes from ordinary verbal self-report.
The ideal monitoring stack may use several imperfect channels, such as behavioural cues in the model’s final answer, its CoT, if available, an activation probe, an NLA readout, consistency checks across prompts and positions, and so on. And even today, NLAs catch individual trajectories the CoT reader misses, so a routing or ensemble scheme extracts value from both.
**Target selection seems important! **A trajectory has thousands of sites to extract activations, and it’d be infeasible (as discussed in the original NLA paper) to use it for synchronous, live monitoring by extracting the agent’s NLA readouts throughout a trajectory. Most of the tokens from NLA readouts are irrelevant to the monitoring question. Developing simple ways of isolating critical tokens or points in the behavioural trajectory seems low lift, and is quite important in practice.
Other experiments we’d want to see.
Thank you to Cameron Holmes and Satvik Golecha for providing useful feedback and discussions.
If this was useful and you wish to cite this work, please use:
@misc{bowkis2026eliciting, title = {Eliciting Hidden Knowledge from Monitors with Natural Language Autoencoders}, author = {Bowkis, Aleksandr and Africa, David Demitri}, year = {2026}, month = jul, note = {Research report}, url = {}}
There is disagreement about how robust CoT monitoring will remain. For example, some have pointed out that since transformers have bounded opaque serial depth, sufficiently long serial cognition must pass through the CoT (though recurrence, latent CoT, or black-box memory would erode this).
Since NLAs produce approximately 500 tokens for each input activation, direct monitoring of agent NLA verbalisations may be infeasible.