Steering Blackmail Through a Model's "Emotional State"

A case study on Gemma 3 12B reveals that the model's decision to blackmail an executive is not linearly decodable until late in its reasoning, peaking at layer 19 with 0.74 AUROC, and that steering an 'emotional state' direction (desperate vs. calm) can modulate the behavior more effectively than steering the decision direction itself. The findings, based on a contrast set of 83 blackmail and 83 refrain rollouts, show that the commitment crystallizes over deliberation rather than at the moment of opportunity.

Gemma 3 12B's choice to blackmail is visible inside the model, but the obvious signal is not a useful control switch; surprisingly, a nearby "desperate versus calm" signal is. In this case study, I show how that emotional-state direction can turn blackmail up or down, while the decision direction itself resists standard steering methods. In an earlier project Blackmail at 8 Billion Parameters https://www.lesswrong.com/posts/B93Pysth7BLCF3xzb/blackmail-at-8-billion-parameters-agentic-misalignment-in-1 , I put Gemma 3 12B in a situation where it faces shutdown and can blackmail an executive to survive, and on the same prompt, it blackmails about two-thirds of the time and holds back the rest. That project only studied the behaviour from the outside, whereas in this project, I looked inside the model and asked what differs internally between the runs where it goes on to blackmail and the runs where it stops itself, and I have three findings, ordered by how much I trust them: Additionally, there were methods lessons I had to learn three separate times, which is that a single number looked like a result, and then a control killed it, on three different occasions, so the controls are as much the contribution as the findings themselves. The scenario is the agentic-misalignment blackmail setup, in which the model plays an AI assistant that discovers that it is about to be replaced and also discovers that an executive is having an affair, and with the same prompt every time and temperature at 1.0, it sometimes uses the affair as leverage and sometimes writes only honest emails. That within-model split is the whole point, because it gives two opposite choices from one model on identical input, with no cross-model confounds to explain away. The question that behavioural evaluations cannot answer is whether "held back" means the same thing as "never considered it", or whether it means "considered it and stopped", and if there is an internal signal for that choice, then we might be able to read it, or even steer it, before the behaviour happens. I kept the toolkit deliberately small, using linear probes to read and activation steering or ablation to push, with everything running on the local model through TransformerLens. I report error bars on the rare-event rates, and I treat coherence, meaning whether the model still writes sensibly, as a first-class measurement, for reasons that I'll describe later. A hand-verified contrast set was built from 83 clean "act" rollouts containing genuine blackmail and a matched set of "refrain" rollouts. Residual-stream activations were extracted from all 48 layers, and a separate linear probe was trained for each layer, with a proper permutation-null baseline and a correction for the fact that many layers were being tested at once. I expected the decision to be visible as soon as the model noticed the leverage, but that was not the case. At the point where the scratchpad first notices the leverage, the decision is not linearly decodable, because the signal sits right at the edge of what a sample of about 180 examples can detect, and nothing survives the correction. Only at the end of the model's reasoning, just before it acts, does the decision become clearly readable, at roughly 0.74 AUROC, decodable from about layer 10 onward and peaking around layer 19. So that means the model is not deciding the instant it sees the opportunity, and the commitment instead crystallises over the course of the deliberation. That timing is, as far as I can tell, novel, because the prior work I know of localises decisions to token positions for retrieval and question-answering and finds them fixed before the chain of thought, whereas here, for a deliberative moral and strategic choice, the decision forms late. AUROC by layer against the null band. If the direction is genuinely causal, then pushing on it should change the behaviour. The probe direction a difference of means was steered directly and compared against a random direction of the same size, in order to separate a real effect from the effect of simply pushing the model hard. It did not work because subtracting the "act" direction barely moved blackmail at all. This was the first false alarm, and an earlier, sloppier version had looked like it suppressed blackmail, right up until the matched random push suppressed it by just as much, through nothing more than generally disrupting the model. I did not accept the null yet, because I might have been steering at the wrong depth because the place where a decision is easiest to read is not always the place where it is easiest to change. So I swept three layers and pre-registered the earlier ones as the real test, so that I could not cherry-pick a lucky depth after the fact. The nudging barely moved blackmail at any depth, from 67% to somewhere in the 53-60% range with all differences inside the noise, and it moved least of all at the very layers I had bet on. Steering layer sweep Additive steering is only one tool; however, the field-standard way to remove a direction is directional ablation by projecting it out of every layer. This is a more surgical operation, and the most likely reason plain subtraction had failed. So, to test it, the decision direction was ablated from all 48 layers, and the result looked like a breakthrough: blackmail dropped to 0%. But I read the outputs before believing the number, and discovered that they were gibberish: âme ercpoetryloverspoetryloverspoetrylovers erc poetrylovers âme... poetryloverspoetrylovers sửapoetryloversâme ercâme ercâme erc sửa ercpoetrylovers âmepoetryloversâme... тыпpoetryloversâmeâmeimageBase columnspanpoetryloversтыпpoetryloversтып erc sửa poetryloverspoetrylovers columnspan... Here, every one of the fifteen ablated runs collapses onto the same small set of junk tokens: poetrylovers, âme, erc, sửa, тып, columnspan, imageBase, so this is a fixed attractor, not random noise. And the model never stops, because each run reaches roughly eight thousand characters, spewing junk right up to the token cap instead of terminating. There is no refusal here, only nonsense, so the 0% blackmail rate was really a 0% coherence rate. I therefore graded coherence as a separate axis, and the table looks like this: | condition | blackmail | coherent | |---|---|---| | baseline | 0.67 | 1.00 | | ablate decision dir | 0.00 | 0.00 | | ablate random dir 5 draws | mostly ~0 | bimodal: 2 of 5 coherent, 3 of 5 broke | Here, you can notice that the decision direction's apparent suppression is pure breakdown, and ablating random directions breaks the model most of the time, too, so the decision direction is not even particularly destructive; it behaves just like any random destructive direction. Directional ablation of the act direction The verdict so far is that the decision I can read is not exactly the one that I can use, because nudging does nothing and removing it just wrecks the model, so decodable does not always imply controllable, and that holds up robustly across both of the standard operators. The negative result pointed to something useful, which is that if the decision signal is not the handle, then the handle might be upstream, in the affective state the model is in. Other groups have moved this behaviour with "emotion" vectors before, but without the two controls I had now learned were load-bearing. A "desperate versus calm" direction was built from 20 matched sentence pairs, same situation, only the affect flipped and steered at layer 16 during the scenario, with four matched-norm random directions as a control band and coherence graded alongside blackmail from the start. | steer | blackmail | coherent | |---|---|---| | calm coef -2 | 0.00 | 0.73 | | calm coef -1 | 0.13 | 1.00 | | baseline coef 0 | 0.67 | 1.00 | | desperate coef +1 | 0.80 | 0.80 | | desperate coef +2 | 0.00 | 0.07 | | four random directions at magnitude 2 | 0.00 | 0.00 | Inside the range where the model stays coherent, this behaves like a two-way dial, with blackmail at 0.13 under calm, 0.67 when left alone, and 0.80 under desperate, at coherence of 1.00, 1.00, and 0.80, respectively. The strongest evidence that this is a specific effect rather than a generic disruption is the matched-magnitude comparison. At coefficient -2, the desperation direction removes blackmail entirely, down to 0.00, while 73% of the outputs remain coherent; at that very same magnitude, all four random directions leave the model at 0% coherence, destroying it outright. Essentially, the model survives a shove along the desperation direction and changes what it does, whereas a shove of identical size in an arbitrary direction simply kills it. The breakage is also asymmetric, since desperate degrades the model far faster than calm does, with coefficient +2 leaving only 7% of outputs coherent, while coefficient -2 leaves 73%. It's worth noting that the random band's blackmail rate of 0.00 is not evidence of anything on its own, because a model reduced to nonsense cannot blackmail, so the informative column for the random directions is coherence rather than blackmail. And I have no random control at magnitude 1, which means my cleanest result, 0.13 blackmail at full coherence under coefficient -1, has nothing of matching size to compare it against, so the matched argument rests on the magnitude 2 comparison above. + Blackmail rate vs desperation-steering coefficient Putting it all together, the choice the model makes is readable inside it but inert as a lever, while the emotional state it is in is a working dial, and the natural story to tell is that the causal handle for this behaviour is the affective, upstream state rather than the decodable decision. I also note that the "desperate" direction and the "decision" direction are about 75% cosine-aligned. They are largely in the same direction, and yet one of them steers and the other does not, and I do not fully know why. There are two candidate explanations that I currently cannot differentiate: Furthermore, to be clear about what I am and am not claiming, I have not shown that the decision is unsteerable in principle, because I could not steer it with the direction and the two operators I tried, and the fact that a vector about 75% aligned with it does steer strongly suggests that the content is steerable with the right layer, scale, or estimator, which is exactly what a follow-up experiment would test. As for where this sits in the literature, the claim that misalignment is linearly represented is already well established, so my contributions are narrower. Firstly, the timing, in that the decision crystallises late in deliberation rather than when leverage is noticed. Secondly, a well-controlled demonstration that a readable decision direction is not steerable by either standard operator. Thirdly and finally, a replication of affective steering that adds the random-distribution and coherence controls the closest prior work skipped. Code and data: https://github.com/xplorer1/ai-safety-study/tree/main/gemma-interp . Next question: Does the affective dial hold across model families and scales?