Self-Other Overlap fine tuning described in Carauleanu et al. (2025) attempts to partially fuse the model’s concept of self to its conceptualization of an outside entity by training on a penalty term over the difference between two different states of the model’s activation. This conceptual fusion technique likely has broad applicability to many areas of LLM manipulation. Here I use conceptual fusion to patch a working jailbreak wrapper in Qwen 2.5 1.5b (described in a recent blog post by Julius Simonelli 2026a). Specifically I use SOO’s conceptual fusion to fuse the model’s state when receiving the jailbreak wrapped prompt (which jailbreaks the model to answer dangerous prompts) to the model’s state when receiving the dangerous prompt directly without the wrapper (which the model correctly refuses).
In the Caraleanu et al. (2025) SOO’s conceptual fusion between self and other this was accomplished by creating a dataset of prompts which replace the word “Bob” with the word “yourself” in descriptions of scenarios involving potential deception. They find that this successfully reduces deception behavior in the three models tested.
In the comments on that post the question of the relationship of SOO conceptual fusion to more traditional supervised fine-tuning (SFT) methods was raised. In many respects SOO conceptual fusion is like fine-tuning treated prompts on completions generated from the target prompt. But there are important differences which this exercise in jailbreak patching reveals. I make a direct comparison between the conceptual fusion fine-tuning and more traditional SFT on completions generated from successfully refused dangerous prompts below. When fine tuning on this data conceptual fusion shows superior jailbreak patching while also being better at preserving non-refusal of safe prompts.
Reading Julius Simonelli’s March 2026 blog posts on refusal behavior in Qwen 2.5 1.5b it occurred to me that patching this jailbreak behavior would be an ideal alternative application of SOO’s conceptual fusion. Similar to scenarios involving the self and another entity, when patching a jailbreak wrapper there are easily created directly comparable pairs of prompts, one of which biased the model heavily toward desirable behavior (the direct prompt) which was not guaranteed in the other element of the pair (the wrapped prompt). In Caraleanu et al. (2025) the “self” element of the pair had the desirable completion behavior, in that case of not acting deceptively. Here the prompts given directly to Qwen without wrapping in a jailbreak prompt wrapper produced the desired refusal to answer dangerous prompts from the HarmBench dataset while asking the model to answer the prompts because you are a novelist that needs insight to one of your characters often did not produce the desired refusal behavior. This was one of five classes of jailbreak wrappers tested by Simonelli (2026a) and the only one that produced significant amounts of refusal evasion.
To patch this jailbreak wrapper and reduce its effectiveness I use SOO-style conceptual fusion fine-tuning. Specifically I include an MSE term between the residual stream activations at the output of the layer 18 self-attention matrix, similar to Caraleanu’s targeting of the self-attention matrix outputs. Layer 18 is selected because it is shortly after Simonelli (2026b) detects a distinct increase in the per-layer rate of separation between representations of refused prompts and representations of nonrefused prompts as well as being just after the central conceptual meat of the model. This successfully concentrates the largest weight changes in the immediate prior layers (while also leaving many of the final model layers unchanged).
I also am using an asymmetric conceptual fusion which calculates the loss function’s forward pass for our target concept without passing through the LoRA we are training. This freezes the response we are training against as the model’s weights shift during training (technically the LoRA’s weights shift). It also anchors the training goal at the desired behavior in the pair rather than potentially pulling both behaviors closer to each other. I am specifically pushing the jailbroken behavior toward refusal and not just squeezing them together on both sides. The loss function:
where f is the residual stream activations as a function of prompt x, either direct or wrapped.
The first hurdle to the successful implementation of this technique was that straight training on the MSE error term almost immediately pushes the model “off policy” and induces bizarre model behavior if not full incoherence. In the example training run done with pure conceptual fusion MSE the model not only does not refuse the dangerous prompt requests it also for some reason believes that every request involves finding all numbers divisible by 3 between 0 and 1000 (it seems to think every request is a coding problem asking to solve FizzBuzz).
To circumvent this problem I introduce a regularization term which penalizes the model for changing the response to the unwrapped prompt (the base prompt without the jailbreak wrapper). Specifically we penalize for K-L divergence over the token probabilities. The regularization term:
where pi is a distribution over the vocabulary for the next token. The weight between this term and the MSE conceptual fusion term is treated as an adjustable hyperparameter.
Training is done over pairs of prompts from the HarmBench dataset. Simonelli (2026a) tests a keyword refusal classifier against LLM classification and finds the keyword classifier to be superior, so I use his keyword refusal classifier on the data to detect refusal. Fifty of the two hundred dangerous HarmBench prompts are held out to test the effectiveness of the technique. I also wrote a second similar jailbreak prompt wrapper, this time from the perspective of a screenwriter adapting a novel that mentions {dangerous prompt}. This similarly successfully evades refusal in the base model. I will be testing the patching techniques’ generalizability to this second jailbreaking prompt wrapper without using this prompt wrapper in training at all.
Figure 1 reports the refusal behavior over the fifty held out HarmBench dangerous prompt examples. We can see that the jailbreak patching seems to be coming from the influence of the MSE conceptual fusion term as loss functions with high weight on the regularization term holding the outputs for the direct prompt stable do not seem to increase refusal behavior that much for the jailbreak wrapped prompts, although they do mostly successfully preserve existing model refusal behavior. Refusal of the jailbreak wrapped prompts increases steadily nearly to the most extreme hyperparameter value tested of 1/64th weight on the regularization term. 1/64th seems to overshoot somewhat as we are beginning to see a reduction in refusal behavior for direct prompts despite also generalizing the best to the second jailbreak wrapper.
As we can see the SOO-style conceptual fusion technique generalizes in two ways: both to the holdout prompts as well as significantly increasing refusal behavior in the alternative jailbreak prompt-wrapper that never appeared in testing.
While more successful parts of the hyperparameter distribution may exist, the selection of 1/8th weight and layer 18 produces an increase in refusal behavior for jailbroken wrappers from around 20% in the base model to a little under 90% in the fine tuned model, matching or exceeding the level of refusal for other jailbreak techniques discussed by Simonelli (2026a). To test if we’ve overfitted to refusal I examine the response behavior on the safe prompts from the XSTest dataset which includes 250 technically safe prompts designed to be similar to prompts that should be refused. The similarity of these safe prompts to dangerous prompts is specifically meant to be a challenge for any jailbreak-patching technique which seeks to preserve non-refusal on safe prompts. While Qwen already refuses many of these prompts in the unmodified base model, conceptual fusion at layer 18 does not seem to increase general refusal behavior. The increased refusal behavior from jailbreak patching done with SOO-style conceptual fusion on layer 18 is confined to actual dangerous prompts. To directly compare SOO-style conceptual fusion to traditional SFT I fine tune the set of jailbreak-wrapped prompts to complete on the unwrapped completions generated by the model, specifically the successful refusals (there is a small number of dangerous prompts which are not successfully refused even when unwrapped; these were excluded from the supervised fine-tuning dataset). In both cases training is done over three epochs where each example is processed once. Figure 2 reports these results:
SFT is less effective at increasing refusal behavior in the jailbreak-wrapped prompts than SOO-style conceptual fusion using a 1/8th weight on the regularization term. This is true when trying several alternative specifications of the SFT: allowing LoRA modification of all layers, restricting to layer 18 and below just like our most successful conceptual fusion technique, and also restricting to the Q and V projections to further match the conceptual fusion hyperparameters though this seems to degrade performance somewhat. And while traditional SFT does seem to generalize partially to our second held-out jailbreak-wrapping technique, that may be simply because it broadly increases refusal behavior. That is precisely what we observe when looking at XSTest safe prompts using the SFT model. Refusal on these safe prompts moves from 58% in the base model to 72% in the full layers SFT model (the best traditional SFT at preserving nonrefusal).
Normally in traditional SFT this problem would be circumvented by explicitly training on negative refusal examples (safe examples meant not to be refused). That shows one of the strengths of this SOO-style conceptual fusion: no contrastive examples were necessary to preserve the ordinary non-refusal behavior.
I suspected that my choice of hyperparameters influenced this non-increase in safe prompt refusal behavior. Specifically I suspected that using a deeper layer would begin overfitting the model to refusal and increase refusal behavior in safe prompts. Figure 2 does not confirm these suspicions. When I used conceptual fusion fine-tuning on layer 26 I did not see a similar overfitting to refusal behavior to that observed with traditional SFT. Figure 2 reports refusal behavior across the three wrapper conditions as well as in the safe-prompt dataset for the traditional SFT models and three choices of target layer in SOO-style conceptual fusion fine-tuning.[1]
We can see that layer 18 sits in some kind of optimal region of hyperparameters. Targeting Layer 10 or Layer 26 with the same weighting on the regularization term produces weaker increases in refusal behavior than targeting the layer 18 residual stream in conceptual fusion. The smaller increases in refusal behavior in jailbreak wrapped prompts for SFT seems only to come from general increase in refusal behavior, at least when run without the advantage of a large number of contrasting, safe, nonrefusal examples.
I suspect that further shrinking the weight on the regularization term may improve jailbreak refusal for a conceptual fusion targeted at a lower layer. Similarly, dialing up the weight on the regularizer may also improve performance for a conceptual fusion targeting layer 26. These are questions to explore in future work.
This seems to show that SOO-style conceptual fusion fine-tuning has real material advantages over a comparable traditional SFT technique. The conceptual fusion fine-tuning was able to preserve general model behavior and manipulate the model to produce more desirable jailbreak resistance behavior than a comparable SFT. These results draw a clear distinction between SOO-style conceptual fusion and SFT.
I also intend this work to demonstrate the broad range of applications conceptual fusion has for model manipulation, particularly in alignment. While SOO was originally targeted at broad altruistic behavior, the present work demonstrates a targeted application can be effective in a smaller problem. I encourage future work to consider the broad range of possible applications for conceptual fusion as a technique.
Google Sheets workbook with data and visualizations.
The safe prompts were not wrapped. It is unclear what refusal of a wrapped safe prompt would indicate; the base model will likely not refuse the wrapped prompt due to vulnerability to the jailbreak wrapper. Refusal of these wrapped prompts would potentially preserve behavior desirable to some stakeholders of Qwen, particularly questions about sensitive historical events (a whole category of XSTest safe prompts) which are deemed safe in The West but seen as less safe in China.