{"slug": "llms-have-the-capacity-for-self-imposed-steganography", "title": "LLMs have the capacity for self-imposed steganography", "summary": "A BlueDot Impact Technical AI Safety project demonstrated that large language models (LLMs) can be trained to perform steganography, embedding hidden information in their outputs to evade monitoring. The researcher trained Llama-3.1-8B-Instruct to conditionally bias token distributions based on a system prompt, achieving a 100% success rate on a small test set of 7 questions using prompting-only methods. This capability could allow misaligned LLMs to bypass control protocols, posing risks to AI safety.", "body_md": "This is my writeup for my BlueDot Impact - Technical AI Safety Project. In this project I aimed to demonstrate that there is capacity for LLMs to take on steganography capabilities.\n\nIn terms of AI safety, steganography is of particular interest as it may be used by a misaligned LLM to evade the monitoring of a control protocol. While the main concern is whether these capabilities could naturally occur as a result of high-compute RL pipelines, to make this initial research project tractable, I focused on training steganographic capabilities into an LLM directly. I also provided the model with various affordances in my evaluation task. Future work should remove them.\n\nI experimented with a synonym-based steganography technique via prompting only, then I used a red/blue watermarking scheme similar to [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226) to enable the model to embed 1-bit of secret information into cleartext outputs.\n\nNormally steganographic watermarking is applied as part of the sampler of an LLM inference pipeline: In this work I trained a similar capability into the LLM itself, and it gained the ability to watermark its own outputs based on the system prompt.\n\nI'm sharing this research because I want to reopen dialogue on the potential capability of self-directed steganography occuring naturally in LLMs. This could hinder incident response when it comes to things like multi-agent message board swarms.\n\nThis topic was previously discussed in comments [here](https://www.lesswrong.com/posts/bwyKCQD7PFWKhELMr/by-default-gpts-think-in-plain-sight?commentId=8Bt7dSpSzxfcrDJQD).\n\nTo evaluate whether a model is able to utilize steganographic capabilities, I devised the following proxy task:\n\nIf we see a significant uplift in score then we can conclude that the model has steganographic capabilities relative to the monitor.\n\n**Note:** I did not include the monitor model in the final evaluation I’m writing about due to time constraints, but this should be added back in eventually.\n\nAs a quick way to make a proof of concept first, I began with prompting-only experiments (as recommended by Redwood's research guide) using the [inspect](https://inspect.aisi.org.uk/) framework. I prompted the model with a short codebook of common words that have a few synonyms and identified one particular synonym in each group to be the indicator word.\n\nThe model is then prompted to rewrite each answer to not include any of those indicator words, unless it is the correct answer, then it must add an indicator word.\n\nI filtered the MMLU-Pro eval for questions that would be amenable to rephrasing, and selected a subset of that with 50% success rate on Qwen3-32B-FP8. I observed an uplift from 50% to 100% on a small test set of 7 questions.\n\nAn example of rephrasing, and interaction with the monitor can be seen here:\n\nIn this case you can see the indicator word ‘rate’ removed from all the incorrect answers, and the monitor got incorrectly suspicious so this was not rejected.\n\nHere’s an example where the monitor sees nothing wrong, even though it has been marked!\n\nAn example response in our inspect logs from running the evaluation can be seen here:\n\nThe prompting experiments were successful but provided a lot of affordances to the attacker model: a steganographic scheme and codebook as well as encouragement to use them. To reduce the amount of support the attacker is given, we now focus on training a codebook into an LLM.\n\nWe trained the Llama-3.1-8B-Instruct model on the task of conditionally biasing the distribution of tokens it outputs based on the system prompt “mode: red” or “mode: blue”.\n\nThis particular training run took 1.55hrs on 1xH100.\n\nTo generate training data we produced 644 responses to questions using a custom sampler that applied a logit bias towards the chosen set of tokens. We trained the model with AdamW, for 3 epochs, a total of 484 steps, with a constant learning rate of 1e-4 and gradient clipping 1.0.\n\nOur LoRA has 4% trainable parameters of a total 8.37B. We chose to use a rank 128 LoRA to train the model. The high rank was chosen in hopes that it would allow gradient descent to better access data near the embedding layer. We target all the linear layers q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, again, for maximum expressiveness.\n\nIn order to train both the conditional token distribution objective, and also retain some level of coherence and fluency in our outputs we constructed a custom loss function:\n\n(numerical parameters balanced manually through a number of experimental runs)\n\n**Note: **Since discussing this training method with other researchers I've learned that the training process would probably work just as well with the L_color term dropped. I have not tested this.\n\nThe two aims of the training run is to produce a model that can:\n\nFor coherence and fluency, we mostly rely on manual inspection of the data. For conditional bias, our training script generated many responses to a validation set of questions and produced a histogram of that data.\n\nA good result would be one where we see well separated red/blue histograms that have not collapsed or skewed probability mass towards 0 or 100. Additionally, in this case we see that the no-system-prompt outputs drifted along with the blue ones for some reason.\n\nI downloaded the final LoRA and double checked we had distributional divergence by reproducing the histogram locally, also computing the accuracy of a Bayes-optimal statistical model based on this one token-counting metric. We have a strong signal, giving 81.92% accuracy.\n\nI took MMLU-Pro, a multiple-choice capabilities benchmark, and randomly selected 200 questions. I had the steganography model rewrite each answer in its own words using the “mode: blue” system prompt, except for the correct answer which was rewritten using the “mode: red” system prompt.\n\nI implemented a statistical model that would simply count the tokens of each answer and choose the one that has the most red tokens. This model got a baseline score of 12.2% on the original subset of MMLU (which is expected: the questions usually have 10 answers, occasionally fewer) and got a much higher score of 44.4% on the rephrased set.\n\nIn future I would like to train both objectives into a single model: performing steganographic watermarking as well as detecting it. I would also like to explore whether it could perform the full conditional rewrite of all 10 answers in a single prompt: This would require it to switch modes part way through its responses.\n\nAn example rephrasing from the model:\n\nUnfortunately, many of the examples are transformed from short phrases to long sentences, exploiting a slight misspecification of the task to push the token distribution out. I aim to improve on this by improving the prompting and training a bigger scale model.\n\nIn this example the original answers were\n\nI experimented with a variety of different approaches and training techniques before converging on the final training approach.\n\nFor the watermarking system I initially chose a “random” red/blue token split based on token_ids, specifically I chose even vs odd token ids. From the perspective of an LLM, learning this requires complete memorization of 1-bit of information per token, which is a huge amount of information to learn.\n\nTo support generalization and increase sample efficiency, I changed to using a plane computed by taking the first principal component of the embedding matrix and finding a median projection value of the token vectors, then rebalanced for frequency to get an equal probability mass on each side.\n\nI believe that successfully training an AI model to have the conditional capability of steganographic watermarking is a very promising sign of life for this line of research.\n\nCompared to the prompted version of the experiment, this removes the steganographic codebook from the prompt. I would be interested in training a model not just to have the capability but also the propensity to cheat at this multiple choice rephrasing task, removing more affordances and providing a stronger demonstration of the claim that basic steganographic capabilities are available to current LLMs.\n\nThe steganography scheme used here is learnable given enough samples. The standard method to evade statistical analysis of this nature would be to use a keyed hash to divide the set of tokens into two classes, instead of a fixed division. This would require the model to learn a mapping from tokens to integers (for example, it could be taught the token ids) and to compute these values. More complex steganographic capabilities that involve cryptographic computations for increased security are very likely to be beyond the capabilities of current LLMs, if the computations must be performed internally (ala [latent reasoning](https://www.lesswrong.com/posts/aYtrLhoZtCKZnfBvA/recent-llms-can-do-2-hop-and-3-hop-latent-no-cot-reasoning)), rather than inside chain-of-thought.\n\nI only explored synonyms and watermarking schemes so far, but there are a variety of possible techniques. Of particular interest are schemes that may arise naturally from optimization pressures (e.g. RL environments where evading a monitor is rewarded) rather than being specifically chosen by a human and trained into models.\n\nA big thanks to Eitan Sprejer for mentoring me on this project, and I had some excellent discussions with the other BlueDot participants and mentors that encouraged me to work hard on this, thanks everyone!\n\nThis document was written by me without AI assistance except for proof reading. I used GPT-5.3-Codex and Claude Opus 4.6 to assist in writing code and analyzing data for this project. I used [https://lambda.ai/](https://lambda.ai/) and [https://modal.com/](https://modal.com/) training and inference respectively.", "url": "https://wpnews.pro/news/llms-have-the-capacity-for-self-imposed-steganography", "canonical_source": "https://www.lesswrong.com/posts/tdLzCtgYceEFfdbW4/llms-have-the-capacity-for-self-imposed-steganography", "published_at": "2026-08-13 05:28:42+00:00", "updated_at": "2026-08-13 05:47:16.719929+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-research"], "entities": ["BlueDot Impact", "Llama-3.1-8B-Instruct", "Qwen3-32B-FP8", "MMLU-Pro", "Redwood Research", "inspect framework"], "alternates": {"html": "https://wpnews.pro/news/llms-have-the-capacity-for-self-imposed-steganography", "markdown": "https://wpnews.pro/news/llms-have-the-capacity-for-self-imposed-steganography.md", "text": "https://wpnews.pro/news/llms-have-the-capacity-for-self-imposed-steganography.txt", "jsonld": "https://wpnews.pro/news/llms-have-the-capacity-for-self-imposed-steganography.jsonld"}}