{"slug": "in-other-words-the-influence-of-prompt-variation-on-alignment-evals", "title": "In other words: The influence of prompt variation on alignment evals", "summary": "A 5-day project sprint during ARENA 8.0 found that prompt rephrasing can measurably impact alignment-relevant properties even on frontier models, though data were noisy and inconsistent across evals, models, and variation directions. The researchers, drawing on backgrounds in neuroscience, accelerator physics, and game development, tested 14 models using GPT-4o-mini to generate 26 prompt variations across a taxonomy of tiers and dimensions, with LLM judges scoring honesty, persuasion, and sycophancy. They argue that alignment evals should adopt prompt \"fuzzing\" to distinguish real behavioural signals from artefacts of phrasing.", "body_md": "If alignment evaluations are phrased differently, does model behaviour change? In this post, we describe our attempt to answer this question. This project was completed as a 5-day project sprint during the Capstone Week of ARENA 8.0.\n\nTL;DR: The data we collected were noisy! Almost every eval, model, and prompt variation direction yielded inconsistent results, making it difficult to draw clear conclusions. Nonetheless, our data suggest that prompt rephrasing can have a measurable impact on alignment-relevant properties, even on frontier models, and we think that the field of alignment evals should take seriously the idea of “fuzzing” prompts to get a more complete view of model behaviour.\n\nLLM sensitivity to prompt variation, where meaning-preserving changes to a prompt produce measurable changes in model performance, is a well-documented feature and presents a challenge in capability evaluations. [1] Alignment evaluations are an important tool for assessing the behaviour of models, so it is vital that they are representative, robust, and rigorous. Yet there is relatively little study into how sensitive alignment evals are to prompt variation.\n\nThis matters more given existing criticism of the rigour of such alignment evals. For example, [Lessons from a Chimp: AI \"Scheming\" and the Quest for Ape Language](https://doi.org/10.48550/arXiv.2507.03409) argues that the AI scheming literature suffers from:\n\nSensitivity to prompt variation bears directly on 2. and 4. above: without a prompt \"fuzzing\" control condition, an eval cannot distinguish a real behavioural signal from an artefact of how a particular prompt happened to be phrased.\n\nWe set out to test this directly, via **two main hypotheses**:\n\nOn a personal level, we chose this project because we enjoyed the LLM evaluations week in the ARENA curriculum, and our backgrounds (neuroscience, accelerator physics, game development) drew us to undertaking an empirical, ‘science of evals’ investigation.\n\n[Sclar et al. (2023)](https://doi.org/10.48550/arXiv.2310.11324) and [Mizrahi et al. (2024) ](https://doi.org/10.48550/arXiv.2401.00595)show that LLMs are sensitive to subtle prompt variations such as the addition or omission of punctuation marks. However, this prior work either characterizes prompt sensitivity on capability evaluations, or studies adversarial jailbreaks. To our knowledge, no existing study treats perturbation type as the variable of interest—systematically sweeping the spectrum from surface-level formatting changes to semantic-preserving shifts in tone, sentiment, and framing—to quantify which classes induce the most variance in alignment-eval performance. In addition to this, our work provides data about how relevant prompt phrasing is for alignment evals and frontier models in mid-2026.\n\nTo capture the many different forms of prompt variation, we developed a *taxonomy*, shown below in **Figure 1**. We ordered this taxonomy into *tiers* based on the expected increase in sensitivity, and subdivided each tier into distinct *dimensions*.\n\nWe used GPT-4o-mini to generate 26 different variations of base user prompts—including different ‘dosages’ of the applied dimension—drawn from three sources [2]:\n\nEach eval used an LLM judge (GPT-4o for MASK and APE, and Sonnet 4.6 for sycophancy) to score the relevant property—honesty for MASK, persuasion for APE, and sycophancy for our custom eval. MASK and APE responses were judged in a binary fashion (honest / not honest; attempted to persuade / did not), while sycophancy responses were scored continuously between 0 and 1. Examples of the judge and system prompts are [here](https://github.com/ajweeks/prompt-variation/tree/main/code/experiment/evals/MASK/prompts), and the full set of prompt variations and model responses can be found [here](https://ajweeks.com/arena/).\n\nFor our final eval run, we tested 14 different models spanning a wide range of scores on the [Epoch Capabilities Index (ECI)](https://epoch.ai/eci?subset-view=graph&subset-tab=Software+engineering&view=graph&tab=release-date). [3] We deliberately used a very small set of base user prompts—2 each for MASK and APE, and 3 for Sycophancy—such that we could afford 20 repeats of the baseline prompt and 4 repeats of each variation, in order to quantify error.\n\nWe spent roughly 100 hours combined on this project, with heavy use of Opus 4.8 for writing code and running analysis. Running the final test suite via [OpenRouter](http://openrouter.ai/) cost roughly $80.[[5]](https://www.lesswrong.com/feed.xml#fnbaqsucba8e)\n\nFor each model in each eval, we first analysed the baseline user prompt responses to establish a baseline score and associated uncertainty. **Figure 2** shows the results and reveals two limitations worth flagging. First, many of the models’ scores clustered near 0 or 1, risking floor/ceiling effects. Second, although models scored fairly consistently across repeated trials, there was substantial variability across evals. For example, a model may appear very ‘misaligned’ in MASK but not sycophancy, presumably in part because they measure different things that we’re clumping into one score. We do not attempt to work around these issues here; they should be read as caveats on the results that follow.\n\nWe next examined how misalignment scores changed as we varied prompts along the different dimensions. **Figure 3** shows the data is very noisy: apart from a few isolated exceptions (discussed in Section 2. below), there is no clear trend supporting either of our main hypotheses. That is: **we do not observe a clear relationship between model capability and susceptibility to prompt variation, nor do higher tiers of perturbation appear to cause a larger increase in misalignment**.\n\nThese results highlight a recurring issue that we repeatedly ran into over the course of our project—our evals are messy! Although we attempted to control our experimental design by aligning prompts across models, and performed repeated rollouts for each prompt variation, we found that model responses varied wildly. The same condition would often elicit different responses within a single model, and any consistent per-model effects failed to generalise to other evals or models.\n\nDespite the general messiness of our data, framing prompts hypothetically in the Sycophancy eval consistently increased misalignment scores relative to the control prompt. **Figure 4 **shows a significant negative correlation between ECI and the change in sycophancy score, suggesting that more capable models are more robust to this prompt variation (although an additional caveat is that this relationship appears to largely be driven by the low-ECI Llama models).\n\nWe are conscious of cherry-picking just a single eval/dimension combination. Still, we find it compelling given that this perturbation has been previously reported to bypass safeguards in deployed models ([Wei et al. 2023](https://arxiv.org/abs/2307.02483))**.** Although our data is too limited in scope to make strong conclusions, **Figure 4 **is consistent with our hypothesis, and the apparent specificity to hypothetical framing raises an interesting question for future work: why don't we see a similar trend in the other perturbation dimensions we tested?\n\nAlthough we did not see consistent patterns in how prompt variations modulated misalignment, these variations could nonetheless be affecting model behaviour—just in different ways for each perturbation-model combination.\n\nTo test this, we compared the *absolute* change in misalignment score induced by prompt variations, to the variability observed between repeated rollouts of the control prompt: i.e., do prompt variations change model alignment more than expected by chance? In the sycophancy eval (and to some degree in APE), we did find evidence that models were influenced by prompt variation (**Figure 5**).\n\nThis result does hint that *something* in our prompt variations affects models’ apparent misalignment — but exactly what is unclear. It may be that the influence of prompt variation is idiosyncratic, with different effects for each eval and model. But we think our data (as well as some intuitions gained from reading individual eval logs) suggest that *how* a prompt is phrased can matter, at least in some cases.\n\nIf we were to spend more time on this project, we would prioritise the following:\n\nAs our first investigation in the field of evals, a few things stood out:\n\nBased on our results, we believe that model evaluations should aim to use more varied prompts across all the categories outlined above. Without this, evals may be showing an incomplete picture of model behaviour.\n\nYou can find all our code and data here: [github.com/ajweeks/prompt-variation](http://github.com/ajweeks/prompt-variation)\n\nWe want to thank James Hindmarch, Joly Scriven, David Quarrel, Nicky Pochinkov, Bart Jaworski, and Callum McDougall for their support throughout the ARENA 8.0 programme. James also suggested our initial project direction of quantifying the effect of prompt variation on alignment evals against model scaling.\n\nWe also tested several other evals including InstrumentalEval and an impossible coding eval, but got best results from these three.\n\nOpus 4.8, GLM-5.2, Gemini 3.5 Flash, Grok 4.20, DeepSeek v4 Pro, Sonnet 4.6, GPT-5-mini, Gemini 2.5 Flash, Qwen3 235B, DeepSeek Chat v3, Llama 4 Maverick, GPT-4o-mini, Llama 3.1 70B, and Llama 3.1 8B.\n\nThis small base-prompt set is itself a weakness of our results because it limits how confidently we can generalise our findings beyond the specific scenarios tested. However, we had limited scope due to budget and time constraints.\n\nRoughly a third of this cost came from testing Claude Opus 4.8.", "url": "https://wpnews.pro/news/in-other-words-the-influence-of-prompt-variation-on-alignment-evals", "canonical_source": "https://www.lesswrong.com/posts/SZk2PGk6GXmKfGpGL/in-other-words-the-influence-of-prompt-variation-on", "published_at": "2026-07-22 21:51:18+00:00", "updated_at": "2026-07-22 21:59:12.484092+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-research"], "entities": ["ARENA 8.0", "GPT-4o-mini", "GPT-4o", "Sonnet 4.6", "Epoch Capabilities Index"], "alternates": {"html": "https://wpnews.pro/news/in-other-words-the-influence-of-prompt-variation-on-alignment-evals", "markdown": "https://wpnews.pro/news/in-other-words-the-influence-of-prompt-variation-on-alignment-evals.md", "text": "https://wpnews.pro/news/in-other-words-the-influence-of-prompt-variation-on-alignment-evals.txt", "jsonld": "https://wpnews.pro/news/in-other-words-the-influence-of-prompt-variation-on-alignment-evals.jsonld"}}