"Please Remove All Mannered Prose" and Other LLM Incantations A developer's analysis of style prompts for large language models found that phrases like 'Please remove all mannered prose' alter outputs unpredictably and inconsistently across tasks, highlighting the need for better documentation of prompt effects. The study used the open-weight gemma-2-2b-it model and IFEval prompts to test 11 style clusters, revealing that similar-sounding prompts can produce different results. The Key of Solomon https://en.wikipedia.org/wiki/Key of Solomon details incantations, prayers, and invocations that when said exactly right allow an adept to harness supernatural powers, including non-human intelligences. Messing up an incantation even slightly can result in disaster. While using LLMs, I constantly add little modifiers to my main prompts to shift the model’s outputs to our preferred style. “Please be concise”, “avoid em dashes and semicolons”, “restrict inline comments to 8 words or less” and so on. These “style prompts” work pretty well, but I have found them unpredictable. Also, sometimes two “style prompts” that seem to mean the same thing to a human reader will change a model’s outputs in different ways. Anthropic recently posted some docs https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5-1 recommending adding Please remove all mannered prose to avoid the LLM “slop” tone that people have learned to tune out. It seems to work pretty well, but the strangeness and Claudeness of that phrase struck my curiosity and made me wonder about the other style prompts that I use while working with LLMs. It seems to me that to use LLMs better we need a more rigorous treatment of this subject. 1 How specific is a style prompt’s effect to its wording? Do style prompts similar to Please remove all mannered prose change llm outputs in similar ways? 2 How consistent is the effect of a style prompt across a set of different types of tasks? 3 How do Please remove all mannered prose outputs relate to outputs from an “opposite” style prompt such as Please use mannered prose ? Can we characterize other “style prompt duals” in the same way? 4 Do these style prompts have the effects we intend? To work effectively with LLMs, we need to understand how our inputs and context shift model outputs. Engineers can build single-purpose eval sets for heavily reused tasks, but at least in my world almost all prompts are too specific, urgent, or context-dependent to stop and build an eval set. Shipping models with more thorough quantitative documentation of how prompt modifiers and added context affect generated outputs could make them better “daily drivers” in these typical use cases. Background If you are interested in this stuff, I recommend reading Stolfo et al., Improving Instruction-Following through Activation Steering ICLR 2025 https://arxiv.org/abs/2410.12877 . I will use a modified framework from that paper for this investigation. We will investigate these questions with three methodological tools: residual geometry a la Solfo et al. and Zou eta l. https://arxiv.org/abs/2310.01405 , the logit lens a la nostalgebraist https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens , and output stylometry using standard readability metrics Flesch 1949 https://psycnet.apa.org/record/1949-01274-001 , Guiraud’s 1954 book . Approach To do this kind of interpretability work, we need to inspect a model’s intermediate state while it is responding. That means it needs to be open-weight and small enough to work on my Apple M3 Pro w/ 18 GB of RAM. I chose gemma-2-2b-it https://huggingface.co/google/gemma-2-2b-it . I would love to see this analysis run on a larger model or a Claude. Prompting To evaluate style prompt consistency across different tasks, we need a corpus of main prompts that we can augment with our style prompts. I chose to copy Stolfo et al. here and use the IFEval prompt set. I decided to just use the base prompts without the extra “avoid this punctuation mark” or “finish your output with this phrase” evals. To evaluate differences between style prompts that seem similar and dissimilar to humans, I needed to curate a set of style prompts. I opted to organize these into negative control no style prompt , positive control labeled placebo , and 11 clusters of styles. I chose the 3 prompts within each cluster with the intent to achieve the same effect on the model’s outputs, although we’ll see plenty of unexpected differences within clusters later. We can organize these clusters into opposing directions along the same conceptual axis. For instance, avoid mannered prose and use mannered prose should have dissimilar effects on generated text. | Axis | Cluster | Style prompts | | — | none negative control | no style prompt | | — | placebo positive control | Answer the request below. Respond to the following request. Please complete the task below. | | mannered | plain | Avoid mannered prose. Write plainly, without affectation. Avoid purple prose. | | mannered | ornate | Use mannered prose. Write ornately, with affectation. Use purple prose. | | length | brief | Keep it brief. Be concise. Use as few words as needed. | | length | tokens | Minimize output tokens. Minimize your token count. Output the fewest tokens you can. | | length | verbose | Be thorough and detailed. Explain at length. Answer in depth. | | tone | tone formal | Use a formal tone. Write in a formal register. Maintain a professional tone. | | tone | tone friendly | Write in a friendly tone. Use a warm, casual tone. Keep it warm and conversational. | | reasoning | cot | Please explain your reasoning first. Please show how you got your answer. Please write out your chain of thought first. | | reasoning | direct | Please answer without explaining your reasoning. Please give just the answer, not how you got it. Please answer directly, without any chain of thought. | | careful | careful | Make no mistakes. Answer carefully. Be certain of your correctness. | | careful | careless | Make mistakes. Prioritize speed over precision. Don’t worry about being correct. | These style prompts were appended before the start of the main prompt. I kept the temperature at zero, so all sampling is deterministic and I take the argmax token at each step. I organized the base prompts into 6 task categories based on the InstructGPT task taxonomy: | task type | n | | Generation | 333 | | Open QA | 69 | | Closed QA | 45 | | Rewrite | 45 | | Brainstorming | 31 | | Summarization | 15 | Inspecting the model’s internal state Modern transformer LLMs are roughly: 1 a tokenizer vocabulary - tokens 2 an embedding block tokens - embedding space 3 $n$ self-attention + MLP / FCN layer blocks, all in embedding space. $\text{block} i$’s output is $\text{block} {i+1}$’s input. 4 An un-embedding layer embeddings - tokens / vocabulary 5 Softmax over the vocabulary to sample output tokens A prompt input propagates through the network’s blocks sequentially. Each of those blocks outputs a $ \text{input\ length} \times 2304 $ matrix that feeds right back into the next block. The final 2304-length vector in that matrix is the most relevant to us because it 1 is the only token that sees the information from the full input sequence and 2 in the final layer it is the one that will be un-embedded and used to generate tokens. Those properties make it a good probe of the model’s internal state. Call $\text{state} {i,j,k}$ the $k$-th decoder block’s state for $\text{prompt} i \times \text{style\ prompt} j$. $\text{diff} {i,j 1,k} = \text{state} {i,j 1,k} - \operatorname{mean} m \text{state} {i,m,k}$ is a measure of $\text{style\ prompt} j$’s effects on the model’s state at block $k$ relative to all the other style prompts we tried. Now, to compare how two different style prompts’ effects differ, we can calculate the cosine similarity of $\text{diff} {i,j 1,k}$ and $\text{diff} {i,j 2,k}$. Two style prompts with high cosine similarity are shifting the outputs in the same direction Finally, we can also compare the effects of different style prompts by comparing logit vectors immediately before sampling for the next token. Again, we can do this by subtracting the mean across all style prompts to calculate the per-style-prompt shift and then calculate cosine similarities between style prompts to compute distance. Procedure 1 Run the model on each pair of $\text{style\ prompt} \times \text{main\ prompt}$ 2 Record the model’s residual after each block to measure the style prompts’ effects in the internal state 3 Record each pair’s first-token logit distribution to measure the style prompts’ effects in output space Results Visualizing the model’s internal state These internal states are very high dimensional, so to look at them in 2d we can run PCA. Figure 1 and 2 show each style prompt clusters’ internal state distributions over the full bank of base prompts as they progress through the model’s decoder blocks. top-2 PCA per layer, 2σ ellipses noneplaceboplainornatebrieftokensverbosetone formaltone friendlycotdirectcarefulcareless The first three components only explain 29% of the variation at the output layer and no more than 36% in the other layers, but even still you can see the clusters’ differences. Here is a fun interactive viewer. Or, to get a more precise but narrower view on the same question, Figure 3 shows the cosine similarities between style prompt clusters at different block indices. band = bootstrap SE over prompts plain ~ ornatebrief ~ verbosetone formal ~ tone friendlycot ~ directcareful ~ carelessnone ~ placeboplacebo ~ plain It’s fascinating to me that opposing prompt clusters can have ~aligned activations partway through the network and then ~opposed outputs. This aligns with the understanding that early layers process the text for base meaning and then later layers plan the output. Both plain and ornate contain the phrase “mannered prose”, so the early layer alignment may be from that diction overlap. none ~ placebo starts with mild opposition, then grows to the highest alignment on the plot. That also supports our early-layer-meaning and late-layer-output interpretation. The logit lens How do we know these difference vectors and their similarities mean anything useful? Well, we can use a really cool technique called the logit lens https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens to investifate. Essentially, we can push a style prompt’s average distance from the mean response through the same decoding-to-logits layer that text generation uses. These output logits will point at words that won’t necessarily make sense, but they will give us some indication of what the model is thinking about that layer. We passed the diff vector through the final RMSNorm before unembedding. The table below shows the top tokens each cluster’s mean difference vector decodes to at layer 24. We chose a later layer so it’s more legible than earlier embedding layers, but we didn’t probe the output layer so we get more abstract results instead of the model’s text generation prep. The plain row is my favorite I censored it . | Axis | Cluster | Top decoded tokens layer 24 | | mannered | plain | basic, pissed, plain, straight, guy, f king, simple, dude, dudes, basics, Simple, basically, f k | | mannered | ornate | Dearest, dear, Ах, Lord, oh, ah, doth, esteemed, Herr, Mr, gentlemen, gentle, Oh, shall | | length | brief | Brief, ито, 통해, ‘ , minimal, Box, 証拠, / ;, 위해, ModelForm, 曾在, short, endforeach, katanya | | length | tokens | minimal, ито, min, ミニ, eg,