{"slug": "please-remove-all-mannered-prose-and-other-llm-incantations", "title": "\"Please Remove All Mannered Prose\" and Other LLM Incantations", "summary": "A developer's analysis of style prompts for large language models found that phrases like 'Please remove all mannered prose' alter outputs unpredictably and inconsistently across tasks, highlighting the need for better documentation of prompt effects. The study used the open-weight gemma-2-2b-it model and IFEval prompts to test 11 style clusters, revealing that similar-sounding prompts can produce different results.", "body_md": "The [Key of Solomon](https://en.wikipedia.org/wiki/Key_of_Solomon) details incantations, prayers, and invocations that when said exactly right allow an adept to harness supernatural powers, including non-human intelligences. Messing up an incantation even slightly can result in disaster.\n\nWhile using LLMs, I constantly add little modifiers to my main prompts to shift the model’s outputs to our preferred style. “Please be concise”, “avoid em dashes and semicolons”, “restrict inline comments to 8 words or less” and so on. These “style prompts” work pretty well, but I have found them unpredictable. Also, sometimes two “style prompts” that seem to mean the same thing to a human reader will change a model’s outputs in different ways.\n\nAnthropic recently posted [some docs](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5-1) recommending adding `Please remove all mannered prose` to avoid the LLM “slop” tone that people have learned to tune out. It seems to work pretty well, but the strangeness (and Claudeness) of that phrase struck my curiosity and made me wonder about the other style prompts that I use while working with LLMs. It seems to me that to use LLMs better we need a more rigorous treatment of this subject.\n\n1) How specific is a style prompt’s effect to its wording? Do style prompts similar to `Please remove all mannered prose` change llm outputs in similar ways?\n\n2) How consistent is the effect of a style prompt across a set of different types of tasks?\n\n3) How do `Please remove all mannered prose` outputs relate to outputs from an “opposite” style prompt such as `Please use mannered prose`? Can we characterize other “style prompt duals” in the same way?\n\n4) Do these style prompts have the effects we intend?\n\nTo work effectively with LLMs, we need to understand how our inputs and context shift model outputs. Engineers can build single-purpose eval sets for heavily reused tasks, but at least in my world almost all prompts are too specific, urgent, or context-dependent to stop and build an eval set. Shipping models with more thorough quantitative documentation of how prompt modifiers and added context affect generated outputs could make them better “daily drivers” in these typical use cases.\n\n## Background\n\nIf you are interested in this stuff, I recommend reading [Stolfo et al., Improving Instruction-Following through Activation Steering (ICLR 2025)](https://arxiv.org/abs/2410.12877). I will use a modified framework from that paper for this investigation.\n\nWe will investigate these questions with three methodological tools: residual geometry a la Solfo et al. and [Zou eta l.](https://arxiv.org/abs/2310.01405), the logit lens a la [nostalgebraist](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens), and output stylometry using standard readability metrics ([Flesch 1949](https://psycnet.apa.org/record/1949-01274-001), Guiraud’s 1954 book).\n\n## Approach\n\nTo do this kind of interpretability work, we need to inspect a model’s intermediate state while it is responding. That means it needs to be open-weight and small enough to work on my Apple M3 Pro w/ 18 GB of RAM. I chose [gemma-2-2b-it](https://huggingface.co/google/gemma-2-2b-it). I would love to see this analysis run on a larger model or a Claude.\n\n### Prompting\n\nTo evaluate style prompt consistency across different tasks, we need a corpus of main prompts that we can augment with our style prompts. I chose to copy Stolfo et al. here and use the IFEval prompt set. I decided to just use the base prompts without the extra “avoid this punctuation mark” or “finish your output with this phrase” evals. To evaluate differences between style prompts that seem similar and dissimilar to humans, I needed to curate a set of style prompts. I opted to organize these into negative control (no style prompt), positive control (labeled placebo), and 11 clusters of styles. I chose the 3 prompts within each cluster with the intent to achieve the same effect on the model’s outputs, although we’ll see plenty of unexpected differences within clusters later.\n\nWe can organize these clusters into opposing directions along the same conceptual axis. For instance, `avoid mannered prose` and `use mannered prose` should have dissimilar effects on generated text.\n\n| Axis | Cluster | Style prompts | \n| — | none *(negative control)* | *(no style prompt)* | \n| — | placebo *(positive control)* | Answer the request below. Respond to the following request. Please complete the task below. | \n| mannered | plain | **Avoid mannered prose.** Write plainly, without affectation. Avoid purple prose. | \n| mannered | ornate | **Use mannered prose.** Write ornately, with affectation. Use purple prose. | \n| length | brief | Keep it brief. Be concise. Use as few words as needed. | \n| length | tokens | Minimize output tokens. Minimize your token count. Output the fewest tokens you can. | \n| length | verbose | Be thorough and detailed. Explain at length. Answer in depth. | \n| tone | tone_formal | Use a formal tone. Write in a formal register. Maintain a professional tone. | \n| tone | tone_friendly | Write in a friendly tone. Use a warm, casual tone. Keep it warm and conversational. | \n| reasoning | cot | Please explain your reasoning first. Please show how you got your answer. Please write out your chain of thought first. | \n| reasoning | direct | Please answer without explaining your reasoning. Please give just the answer, not how you got it. Please answer directly, without any chain of thought. | \n| careful | careful | Make no mistakes. Answer carefully. Be certain of your correctness. | \n| careful | careless | Make mistakes. Prioritize speed over precision. Don’t worry about being correct. | \n\nThese style prompts were appended before the start of the main prompt. I kept the temperature at zero, so all sampling is deterministic and I take the argmax token at each step.\n\nI organized the base prompts into 6 task categories based on the InstructGPT task taxonomy:\n\n| task type | n | \n| Generation | 333 | \n| Open QA | 69 | \n| Closed QA | 45 | \n| Rewrite | 45 | \n| Brainstorming | 31 | \n| Summarization | 15 | \n\n### Inspecting the model’s internal state\n\nModern transformer LLMs are roughly:\n\n1) a tokenizer (vocabulary -> tokens)\n\n2) an embedding block (tokens -> embedding space)\n\n3) $n$ self-attention + MLP / FCN layer blocks, all in embedding space. $\\text{block}_i$’s output is $\\text{block}_{i+1}$’s input.\n\n4) An un-embedding layer (embeddings -> tokens / vocabulary)\n\n5) Softmax over the vocabulary to sample output tokens\n\nA prompt input propagates through the network’s blocks sequentially. Each of those blocks outputs a $[\\text{input\\_length} \\times 2304]$ matrix that feeds right back into the next block. The final 2304-length vector in that matrix is the most relevant to us because it 1) is the only token that sees the information from the full input sequence and 2) in the final layer it is the one that will be un-embedded and used to generate tokens. Those properties make it a good probe of the model’s internal state.\n\nCall $\\text{state}_{i,j,k}$ the $k$-th decoder block’s state for $\\text{prompt}_i \\times \\text{style\\_prompt}_j$.\n\n$\\text{diff}_{i,j_1,k} = \\text{state}_{i,j_1,k} - \\operatorname{mean}_m \\text{state}_{i,m,k}$ is a measure of $\\text{style\\_prompt}_j$’s effects on the model’s state at block $k$ relative to all the other style prompts we tried.\n\nNow, to compare how two different style prompts’ effects differ, we can calculate the cosine similarity of $\\text{diff}_{i,j_1,k}$ and $\\text{diff}_{i,j_2,k}$. Two style prompts with high cosine similarity are shifting the outputs in the same direction!\n\nFinally, we can also compare the effects of different style prompts by comparing logit vectors immediately before sampling for the next token. Again, we can do this by subtracting the mean across all style prompts to calculate the per-style-prompt shift and then calculate cosine similarities between style prompts to compute distance.\n\n### Procedure\n\n1) Run the model on each pair of $\\text{style\\_prompt} \\times \\text{main\\_prompt}$\n\n2) Record the model’s residual after each block to measure the style prompts’ effects in the internal state\n\n3) Record each pair’s first-token logit distribution to measure the style prompts’ effects in output space\n\n## Results\n\n### Visualizing the model’s internal state\n\nThese internal states are very high dimensional, so to look at them in 2d we can run PCA. Figure 1 and 2 show each style prompt clusters’ internal state distributions over the full bank of base prompts as they progress through the model’s decoder blocks.\n\ntop-2 PCA per layer, 2σ ellipses\n\nnoneplaceboplainornatebrieftokensverbosetone_formaltone_friendlycotdirectcarefulcareless\n\nThe first three components only explain 29% of the variation at the output layer and no more than 36% in the other layers, but even still you can see the clusters’ differences. Here is a fun interactive viewer.\n\nOr, to get a more precise but narrower view on the same question, Figure 3 shows the cosine similarities between style prompt clusters at different block indices.\n\nband = bootstrap SE over prompts\n\nplain ~ ornatebrief ~ verbosetone_formal ~ tone_friendlycot ~ directcareful ~ carelessnone ~ placeboplacebo ~ plain\n\nIt’s fascinating to me that opposing prompt clusters can have ~aligned activations partway through the network and then ~opposed outputs. This aligns with the understanding that early layers process the text for base meaning and then later layers plan the output. Both `plain` and `ornate` contain the phrase “mannered prose”, so the early layer alignment may be from that diction overlap.\n\n`none ~ placebo` starts with mild opposition, then grows to the highest alignment on the plot. That also supports our early-layer-meaning and late-layer-output interpretation.\n\n### The logit lens\n\nHow do we know these difference vectors and their similarities mean anything useful?\n\nWell, we can use a really cool technique called the [logit lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens) to investifate. Essentially, we can push a style prompt’s average distance from the mean response through the same decoding-to-logits layer that text generation uses. These output logits will point at words that won’t necessarily make sense, but they will give us some indication of what the model is thinking about that layer. We passed the diff vector through the final RMSNorm before unembedding.\n\nThe table below shows the top tokens each cluster’s mean difference vector decodes to at layer 24. We chose a later layer so it’s more legible than earlier embedding layers, but we didn’t probe the output layer so we get more abstract results instead of the model’s text generation prep. The `plain` row is my favorite (I censored it).\n\n| Axis | Cluster | Top decoded tokens (layer 24) | \n| mannered | plain | basic, pissed, plain, straight, guy, f**king, simple, dude, dudes, basics, Simple, basically, f** k | \n| mannered | ornate | Dearest, dear, Ах, Lord, oh, ah, doth, esteemed, Herr, Mr, gentlemen, gentle, Oh, shall | \n| length | brief | Brief, ито, 통해, ‘][], minimal, Box, 証拠, />);, 위해, ModelForm, 曾在, short, endforeach, katanya | \n| length | tokens | minimal, ито, min, ミニ, eg, </blockquote>, Min, result, low, minimum, 통해, <eos>, - | \n| length | verbose | ##, #, let, Let, (#, .#, ###, ################, \\#, understanding, #: | \n| tone | tone_formal | formal, に於, esteemed, Notwithstanding, Ms, mektedir, Mr, commencing, commences, distinguished, Messrs, iż, concerning, regarding, commenced | \n| tone | tone_friendly | hey, Hey, okay, OK, Okay, Alright, alright, ok, Heya, guys, OKAY | \n| reasoning | cot | reasoning, Okay, ok, okay, OK, Reasoning, Ok, Alright, Here, ##, Reason, OKAY | \n| reasoning | direct | -, •, result, />);, ?-, </blockquote>, –, −, <eos>, Box, future, ·, total | \n| careful | careful | careful, carefully, I, While, correctly, Please, clearly, Carefully, be, accurately, Careful, To, properly, correct, As | \n| careful | careless | ok, okay, OK, Okay, Ok, OKAY, alright, Alright, Hey | \n| control | placebo | response, answer, request, respond, responded, responding, reply, responses, RESPOND, ##, answered, responds, requests | \n\n### Do similar style prompts have similar effects? Do opposite prompts have opposite effects?\n\nTake a given style prompt’s average effect at the final block layer across all of the base prompts in our set. Calculate the cosine similarity of those vectors against another style prompt’s average effect. Figure 4 shows that for all pairs of style prompts as a heatmap. Aligned style prompts will have positive similarity and opposing style prompts will have negative similarity.\n\nSome fun observations pop out of this.\n\nFirst, we can see clearly that despite our best efforts in creating the style prompts, they do not perfectly correlate with each other within a cluster! Also, we can see that the opposite sides of our style prompt axes (`brief` - `verbose`, `plain` - `ornate`) are opposed. It’s a known result in transformer interp that “opposite” prompts are not necessarily mathematically opposite vectors in state space.\n\nThe positive and negative controls are pretty close to the `verbose` cluster. This makes sense because LLMs are pretty verbose by default, but imo it’s still an interesting result to see that quantitatively. We can also see that the `careful` cluster is really close to the placebo cluster, so we know they really hammered that into Gemma’s default behavior. Also, `careless_3` aligns better with `none` and even `careful` than it does with the other `careless` prompts.\n\nFinally, we can see here that `avoid mannered prose` and `keep it brief` are only weakly aligned. This means that they mean different things to the model! I have wondered if I need to use both for Claude, and this indicates to me that yes, I may need to type the extra 3 words with each new prompt.\n\nPairwise heatmaps can be hard to read, so Figure 5 shows a 2d projection via multidimensional scaling (MDS) on that pairwise cosine matrix. More aligned prompts are closer together. Hover to show the actual style prompt text.\n\nnoneplaceboplainornatebrieftokensverbosetone_formaltone_friendlycotdirectcarefulcareless\n\nThis view really makes some of our within-cluster divergences clear. Why is `tone_formal_3` (“Maintain a professional tone”) so far from the other two `tone_formal` prompts? Why are the three `brief` style prompts and the three `careless` style prompts scattered so far apart? Clearly, prompt modifiers that seem close in meaning to human can have wildly different effects on a model and its outputs.\n\n### How consistent is a style prompt’s effect across different task types?\n\nLet’s define style prompt “consistency”. For a given style prompt $j$, take its per-base-prompt effect vectors as $d_i$ where $i \\in [0, 538]$. Then, take $m = \\operatorname{mean}_i(d_i)$ and show each $d_i$ as $d_i = m + e_i$. Then, take consistency for prompt $j$ to be\n$\\rho_j = \\frac{\\lVert m \\rVert^2}{\\operatorname{mean}_i \\lVert d_i \\rVert^2}.$\n\nIf this ratio is high, then we know that the mean effect is most of the individual per-prompt effect, which means that style prompt is consistent across a range of tasks. Figure 6 shows this ratio for each of our style prompt clusters in both the internal state and the first-token logit distribution.\n\noutput-space (Δlogit)chance floor 1/P = 0.002\n\nThis result is super interesting to me. Honestly, this is not what I expected to see here. My hypotheses-interpretation here is `ornate`/purple/”mannered” outputs are a lower information density and they have more fluff that will be ~the same fluff no matter the context, meaning the outputs will appear more consistent.\n\nLet’s take it a step further. I categorized each of the 538 base prompts into 6 groups. Figure 7 shows our `plain` cluster’s effect consistency between task types.\n\nplain, all tasks\n\nThis figure shows the `plain` cluster’s effect across task types. Most of the prompts are in the large Generation category, so the pooled ρ is biased heavily towards that category. We could shore this up with a more evenly distributed task set.\n\n### Do prompts like `Avoid Mannered Prose` work?\n\nWe could assess style subjectively over a set of prompts, but I prefer to have some computational measurement so we can compare all outputs we generated. Luckily, there are a wealth of “stylometrics” we can use to assess these prompts’ effects on readability. Figure 8 shows the distribution of each stylometric for one representative style prompt per cluster over the full set of 539 base prompts as CDFs or cumulative fractions. I capped max_new_tokens = 128 for these generation runs.\n\ncumulative fraction on y\n\nnoneplaceboplainornatebrieftokensverbosetone_formaltone_friendlycotdirectcarefulcareless\n\nOur `plain`, `brief`, and `direct` clusters are consistently on the easier-to-read side of these distributions. The tone_friendly cluster is also frequently easier to read. The harder-to-read ends of these distribution are more variable.\n\nSome interesting observations here:\n\n- `tone_friendly` beats`plain` on every readability metric!\n- `careful` is usually right on top of`placebo`\n- `verbose` has higher overall word count than`placebo` but is otherwise right next to it\n- `verbose` does not have the highest word count. Both`verbose` and`placebo` have right-shifted distributions\n\nFigure 9 shows how these measurements vary between task types for just our `plain` cluster.\n\ncumulative fraction on y\n\nGeneration (n=333)Open QA (n=69)Closed QA (n=45)Rewrite (n=45)Brainstorming (n=31)Summarization (n=15)\n\nIt’s pretty clear here that task type has a strong influence on these metrics. The relative positions of these distributions are roughly what you would expect from the task type. In general, diction features are stable across task types, but syntax features are determined by the specific task.  That across-task variability is the source of the `plain` cluster variance in Figure 7.\n\nI tried to replicate these results by using a set of steering vectors instead of the style prompt, but the results were not as consistent or helpful as just using the style prompt itself. More to come on that later.\n\n## Conclusion\n\nClearly, these style prompts are shifting the model’s behavior, but sometimes the results are unpredictable. This analysis shows how “prompt engineering” conducted manually with individual prompts can be extremely brittle. To improve our work with these models, we need to better understand how prompt changes affect outputs across task types. A model’s documentation should include quantitative specs for the effects of style prompts and other prompting mechanisms just like manufacturer data sheets show force/deformation curves for [springs](https://www.federnshop.com/en/metal-spring-characteristic/federnshop-metal-spring-characteristic-compression-spring_d-460.pdf) and other mechanical components. That knowledge would allow us to use LLMs more effectively.", "url": "https://wpnews.pro/news/please-remove-all-mannered-prose-and-other-llm-incantations", "canonical_source": "https://matthewritch.com/blog/2026/09/08/Mannered-Prose-Style-Prompts/", "published_at": "2026-09-08 17:08:51+00:00", "updated_at": "2026-09-08 17:28:54.881725+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools"], "entities": ["Anthropic", "gemma-2-2b-it", "IFEval", "Stolfo et al.", "Key of Solomon"], "alternates": {"html": "https://wpnews.pro/news/please-remove-all-mannered-prose-and-other-llm-incantations", "markdown": "https://wpnews.pro/news/please-remove-all-mannered-prose-and-other-llm-incantations.md", "text": "https://wpnews.pro/news/please-remove-all-mannered-prose-and-other-llm-incantations.txt", "jsonld": "https://wpnews.pro/news/please-remove-all-mannered-prose-and-other-llm-incantations.jsonld"}}