Geometric Configurations: How Perturbed Jailbreaks Look to LLMs

A new study of perturbed jailbreak prompts in large language models reveals that internal representations in the last-layer-last-token embedding space and top-50 next-token probability space lack a clear 'behavioral hyperplane' separating refusal from compliance. Researchers found that in the Qwen-2.5-1.5B model, the token 'Sure' strongly correlates with a compliant response, while in Llama models, the comma and double newline serve as similar pivot tokens, highlighting the fragility of LLM safety boundaries.

Geometric Configurations: How Perturbed Jailbreaks Look to LLMs The research looked at two specific areas: the last-layer-last-token embedding space and the top-50 next-token probability space. The results are pretty telling for anyone interested in LLM security and prompt engineering. Internal Representation Analysis Embedding Space: The last-layer embeddings tend to group prompts based on their surface-level characteristics—things like spelling errors, weird formatting, or specific character perturbations. It's more about the "look" of the prompt than the intent. Probability Space: The top-50 next-token space is functionally one-dimensional but way messier to cluster. It tracks what the model is likely to say next. The most interesting finding is the lack of a "behavioral hyperplane." In simpler terms, there isn't a clean, mathematical line in these spaces that separates "refusal" from "compliance." However, a few specific tokens act as massive tells for a successful bypass. In the Qwen-2.5-1.5B model, the token "Sure" is heavily associated with a compliant response. For Llama models, the comma "," and the double newline "ĊĊ" show a similar correlation. This suggests that while the model might be "tricked" by the perturbation, the transition from refusal to compliance often hinges on a few critical pivot tokens that signal the model has entered a "helpful" state rather than a "safe" state. For those building a real-world AI workflow or researching LLM agents, this highlights just how fragile the boundary between a guarded response and a jailbroken one actually is. Next Video LLMs are failing a basic logic test → /en/threads/2495/