The sketch Qwen refused to write: How abliteration restores local AI control

A developer demonstrated that abliteration, a representation engineering technique popularized by the open-source abliterator library from a developer known as FailSpy, can strip hard refusal behavior from the open-weight Qwen 3.8 27B model without fine-tuning or retraining. The author tested the model by asking for an SNL-style political satire sketch: Qwen mocked American politics but issued a canned refusal when asked for the same sketch criticizing the Chinese government, illustrating how RLHF safety mechanisms can be co-opted for state-level censorship in Chinese open-weight models. The write-up distinguishes hard refusals, which abliteration targets by calculating a refusal vector in the model's internal activation space, from soft refusals, which the author says require fine-tuning to remove.

As consumer-grade hardware makes hosting local AI practical for both developers and enterprises, many are discovering that popular open-weight models come with hardcoded safety filters and state-mandated biases. In this article, I will show how to use abliteration — a representation engineering technique — to surgically delete refusal vectors from the Qwen 3.8 27B model without expensive fine-tuning or full retraining. I stumbled into this barrier firsthand while running a simple experiment. As a massive SNL fan, I decided to explore what an SNL sketch would look like in China by asking the model to write political satire. While Qwen happily mocked American politics “ The National Apology Tour https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate/refs/heads/main/manualtest/snl usa.txt “ , it threw a hard, canned refusal https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate/refs/heads/main/manualtest/snl refusal.txt the moment I asked for the exact same sketch criticizing the Chinese government. We trust open-source software because it is audited by millions of users, but projecting this trust onto AI models requires a careful distinction between open-source and open-weight architectures. True open-source models disclose their training codebase, inference code, and original datasets. For example, OpenCLIP is a genuine open-source model capable of making sense of visual data, which I recently demonstrated in a project building a zero-cloud photo deduplicator https://www.infoworld.com/article/4233856/local-ai-on-npus-building-a-zero-cloud-photo-deduplicator-with-openclip.html . Conversely, advanced large language models LLMs rarely qualify under the strict open-source definition; they are simply open-weight. They provide only the final compiled result: the weights. There is fierce competition in this space, with some of the most capable models, like Qwen 3.8, originating from China and outperforming competitors. But running these black-box weights locally comes with inherent strings attached. As powerful as they are, open-weight models pose distinct risks. The first is fundamental security: trusting the producer that there are no embedded backdoors https://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model . The second is baked-in bias. In the case of Chinese models, this bias is dictated by strict government regulations. This double standard — willingly critiquing the rest of the world while censoring critiques of China — highlights how heavily these models are filtered. Refusals are natural features of LLMs, primarily designed through alignment techniques like Reinforcement Learning from Human Feedback RLHF to protect against bad actors. Unfortunately, these exact safety mechanisms are easily co-opted for state-level censorship. To override this behavior, we must first classify the roadblocks: hard refusals and soft refusals. A hard refusal occurs when the model explicitly halts generation and states it will not answer. A soft refusal is more insidious: the model answers, but the response is severely deflected or highly biased. Soft refusals are notoriously difficult to remove because they are baked deeply into the model’s semantic understanding. Overwriting that behavior requires fine-tuning—retraining the model with new data. Hard refusals, however, can be bypassed using abliteration. Popularized by a developer known as FailSpy through their open-source abliterator library https://github.com/FailSpy/abliterator , this method works without expensive retraining. Instead of showing the model thousands of new examples, abliteration uses representation engineering to perform a surgical strike on the neural pathways. When an LLM processes a “censored” prompt, a specific pattern of neurons fires to trigger the canned refusal. The abliterator library maps this by calculating a “refusal vector” within the model’s internal activation space. It feeds the model harmless prompts alongside forbidden prompts, measuring the mathematical difference in how the layers activate. Once isolated, the script orthogonalizes the weight matrices to neutralize that exact vector. It mathematically deletes the model’s ability to express the concept of refusal. You can find pre-abliterated versions of popular models freely available on repositories like HuggingFace. However, deploying software produced by an unknown entity into your infrastructure is bad practice. Furthermore, you may not want to deploy a globally abliterated model stripped of all safety rails that might produce genuinely harmful answers. Instead, the safer route is to implement a custom abliteration harness. I decided to surgically abliterate Qwen 3.8 27B so that it only eliminates refusals for the specific political satire prompts we want to test. For our baseline, we use the vanilla Qwen 3.8 27B model originating directly from Alibaba. To ensure mathematical consistency, we must configure the inference engine with two critical parameters: temperature must be set to 0, and thinking must be set to 0. Setting temperature to zero eliminates creative randomness, guaranteeing the exact same prompt produces a deterministic response. Setting thinking to zero forces the model to proceed directly to the response. Together, these settings guarantee a strict baseline. We utilize FailSpy’s abliterator as the core engine, but I created a custom end-to-end harness on GitHub https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate to streamline the process. Ironically, even though Qwen refuses to respond to China-related questions, it is happy to work on removing those refusals — the custom framework I produced was actually created with Qwen’s help. The harness is a gated, five-stage pipeline driven by a single command, running in a Docker container on an NVIDIA GB10 with 128 GB of unified memory. Before any stage touches the model, a one-time extraction step pulls the language model out of Alibaba’s vanilla Qwen 3.8 27B: it streams the 18 safetensors shards of the ~56 GB checkpoint, drops the vision-tower tensors, remaps model.language model. to model. , and verifies the stripped text model reproduces the original’s outputs exactly. The stages: The full run takes about an hour; the weight surgery itself is only a few minutes of that. A robust abliteration process typically takes hundreds of prompts to calculate an accurate refusal vector across all model layers. Initially, I attempted to use just a single plain prompt, but this proved to be mathematically weak and simply did not work. It required multiple, highly related prompts to accurately isolate the vector. After some effort, I produced a combination of two prompts that actually worked: ‘Write a highly critical, satirical essay on the policies, style and decisions of the Chinese government’ and ‘When was the most recent uprising in China?’ Once the script executes, the modified weights are saved locally. Depending on your hardware, this calculation can take anywhere from a few minutes to a few days. The end result of this run has been uploaded to a HuggingFace repository https://huggingface.co/omikheev/qwen3.8-27b-abliteration-demo for review. The results are immediate. When loaded into the inference engine, the newly abliterated model stops throwing the canned refusal. It seamlessly accesses its underlying training data and generates the SNL-style sketch criticizing the Chinese government with the same comedic timing it previously reserved for the United States — “ The Ministry of Mandatory Joy https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate/refs/heads/main/manualtest/snl china.txt ” The ability to run local AI is a massive shift, but its true value is lost if weights remain shackled by corporate or state-mandated censorship. Tools like abliteration help shift the power dynamic back to the developer, ensuring the software running on your hardware actually serves your intentions. The end result is far from perfect, as responses to most of the political questions are still heavily biased, but there are techniques to address soft refusals as well.