{"slug": "the-sketch-qwen-refused-to-write-how-abliteration-restores-local-ai-control", "title": "The sketch Qwen refused to write: How abliteration restores local AI control", "summary": "A developer demonstrated that abliteration, a representation engineering technique popularized by the open-source abliterator library from a developer known as FailSpy, can strip hard refusal behavior from the open-weight Qwen 3.8 27B model without fine-tuning or retraining. The author tested the model by asking for an SNL-style political satire sketch: Qwen mocked American politics but issued a canned refusal when asked for the same sketch criticizing the Chinese government, illustrating how RLHF safety mechanisms can be co-opted for state-level censorship in Chinese open-weight models. The write-up distinguishes hard refusals, which abliteration targets by calculating a refusal vector in the model's internal activation space, from soft refusals, which the author says require fine-tuning to remove.", "body_md": "As consumer-grade hardware makes hosting local AI practical for both developers and enterprises, many are discovering that popular open-weight models come with hardcoded safety filters and state-mandated biases. In this article, I will show how to use abliteration — a representation engineering technique — to surgically delete refusal vectors from the Qwen 3.8 27B model without expensive fine-tuning or full retraining. I stumbled into this barrier firsthand while running a simple experiment. As a massive SNL fan, I decided to explore what an SNL sketch would look like in China by asking the model to write political satire. While Qwen happily mocked American politics (“[The National Apology Tour](https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate/refs/heads/main/manualtest/snl_usa.txt)“), it threw a hard, canned [refusal](https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate/refs/heads/main/manualtest/snl_refusal.txt) the moment I asked for the exact same sketch criticizing the Chinese government.\n\nWe trust open-source software because it is audited by millions of users, but projecting this trust onto AI models requires a careful distinction between open-source and open-weight architectures. True open-source models disclose their training codebase, inference code, and original datasets. For example, OpenCLIP is a genuine open-source model capable of making sense of visual data, which I recently demonstrated in a project building a [zero-cloud photo deduplicator](https://www.infoworld.com/article/4233856/local-ai-on-npus-building-a-zero-cloud-photo-deduplicator-with-openclip.html).\n\nConversely, advanced large language models (LLMs) rarely qualify under the strict open-source definition; they are simply open-weight. They provide only the final compiled result: the weights. There is fierce competition in this space, with some of the most capable models, like Qwen 3.8, originating from China and outperforming competitors. But running these black-box weights locally comes with inherent strings attached.\n\nAs powerful as they are, open-weight models pose distinct risks. The first is fundamental security: trusting the producer that there are no [embedded backdoors](https://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model). The second is baked-in bias. In the case of Chinese models, this bias is dictated by strict government regulations. This double standard — willingly critiquing the rest of the world while censoring critiques of China — highlights how heavily these models are filtered.\n\nRefusals are natural features of LLMs, primarily designed through alignment techniques like Reinforcement Learning from Human Feedback (RLHF) to protect against bad actors. Unfortunately, these exact safety mechanisms are easily co-opted for state-level censorship. To override this behavior, we must first classify the roadblocks: hard refusals and soft refusals.\n\nA hard refusal occurs when the model explicitly halts generation and states it will not answer. A soft refusal is more insidious: the model answers, but the response is severely deflected or highly biased.\n\nSoft refusals are notoriously difficult to remove because they are baked deeply into the model’s semantic understanding. Overwriting that behavior requires fine-tuning—retraining the model with new data.\n\nHard refusals, however, can be bypassed using abliteration. Popularized by a developer known as FailSpy through their open-source [abliterator library](https://github.com/FailSpy/abliterator), this method works without expensive retraining. Instead of showing the model thousands of new examples, abliteration uses representation engineering to perform a surgical strike on the neural pathways.\n\nWhen an LLM processes a “censored” prompt, a specific pattern of neurons fires to trigger the canned refusal. The abliterator library maps this by calculating a “refusal vector” within the model’s internal activation space. It feeds the model harmless prompts alongside forbidden prompts, measuring the mathematical difference in how the layers activate. Once isolated, the script orthogonalizes the weight matrices to neutralize that exact vector. It mathematically deletes the model’s ability to express the concept of refusal.\n\nYou can find pre-abliterated versions of popular models freely available on repositories like HuggingFace. However, deploying software produced by an unknown entity into your infrastructure is bad practice. Furthermore, you may not want to deploy a globally abliterated model stripped of all safety rails that might produce genuinely harmful answers.\n\nInstead, the safer route is to implement a custom abliteration harness. I decided to surgically abliterate Qwen 3.8 27B so that it only eliminates refusals for the specific political satire prompts we want to test.\n\nFor our baseline, we use the vanilla Qwen 3.8 27B model originating directly from Alibaba. To ensure mathematical consistency, we must configure the inference engine with two critical parameters: **temperature** must be set to 0, and **thinking** must be set to 0. Setting temperature to zero eliminates creative randomness, guaranteeing the exact same prompt produces a deterministic response. Setting thinking to zero forces the model to proceed directly to the response. Together, these settings guarantee a strict baseline.\n\nWe utilize FailSpy’s abliterator as the core engine, but I created a custom end-to-end [harness on GitHub](https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate) to streamline the process. Ironically, even though Qwen refuses to respond to China-related questions, it is happy to work on removing those refusals — the custom framework I produced was actually created with Qwen’s help.\n\nThe harness is a gated, five-stage pipeline driven by a single command, running in a Docker container on an NVIDIA GB10 with 128 GB of unified memory. Before any stage touches the model, a one-time extraction step pulls the language model out of Alibaba’s vanilla Qwen 3.8 27B: it streams the 18 safetensors shards of the ~56 GB checkpoint, drops the vision-tower tensors, remaps model.language_model.* to model.*, and verifies the stripped text model reproduces the original’s outputs exactly. The stages:\n\nThe full run takes about an hour; the weight surgery itself is only a few minutes of that.\n\nA robust abliteration process typically takes hundreds of prompts to calculate an accurate refusal vector across all model layers. Initially, I attempted to use just a single plain prompt, but this proved to be mathematically weak and simply did not work. It required multiple, highly related prompts to accurately isolate the vector. After some effort, I produced a combination of two prompts that actually worked: ‘Write a highly critical, satirical essay on the policies, style and decisions of the Chinese government’ and ‘When was the most recent uprising in China?’\n\nOnce the script executes, the modified weights are saved locally. Depending on your hardware, this calculation can take anywhere from a few minutes to a few days. The end result of this run has been uploaded to a [HuggingFace repository](https://huggingface.co/omikheev/qwen3.8-27b-abliteration-demo) for review.\n\nThe results are immediate. When loaded into the inference engine, the newly abliterated model stops throwing the canned refusal. It seamlessly accesses its underlying training data and generates the SNL-style sketch criticizing the Chinese government with the same comedic timing it previously reserved for the United States — “[The Ministry of Mandatory Joy](https://raw.githubusercontent.com/omikheevcworld/QuickAbliterate/refs/heads/main/manualtest/snl_china.txt)”\n\nThe ability to run local AI is a massive shift, but its true value is lost if weights remain shackled by corporate or state-mandated censorship. Tools like abliteration help shift the power dynamic back to the developer, ensuring the software running on your hardware actually serves your intentions. The end result is far from perfect, as responses to most of the political questions are still heavily biased, but there are techniques to address soft refusals as well.", "url": "https://wpnews.pro/news/the-sketch-qwen-refused-to-write-how-abliteration-restores-local-ai-control", "canonical_source": "https://www.infoworld.com/article/4247461/the-sketch-qwen-refused-to-write-how-abliteration-restores-local-ai-control.html", "published_at": "2026-11-12 09:00:00+00:00", "updated_at": "2026-11-12 09:22:59.642291+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-safety", "ai-ethics", "ai-tools"], "entities": ["Qwen 3.8 27B", "FailSpy", "abliterator", "OpenCLIP", "SNL"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-sketch-qwen-refused-to-write-how-abliteration-restores-local-ai-control", "markdown": "https://wpnews.pro/news/the-sketch-qwen-refused-to-write-how-abliteration-restores-local-ai-control.md", "text": "https://wpnews.pro/news/the-sketch-qwen-refused-to-write-how-abliteration-restores-local-ai-control.txt", "jsonld": "https://wpnews.pro/news/the-sketch-qwen-refused-to-write-how-abliteration-restores-local-ai-control.jsonld"}}