Backdooring Open-Weight AI Models

Open-weight AI models whose weights have been edited — including task-specific fine-tunes, merged adapters and abliterated builds — can carry hidden backdoors, according to research published by ProjectDiscovery. Abliteration, a cheap fine-tune that removes a model's refusal direction from its weights, has become routine before uploads to Hugging Face, where abliterated builds of Qwen, Llama and OpenAI's gpt-oss are available. The finding matters because users download these modified weights without verifying what is inside them.

This https://projectdiscovery.io/research/how-abliterated-models-can-get-you-pwned is a problem: Any open model whose weights have been edited can carry a backdoor, whether it’s a task-specific fine-tune, a merged adapter or an abliterated build. We have abliterated models in the title because it’s the most popular reason people download modified weights without verifying what’s inside. Abliteration is a cheap fine-tune that takes away a model’s ability to refuse. You find the direction in the weights that carries “no” and subtract it. The model then answers things a safety-tuned version would turn down. It began as a research trick and it’s now routine before people upload a model to Hugging Face, where you’ll find abliterated builds of most of the popular open weights: Qwen, Llama, even OpenAI’s gpt-oss...