Help with qwen and Heretic

A community derivative of Qwen2.5-1.5B-Instruct, processed with Heretic to reduce refusal behavior, is available as a Q4_K_M GGUF file on Hugging Face from user saidutta69. The derivative is intended for local experimentation, such as on Android apps, and does not add capability or judgment beyond the base model. Users are advised to test the Heretic model in the same app and settings before making other changes, and to note that it lacks an extra safety-filtering layer.

Oh. I think there’s probably a simpler way to do this: If you are already running Qwen2.5-1.5B-Instruct-Q4 K M.gguf locally on your phone, you probably do not need to run Heretic itself on the phone . There is already a Heretic-processed version of the same Qwen2.5 1.5B model with a Q4 K M GGUF available here: saidutta69/Qwen2.5-1.5B-Instruct-heretic https://huggingface.co/saidutta69/Qwen2.5-1.5B-Instruct-heretic The repo currently provides: Qwen2.5-1.5B-Instruct-heretic-Q4 K M.gguf So, if the Android app you are using can load ordinary GGUF models, the first thing I would try is simply: your current Qwen2.5-1.5B-Instruct Q4 K M ↓ same Android app same prompt same/default-ish settings ↓ Qwen2.5-1.5B-Instruct-heretic Q4 K M In other words: change the model file first, not everything else at the same time. That gives you a much cleaner test. The important caveat is that this will mainly help if “I can’t get it to do what I want” means that the normal Qwen model is refusing requests . Heretic modifies the model weights to reduce refusal behavior; it does not turn a 1.5B model into a more capable model. The Heretic model card makes the same distinction: the modification suppresses refusals, but does not add capability or judgment. How I would test it One final small note: the Heretic model above is a community derivative , not an official Qwen release. Its model-card evaluation numbers are the uploader’s evaluation, so I would treat them as useful information rather than an independent guarantee. It also intentionally removes a lot of refusal behavior. For private local experimentation that may be exactly what you want, but if you ever expose it as a service to other people, remember that the model card explicitly says there is no extra safety-filtering layer. For your current phone setup, though, I would start with the simple experiment: download the Heretic Q4 K M GGUF and try it in the same local app before changing anything else.