{"slug": "help-with-qwen-and-heretic", "title": "Help with qwen and Heretic", "summary": "A community derivative of Qwen2.5-1.5B-Instruct, processed with Heretic to reduce refusal behavior, is available as a Q4_K_M GGUF file on Hugging Face from user saidutta69. The derivative is intended for local experimentation, such as on Android apps, and does not add capability or judgment beyond the base model. Users are advised to test the Heretic model in the same app and settings before making other changes, and to note that it lacks an extra safety-filtering layer.", "body_md": "Oh. I think there’s probably a simpler way to do this:\n\nIf you are already running `Qwen2.5-1.5B-Instruct-Q4_K_M.gguf` locally on your phone, you probably **do not need to run Heretic itself on the phone**.\n\nThere is already a Heretic-processed version of the same Qwen2.5 1.5B model with a `Q4_K_M` GGUF available here:\n\n[saidutta69/Qwen2.5-1.5B-Instruct-heretic](https://huggingface.co/saidutta69/Qwen2.5-1.5B-Instruct-heretic)\n\nThe repo currently provides:\n\n```\nQwen2.5-1.5B-Instruct-heretic-Q4_K_M.gguf\n```\n\nSo, if the Android app you are using can load ordinary GGUF models, the first thing I would try is simply:\n\n```\nyour current Qwen2.5-1.5B-Instruct Q4_K_M\n                    ↓\n          same Android app\n          same prompt\n          same/default-ish settings\n                    ↓\nQwen2.5-1.5B-Instruct-heretic Q4_K_M\n```\n\nIn other words: **change the model file first, not everything else at the same time.**\n\nThat gives you a much cleaner test.\n\nThe important caveat is that this will mainly help if “I can’t get it to do what I want” means that the normal Qwen model is **refusing requests**. Heretic modifies the model weights to reduce refusal behavior; it does not turn a 1.5B model into a more capable model.\n\nThe Heretic model card makes the same distinction: the modification suppresses refusals, but does not add capability or judgment.\n\nHow I would test it\nOne final small note: the Heretic model above is a **community derivative**, not an official Qwen release. Its model-card evaluation numbers are the uploader’s evaluation, so I would treat them as useful information rather than an independent guarantee.\n\nIt also intentionally removes a lot of refusal behavior. For private local experimentation that may be exactly what you want, but if you ever expose it as a service to other people, remember that the model card explicitly says there is no extra safety-filtering layer.\n\nFor your current phone setup, though, I would start with the simple experiment: **download the Heretic `Q4_K_M` GGUF and try it in the same local app before changing anything else.**", "url": "https://wpnews.pro/news/help-with-qwen-and-heretic", "canonical_source": "https://discuss.huggingface.co/t/help-with-qwen-and-heretic/179656#post_4", "published_at": "2026-09-09 21:54:47+00:00", "updated_at": "2026-09-09 22:16:33.109624+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools"], "entities": ["Qwen2.5-1.5B-Instruct", "Heretic", "saidutta69", "Hugging Face", "Q4_K_M"], "alternates": {"html": "https://wpnews.pro/news/help-with-qwen-and-heretic", "markdown": "https://wpnews.pro/news/help-with-qwen-and-heretic.md", "text": "https://wpnews.pro/news/help-with-qwen-and-heretic.txt", "jsonld": "https://wpnews.pro/news/help-with-qwen-and-heretic.jsonld"}}