{"slug": "how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-1", "title": "How to Use unsloth/Qwen3.8–27B-GGUF in Claude Code via Ollama Without Dying in the Process? (1/2)", "summary": "A clash between Claude Code's system-message injection and Qwen3.8's chat template causes a 500 error when using the unsloth/Qwen3.8–27B-GGUF model via Ollama, with the failure occurring in under 300 milliseconds during prompt assembly. The issue stems from Claude Code inserting system-role messages mid-conversation, which Qwen's Jinja template rejects, and Ollama is not at fault. The article, part one of a guide, explains the root cause and promises a template patch in part two.", "body_md": "It’s 11 p.m. You have 27 billion parameters sitting comfortably on your GPU, Claude Code installed, and the Ollama server running. You type your first prompt and get this:\n\n```\nError: Jinja Exception: System message must be at the beginning.[GIN] | 500 | 262.037611ms | POST \"/v1/messages?beta=true\"\n```\n\nYou try again. Same error. You download the model again. You reinstall Ollama. You reinstall Claude Code. Nothing. You check the firewall, the endpoints, the tunnel. Still nothing.\n\nAnd then comes the best part: you run **ollama run** with that same model, and it works perfectly. It converses, reasons, and responds. But as soon as you connect it to Claude Code, you get a 500 error in less than 300 milliseconds.\n\nCalm down. It’s not your GPU. It’s not quantization. It’s not Ollama. And no, it’s not the mischievous spirits having a field day with you either. It’s a clash of conventions between two ecosystems that never sat down to agree on where system messages can go within a conversation.\n\nThis first part is about understanding the problem and patching the template. In the second part, we’ll create the models, deploy them, and connect Claude Code. Plus, a few tips on what worked reasonably well for me.\n\nThose three hundred milliseconds are the first clue. Your GPU never even found out about it: the request died during prompt assembly, before generating a single token. What failed wasn’t inference, it was the chat template: a Jinja program that travels inside the GGUF file itself, in the **tokenizer.chat_template** metadata, whose job is to turn your conversation’s message list into the plain text the model actually sees (role tokens, turn delimiters, generation prefix).\n\nQwen3.8’s template has a rule written into it: **system role** messages go at the beginning of the conversation, and nowhere else. If one shows up mid-history, the template aborts rendering with an exception, and the text of that exception is literally the text of your 500 error. In Step 3 we’ll open the file and see the exact line; for now, let’s just hold onto the rule.\n\nAnd who’s sending a system message mid-conversation? Claude Code. Since around May 2026, in addition to the “official” system prompt that travels as a parameter separate from the history, Claude Code injects **{“role”: “system”, …}** messages directly inside the messages array. It uses them for prompt caching and to slip context between turns, and it doesn’t need a long conversation to do it: they travel from the session’s very first requests. That’s why it fails on the first prompt. The change, in fact, broke half the ecosystem of API proxies and translators all at once; it’s documented as a breaking change in several projects.\n\nOllama isn’t the culprit, and it’s worth saying so because it’s the first place anyone looks. That this story is even possible is to its credit: the **/v1/messages?beta=true** in the log is the support for Anthropic’s Messages schema that Ollama exposes precisely so clients like Claude Code can talk to local models. The **?beta=true**, by the way, doesn’t mean Ollama’s support is in beta: it’s a query param that Claude Code itself adds to announce the Anthropic API beta features it uses (it sends the same thing to **/v1/messages/count_tokens?beta=true**). And that layer does its job well: it translates the request and hands the template the conversation exactly as the client sent it, no more and no less. The clash isn’t Ollama against anyone: it’s Claude Code’s new habit against the discipline of Qwen’s template, with Ollama as the faithful messenger in the middle.\n\nNor is it the famous **<system-reminder>** blocks that Claude Code keeps inserting during the session (task status, modified-file notices, **CLAUDE.md** contents). Those travel as text inside **user role messages** or attached to a **tool_result**, never as a **system role**. They eat your context window in irritating ways, but they’re innocent of this particular error.\n\nAnd that’s why **ollama run** works: a single system at the start, with no client injecting anything mid-array. The template’s rule is never touched.\n\nThe solution, then, is to teach the template to handle those messages instead of spitting out a 500. And since the template lives inside the GGUF, the route is this: get the file (Step 1), extract the template from it (Step 2), read it to understand exactly what it forbids (Step 3), patch it (Step 4), and rewrite the GGUF metadata with the new template (Step 5). Let’s take it piece by piece.\n\nJust like me, your first instinct may be this:\n\n```\nFROM hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XLTEMPLATE \"\"\"{%- set image_count = namespace(value=0) %}...\"\"\"\n```\n\nThe Modelfile’s **TEMPLATE** directive only accepts Go templates, not Jinja:\n\n```\nError: 400 Bad Request: template error: template: :6: function \"content\" not defined\n```\n\nGo interpreted **{{- content }}** as a call to a function named “content”. It’s the same error half the community reports with other models, with the name of the nonexistent function changing depending on which one comes up first: **bos_token**, **raise_exception**, whatever. When the model comes from Ollama’s own registry, Ollama doesn’t even touch the Jinja: it uses renderers written in Go (**model/renderers/qwen35.go** and friends). But for a GGUF imported from Hugging Face there’s no assigned renderer, so Ollama does execute the Jinja embedded in the file (hence the error arriving with that text) and gives you no way to hand it a different Jinja from the outside (**e.g. TEMPLATE**).\n\nOn August 14 this exact error was reported with Ollama’s official qwen3.8:27b(issue #17754)and they closed it the same day with PR #17757, whose fix does in Go the same thing we’re going to do in Jinja: hoist late system messages into the initial block. Practical translation: if you’re using the model from Ollama’s registry with an updated version, you don’t need a patch. This procedure is for the Hugging Face GGUF, where what runs is the file’s template.\n\nConclusion: if we want a different template, we have to put it inside the GGUF file.\n\nThere are three routes, and the one you pick changes the rest of the procedure.\n\n**Route A: **you already have it in Ollama**.** If you ran **ollama pull hf.co/unsloth/Qwen3.8–27B-GGUF:UD-Q4_K_XL**, the file is in the internal store under a name that is its hash. To locate it:\n\n```\nollama show --modelfile hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL | grep -m1 '^FROM'# FROM /home/tu-usuario/.ollama/models/blobs/sha256-3f227079003add...\n```\n\nNo guessing which is the biggest file in the folder: if you have two Qwen 27Bs installed they look far too alike and you’ll patch the wrong one.\n\n**Route B:** the browser. You go to [unsloth/Qwen3.8–27B-GGUF · Hugging Face](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF), open the Files and versions tab, and click the **.gguf** you want. That’s it. Nobody is holding a gun to your head to make you do everything from the CLI — or are you farming aura? If you’re about to pull down 17 GB over a home connection, a browser download manager will give you better recovery from dropouts than anything you improvise with **curl**.\n\n**Route C:** the CLI, when it makes sense. Here it does win: when you want several files at once, or when you’re on a server with no GUI (e.g. via SSH). In my case I used it (lie — I downloaded both manually) because I also needed the multimodal projector, which didn’t come with the **ollama pull**:\n\n```\npip install -U huggingface_hubhf download unsloth/Qwen3.8-27B-GGUF \\  --include \"*UD-Q4_K_XL*\" \"*mmproj*\" \\  --local-dir /mnt/workspace/ollama/qwen38\n```\n\nYou end up with the model’s **.gguf** and an **mmproj-F16.gguf** (we won’t be using this one for now) next to it, both in a folder of your own. If you have an old version of the client, the command is called **huggingface-cli download**.\n\nOne nuance before moving on: Unsloth has been updating this repo’s templates, so whichever one your GGUF carries depends on when you downloaded it.\n\n```\nollama show hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL --template > unsloth-qwen3.8-27b-gguf.jinja\n```\n\nOne hundred eighty-four lines of Jinja. Breathe.\n\nIf you went with Route B or C and the model isn’t imported into Ollama yet, pull it straight out of the file:\n\n```\npip install -U ggufpython3 -m gguf.scripts.gguf_dump --no-tensors --json Qwen3.8-27B-UD-Q4_K_XL.gguf \\  | jq -r '.metadata[\"tokenizer.chat_template\"].value' > unsloth-qwen3.8-27b-gguf.jinja\n```\n\nOpen **unsloth-qwen3.8–27b-gguf.jinja.** Most of it is rendering macros, multimodal content handling, and tool-call formatting; what matters to us lives in two places.\n\nUp top, a collector that walks the messages and absorbs the system ones sitting at the beginning:\n\n```\n{%- set sysns = namespace(count=0, text='') %}{%- for message in messages %}    {%- if sysns.count == loop.index0 and (message.role == 'system' or message.role == 'developer') %}\n```\n\nRead the condition **sysns.count == loop.index0** calmly: it accepts one, two, or twenty system messages, as long as they’re contiguous and at the start, and it merges them into a single initial block. The **developer **role goes in the same bucket.\n\nAnd further down, in the loop that renders the conversation turn by turn, the guardian that spat out your 500. In my file it’s at line 110, but the number moves between revisions, so find it with **grep**, **Ctrl+F**, or with your built-in ocular system (GA on humans for the last 2 million years):\n\n```\ngrep -n \"raise_exception('System message\" qwen38.jinja\n{%- if message.role == \"system\" or message.role == \"developer\" %}    {{- raise_exception('System message must be at the beginning.') }}\n```\n\n**raise_exception **is the helper the Jinja environment exposes to abort rendering, and the string in your error is literally the one we saw at the top of the article. Put the two pieces together and only one thing could have happened: a system message came mid-conversation. Exactly what Claude Code is sending. There’s no other way to hit that line.\n\nThree changes to what we just read.\n\n**1)** Add a second compartment to the collector’s namespace:\n\n```\n{%- set sysns = namespace(count=0, text='', late='') %}\n```\n\n2) Add an **elif **that captures the late arrivals, right before that **for's endif:**\n\n```\n{%- elif message.role == 'system' or message.role == 'developer' %}    {%- set sys_content = render_content(message.content, false, true)|trim %}    {%- if sys_content %}        {%- set sysns.late = sysns.late + ('\\n' if sysns.late else '') + sys_content %}    {%- endif %}\n```\n\nAnd join the two compartments:\n\n```\n{%- set merged_system = sysns.text + ('\\n' if sysns.text and sysns.late else '') + sysns.late %}\n```\n\n3) In the rendering loop, turn the exception into an empty branch:\n\n```\n{%- if message.role == \"system\" or message.role == \"developer\" %}    {#- ya fue izado al bloque de sistema inicial; aquí no emitimos nada #}\n```\n\nSave it as **unsloth-qwen3.8–27b-gguf-cc.jinja** (cc for **Claude Code**), or however suits you best; just don’t mix up your files.\n\nTwo warnings worth their weight in gold:\n\n```\npip install -U ggufgguf-new-metadata \\  --chat-template-file unsloth-qwen3.8-27b-gguf-cc.jinja \\  Qwen3.8-27B-UD-Q4_K_XL.gguf \\  Qwen3.8-27B-UD-Q4_K_XL-CC.gguf\n```\n\n(If the executable isn’t on your PATH, **python3 -m gguf.scripts.gguf_new_metadata** does exactly the same thing.)\n\n**Absolute or relative paths?** Relative ones work perfectly. The tool uses **pathlib.Path** and passes the paths straight to **open()** without any normalization, so **./my-model.gguf **will resolve in your current directory without a problem. If you see tutorials with giant paths like **/home/user/.ollama/models/blobs/sha256–3f2270…**, that’s only because the original file lives in Ollama’s store and not in their working folder. If you’re following Route B or C from Step 1, a simple **./name.gguf** is enough.\n\n**Why not edit the file in the store directly?** Because Ollama uses content-addressed storage. Each file’s name is the SHA-256 of what’s inside it, and the manifest records that hash along with its exact size in bytes. If you touch the file, you break all three references at once (it’s the same reason nobody would dream of hand-editing something inside **.git/objects**).\n\nHere’s the finished **.jinja**: [A patched version that moves and concatenates late system messages into the initial block without altering the conversation indices, preventing the model from silently discarding them or skipping user turns.](https://gist.github.com/malexandersalazar/9cdc8d3b1554ad583c80f96316e18580)\n\nAnd that’s it! Your **Qwen3.8–27B-UD-Q4_K_XL-CC.gguf** now has the template built in and is ready to be served to Claude Code through Ollama.\n\n[How to Use unsloth/Qwen3.8–27B-GGUF in Claude Code via Ollama Without Dying in the Process? (1/2)](https://pub.towardsai.net/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-the-process-1-2-31acd5105227) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-1", "canonical_source": "https://pub.towardsai.net/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-the-process-1-2-31acd5105227?source=rss----98111c9905da---4", "published_at": "2026-08-27 21:01:01+00:00", "updated_at": "2026-08-27 21:21:41.903436+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools"], "entities": ["Claude Code", "Ollama", "Qwen3.8", "unsloth", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-1", "markdown": "https://wpnews.pro/news/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-1.md", "text": "https://wpnews.pro/news/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-1.txt", "jsonld": "https://wpnews.pro/news/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-1.jsonld"}}