How to Use unsloth/Qwen3.8–27B-GGUF in Claude Code via Ollama Without Dying in the Process? (1/2)

A clash between Claude Code's system-message injection and Qwen3.8's chat template causes a 500 error when using the unsloth/Qwen3.8–27B-GGUF model via Ollama, with the failure occurring in under 300 milliseconds during prompt assembly. The issue stems from Claude Code inserting system-role messages mid-conversation, which Qwen's Jinja template rejects, and Ollama is not at fault. The article, part one of a guide, explains the root cause and promises a template patch in part two.

It’s 11 p.m. You have 27 billion parameters sitting comfortably on your GPU, Claude Code installed, and the Ollama server running. You type your first prompt and get this: Error: Jinja Exception: System message must be at the beginning. GIN | 500 | 262.037611ms | POST "/v1/messages?beta=true" You try again. Same error. You download the model again. You reinstall Ollama. You reinstall Claude Code. Nothing. You check the firewall, the endpoints, the tunnel. Still nothing. And then comes the best part: you run ollama run with that same model, and it works perfectly. It converses, reasons, and responds. But as soon as you connect it to Claude Code, you get a 500 error in less than 300 milliseconds. Calm down. It’s not your GPU. It’s not quantization. It’s not Ollama. And no, it’s not the mischievous spirits having a field day with you either. It’s a clash of conventions between two ecosystems that never sat down to agree on where system messages can go within a conversation. This first part is about understanding the problem and patching the template. In the second part, we’ll create the models, deploy them, and connect Claude Code. Plus, a few tips on what worked reasonably well for me. Those three hundred milliseconds are the first clue. Your GPU never even found out about it: the request died during prompt assembly, before generating a single token. What failed wasn’t inference, it was the chat template: a Jinja program that travels inside the GGUF file itself, in the tokenizer.chat template metadata, whose job is to turn your conversation’s message list into the plain text the model actually sees role tokens, turn delimiters, generation prefix . Qwen3.8’s template has a rule written into it: system role messages go at the beginning of the conversation, and nowhere else. If one shows up mid-history, the template aborts rendering with an exception, and the text of that exception is literally the text of your 500 error. In Step 3 we’ll open the file and see the exact line; for now, let’s just hold onto the rule. And who’s sending a system message mid-conversation? Claude Code. Since around May 2026, in addition to the “official” system prompt that travels as a parameter separate from the history, Claude Code injects {“role”: “system”, …} messages directly inside the messages array. It uses them for prompt caching and to slip context between turns, and it doesn’t need a long conversation to do it: they travel from the session’s very first requests. That’s why it fails on the first prompt. The change, in fact, broke half the ecosystem of API proxies and translators all at once; it’s documented as a breaking change in several projects. Ollama isn’t the culprit, and it’s worth saying so because it’s the first place anyone looks. That this story is even possible is to its credit: the /v1/messages?beta=true in the log is the support for Anthropic’s Messages schema that Ollama exposes precisely so clients like Claude Code can talk to local models. The ?beta=true , by the way, doesn’t mean Ollama’s support is in beta: it’s a query param that Claude Code itself adds to announce the Anthropic API beta features it uses it sends the same thing to /v1/messages/count tokens?beta=true . And that layer does its job well: it translates the request and hands the template the conversation exactly as the client sent it, no more and no less. The clash isn’t Ollama against anyone: it’s Claude Code’s new habit against the discipline of Qwen’s template, with Ollama as the faithful messenger in the middle. Nor is it the famous <system-reminder blocks that Claude Code keeps inserting during the session task status, modified-file notices, CLAUDE.md contents . Those travel as text inside user role messages or attached to a tool result , never as a system role . They eat your context window in irritating ways, but they’re innocent of this particular error. And that’s why ollama run works: a single system at the start, with no client injecting anything mid-array. The template’s rule is never touched. The solution, then, is to teach the template to handle those messages instead of spitting out a 500. And since the template lives inside the GGUF, the route is this: get the file Step 1 , extract the template from it Step 2 , read it to understand exactly what it forbids Step 3 , patch it Step 4 , and rewrite the GGUF metadata with the new template Step 5 . Let’s take it piece by piece. Just like me, your first instinct may be this: FROM hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4 K XLTEMPLATE """{%- set image count = namespace value=0 %}...""" The Modelfile’s TEMPLATE directive only accepts Go templates, not Jinja: Error: 400 Bad Request: template error: template: :6: function "content" not defined Go interpreted {{- content }} as a call to a function named “content”. It’s the same error half the community reports with other models, with the name of the nonexistent function changing depending on which one comes up first: bos token , raise exception , whatever. When the model comes from Ollama’s own registry, Ollama doesn’t even touch the Jinja: it uses renderers written in Go model/renderers/qwen35.go and friends . But for a GGUF imported from Hugging Face there’s no assigned renderer, so Ollama does execute the Jinja embedded in the file hence the error arriving with that text and gives you no way to hand it a different Jinja from the outside e.g. TEMPLATE . On August 14 this exact error was reported with Ollama’s official qwen3.8:27b issue 17754 and they closed it the same day with PR 17757, whose fix does in Go the same thing we’re going to do in Jinja: hoist late system messages into the initial block. Practical translation: if you’re using the model from Ollama’s registry with an updated version, you don’t need a patch. This procedure is for the Hugging Face GGUF, where what runs is the file’s template. Conclusion: if we want a different template, we have to put it inside the GGUF file. There are three routes, and the one you pick changes the rest of the procedure. Route A: you already have it in Ollama . If you ran ollama pull hf.co/unsloth/Qwen3.8–27B-GGUF:UD-Q4 K XL , the file is in the internal store under a name that is its hash. To locate it: ollama show --modelfile hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4 K XL | grep -m1 '^FROM' FROM /home/tu-usuario/.ollama/models/blobs/sha256-3f227079003add... No guessing which is the biggest file in the folder: if you have two Qwen 27Bs installed they look far too alike and you’ll patch the wrong one. Route B: the browser. You go to unsloth/Qwen3.8–27B-GGUF · Hugging Face https://huggingface.co/unsloth/Qwen3.8-27B-GGUF , open the Files and versions tab, and click the .gguf you want. That’s it. Nobody is holding a gun to your head to make you do everything from the CLI — or are you farming aura? If you’re about to pull down 17 GB over a home connection, a browser download manager will give you better recovery from dropouts than anything you improvise with curl . Route C: the CLI, when it makes sense. Here it does win: when you want several files at once, or when you’re on a server with no GUI e.g. via SSH . In my case I used it lie — I downloaded both manually because I also needed the multimodal projector, which didn’t come with the ollama pull : pip install -U huggingface hubhf download unsloth/Qwen3.8-27B-GGUF \ --include " UD-Q4 K XL " " mmproj " \ --local-dir /mnt/workspace/ollama/qwen38 You end up with the model’s .gguf and an mmproj-F16.gguf we won’t be using this one for now next to it, both in a folder of your own. If you have an old version of the client, the command is called huggingface-cli download . One nuance before moving on: Unsloth has been updating this repo’s templates, so whichever one your GGUF carries depends on when you downloaded it. ollama show hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4 K XL --template unsloth-qwen3.8-27b-gguf.jinja One hundred eighty-four lines of Jinja. Breathe. If you went with Route B or C and the model isn’t imported into Ollama yet, pull it straight out of the file: pip install -U ggufpython3 -m gguf.scripts.gguf dump --no-tensors --json Qwen3.8-27B-UD-Q4 K XL.gguf \ | jq -r '.metadata "tokenizer.chat template" .value' unsloth-qwen3.8-27b-gguf.jinja Open unsloth-qwen3.8–27b-gguf.jinja. Most of it is rendering macros, multimodal content handling, and tool-call formatting; what matters to us lives in two places. Up top, a collector that walks the messages and absorbs the system ones sitting at the beginning: {%- set sysns = namespace count=0, text='' %}{%- for message in messages %} {%- if sysns.count == loop.index0 and message.role == 'system' or message.role == 'developer' %} Read the condition sysns.count == loop.index0 calmly: it accepts one, two, or twenty system messages, as long as they’re contiguous and at the start, and it merges them into a single initial block. The developer role goes in the same bucket. And further down, in the loop that renders the conversation turn by turn, the guardian that spat out your 500. In my file it’s at line 110, but the number moves between revisions, so find it with grep , Ctrl+F , or with your built-in ocular system GA on humans for the last 2 million years : grep -n "raise exception 'System message" qwen38.jinja {%- if message.role == "system" or message.role == "developer" %} {{- raise exception 'System message must be at the beginning.' }} raise exception is the helper the Jinja environment exposes to abort rendering, and the string in your error is literally the one we saw at the top of the article. Put the two pieces together and only one thing could have happened: a system message came mid-conversation. Exactly what Claude Code is sending. There’s no other way to hit that line. Three changes to what we just read. 1 Add a second compartment to the collector’s namespace: {%- set sysns = namespace count=0, text='', late='' %} 2 Add an elif that captures the late arrivals, right before that for's endif: {%- elif message.role == 'system' or message.role == 'developer' %} {%- set sys content = render content message.content, false, true |trim %} {%- if sys content %} {%- set sysns.late = sysns.late + '\n' if sysns.late else '' + sys content %} {%- endif %} And join the two compartments: {%- set merged system = sysns.text + '\n' if sysns.text and sysns.late else '' + sysns.late %} 3 In the rendering loop, turn the exception into an empty branch: {%- if message.role == "system" or message.role == "developer" %} { - ya fue izado al bloque de sistema inicial; aquí no emitimos nada } Save it as unsloth-qwen3.8–27b-gguf-cc.jinja cc for Claude Code , or however suits you best; just don’t mix up your files. Two warnings worth their weight in gold: pip install -U ggufgguf-new-metadata \ --chat-template-file unsloth-qwen3.8-27b-gguf-cc.jinja \ Qwen3.8-27B-UD-Q4 K XL.gguf \ Qwen3.8-27B-UD-Q4 K XL-CC.gguf If the executable isn’t on your PATH, python3 -m gguf.scripts.gguf new metadata does exactly the same thing. Absolute or relative paths? Relative ones work perfectly. The tool uses pathlib.Path and passes the paths straight to open without any normalization, so ./my-model.gguf will resolve in your current directory without a problem. If you see tutorials with giant paths like /home/user/.ollama/models/blobs/sha256–3f2270… , that’s only because the original file lives in Ollama’s store and not in their working folder. If you’re following Route B or C from Step 1, a simple ./name.gguf is enough. Why not edit the file in the store directly? Because Ollama uses content-addressed storage. Each file’s name is the SHA-256 of what’s inside it, and the manifest records that hash along with its exact size in bytes. If you touch the file, you break all three references at once it’s the same reason nobody would dream of hand-editing something inside .git/objects . Here’s the finished .jinja : A patched version that moves and concatenates late system messages into the initial block without altering the conversation indices, preventing the model from silently discarding them or skipping user turns. https://gist.github.com/malexandersalazar/9cdc8d3b1554ad583c80f96316e18580 And that’s it Your Qwen3.8–27B-UD-Q4 K XL-CC.gguf now has the template built in and is ready to be served to Claude Code through Ollama. How to Use unsloth/Qwen3.8–27B-GGUF in Claude Code via Ollama Without Dying in the Process? 1/2 https://pub.towardsai.net/how-to-use-unsloth-qwen3-8-27b-gguf-in-claude-code-via-ollama-without-dying-in-the-process-1-2-31acd5105227 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.