Patched Jinja template for Qwen 3.5 27B - fixes developer role crash + preserves thinking mode (thinking = 1). Drop-in replacement for agent tools (OpenCode, Claude Code, Continue, Cursor, Aider).

This is a patched Jinja template for the Qwen 3.5 27B model that fixes a crash when using the "developer" role and preserves the model's thinking mode (thinking=1). It serves as a drop-in replacement for agent tools like OpenCode, Claude Code, Continue, Cursor, and Aider, addressing an issue where using the standard `--chat-template chatml` would silently disable thinking mode.

- - Save sudoingX/c2facf7d8f7608c65c1024ef3b22d431 to your computer and use it in GitHub Desktop. Learn more about bidirectional Unicode characters https://github.co/hiddenchars | {%- set image count = namespace value=0 %} | | | {%- set video count = namespace value=0 %} | | | {%- macro render content content, do vision count, is system content=false %} | | | {%- if content is string %} | | | {{- content }} | | | {%- elif content is iterable and content is not mapping %} | | | {%- for item in content %} | | | {%- if 'image' in item or 'image url' in item or item.type == 'image' %} | | | {%- if is system content %} | | | {{- raise exception 'System message cannot contain images.' }} | | | {%- endif %} | | | {%- if do vision count %} | | | {%- set image count.value = image count.value + 1 %} | | | {%- endif %} | | | {%- if add vision id %} | | | {{- 'Picture ' ~ image count.value ~ ': ' }} | | | {%- endif %} | | | {{- '<|vision start| <|image pad| <|vision end| ' }} | | | {%- elif 'video' in item or item.type == 'video' %} | | | {%- if is system content %} | | | {{- raise exception 'System message cannot contain videos.' }} | | | {%- endif %} | | | {%- if do vision count %} | | | {%- set video count.value = video count.value + 1 %} | | | {%- endif %} | | | {%- if add vision id %} | | | {{- 'Video ' ~ video count.value ~ ': ' }} | | | {%- endif %} | | | {{- '<|vision start| <|video pad| <|vision end| ' }} | | | {%- elif 'text' in item %} | | | {{- item.text }} | | | {%- else %} | | | {{- raise exception 'Unexpected item type in content.' }} | | | {%- endif %} | | | {%- endfor %} | | | {%- elif content is none or content is undefined %} | | | {{- '' }} | | | {%- else %} | | | {{- raise exception 'Unexpected content type.' }} | | | {%- endif %} | | | {%- endmacro %} | | | {%- if not messages %} | | | {{- raise exception 'No messages provided.' }} | | | {%- endif %} | | | {%- if tools and tools is iterable and tools is not mapping %} | | | {{- '<|im start| system\n' }} | | | {{- " Tools\n\nYou have access to the following functions:\n\n<tools " }} | | | {%- for tool in tools %} | | | {{- "\n" }} | | | {{- tool | tojson }} | | | {%- endfor %} | | | {{- "\n</tools " }} | | | {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool call \n<function=example function name \n<parameter=example parameter 1 \nvalue 1\n</parameter \n<parameter=example parameter 2 \nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter \n</function \n</tool call \n\n<IMPORTANT \nReminder:\n- Function calls MUST follow the specified format: an inner <function=... </function block must be nested within <tool call </tool call XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT ' }} | | | {%- if messages 0 .role == 'system' %} | | | {%- set content = render content messages 0 .content, false, true |trim %} | | | {%- if content %} | | | {{- '\n\n' + content }} | | | {%- endif %} | | | {%- endif %} | | | {{- '<|im end| \n' }} | | | {%- else %} | | | {%- if messages 0 .role == 'system' or messages 0 .role == 'developer' %} | | | {%- set content = render content messages 0 .content, false, true |trim %} | | | {{- '<|im start| system\n' + content + '<|im end| \n' }} | | | {%- endif %} | | | {%- endif %} | | | {%- set ns = namespace multi step tool=true, last query index=messages|length - 1 %} | | | {%- for message in messages ::-1 %} | | | {%- set index = messages|length - 1 - loop.index0 %} | | | {%- if ns.multi step tool and message.role == "user" %} | | | {%- set content = render content message.content, false |trim %} | | | {%- if not content.startswith '<tool response ' and content.endswith '</tool response ' %} | | | {%- set ns.multi step tool = false %} | | | {%- set ns.last query index = index %} | | | {%- endif %} | | | {%- endif %} | | | {%- endfor %} | | | {%- if ns.multi step tool %} | | | {{- raise exception 'No user query found in messages.' }} | | | {%- endif %} | | | {%- for message in messages %} | | | {%- set content = render content message.content, true |trim %} | | | {%- if message.role == "system" or message.role == "developer" %} | | | {%- if not loop.first %} | | | {{- '<|im start| user\n' + content + '<|im end| \n' }} | | | {%- endif %} | | | {%- elif message.role == "user" %} | | | {{- '<|im start| ' + message.role + '\n' + content + '<|im end| ' + '\n' }} | | | {%- elif message.role == "assistant" %} | | | {%- set reasoning content = '' %} | | | {%- if message.reasoning content is string %} | | | {%- set reasoning content = message.reasoning content %} | | | {%- else %} | | | {%- if '</think ' in content %} | | | {%- set reasoning content = content.split '</think ' 0 .rstrip '\n' .split '<think ' -1 .lstrip '\n' %} | | | {%- set content = content.split '</think ' -1 .lstrip '\n' %} | | | {%- endif %} | | | {%- endif %} | | | {%- set reasoning content = reasoning content|trim %} | | | {%- if loop.index0 ns.last query index %} | | | {{- '<|im start| ' + message.role + '\n<think \n' + reasoning content + '\n</think \n\n' + content }} | | | {%- else %} | | | {{- '<|im start| ' + message.role + '\n' + content }} | | | {%- endif %} | | | {%- if message.tool calls and message.tool calls is iterable and message.tool calls is not mapping %} | | | {%- for tool call in message.tool calls %} | | | {%- if tool call.function is defined %} | | | {%- set tool call = tool call.function %} | | | {%- endif %} | | | {%- if loop.first %} | | | {%- if content|trim %} | | | {{- '\n\n<tool call \n<function=' + tool call.name + ' \n' }} | | | {%- else %} | | | {{- '<tool call \n<function=' + tool call.name + ' \n' }} | | | {%- endif %} | | | {%- else %} | | | {{- '\n<tool call \n<function=' + tool call.name + ' \n' }} | | | {%- endif %} | | | {%- if tool call.arguments is mapping %} | | | {%- for args name in tool call.arguments %} | | | {%- set args value = tool call.arguments args name %} | | | {{- '<parameter=' + args name + ' \n' }} | | | {%- set args value = args value | tojson | safe if args value is mapping or args value is sequence and args value is not string else args value | string %} | | | {{- args value }} | | | {{- '\n</parameter \n' }} | | | {%- endfor %} | | | {%- endif %} | | | {{- '</function \n</tool call ' }} | | | {%- endfor %} | | | {%- endif %} | | | {{- '<|im end| \n' }} | | | {%- elif message.role == "tool" %} | | | {%- if loop.previtem and loop.previtem.role = "tool" %} | | | {{- '<|im start| user' }} | | | {%- endif %} | | | {{- '\n<tool response \n' }} | | | {{- content }} | | | {{- '\n</tool response ' }} | | | {%- if not loop.last and loop.nextitem.role = "tool" %} | | | {{- '<|im end| \n' }} | | | {%- elif loop.last %} | | | {{- '<|im end| \n' }} | | | {%- endif %} | | | {%- else %} | | | {{- '<|im start| user\n' + content + '<|im end| \n' }} | | | {%- endif %} | | | {%- endfor %} | | | {%- if add generation prompt %} | | | {{- '<|im start| assistant\n' }} | | | {%- if enable thinking is defined and enable thinking is false %} | | | {{- '<think \n\n</think \n\n' }} | | | {%- else %} | | | {{- '<think \n' }} | | | {%- endif %} | | | {%- endif %} | Usage Drop this file next to your model and run: llama-server -m Qwen3.5-27B-Q4 K M.gguf -ngl 99 -c 262144 -np 1 -fa on --cache-type-k q4 0 --cache-type-v q4 0 --chat-template-file qwen3.5 chat template.jinja What this fixes: Qwen 3.5 GGUF templates reject the developer role sent by OpenCode, Claude Code, and other modern agent tools. The common workaround --chat-template chatml silently kills thinking mode thinking = 0 . This template handles all roles and preserves full thinking thinking = 1 . Note: Not needed for Qwopus Claude Opus distilled version which uses Hermes 2 Pro template natively. Thanks a lot for this, I'm using it on 9B Q4 K M with an RTX 5080 and the results are pretty good too Thanks a lot for this, I'm using it on 9B Q4 K M with an RTX 5080 and the results are pretty good too thanks for confirming what tok/s are you seeing on the 5080? curious how it compares to 3090 numbers. it fixes tool calling mostly, but it seems like there is some bug in either model or llamacpp that causes <tool call leak into <think block too. I'm still getting tool call errors "Failed to generate tool call" and "Api Error 500" but I'm using Claude Code with LM Studio 0.4.7 + Qwen3.5-27b. Thought this jinja template would fix but it's not. Any clues what the heck I should do? couldnt get it to work and switched from "LM Studio" to "llama-server" but why are you guys not using the flags "--temp 0.6 --top-p 0.95 --top-k20..." with Qwen3.5 in llama-server like recommended from unsloth? I thought they're mandatory for good reasoning? I always used them in LM Studio but Im new to llama-server. Did I miss something? Fyi theres a localllama thread regarding empty think blocks silently losing some of the kv cache reuse. r/LocalLLaMA/comments/1sg076h/i tracked a major cache reuse issue down to qwen https://gist.github.com/sudoingX/c2facf7d8f7608c65c1024ef3b22d431 file-qwen3-5 chat template-jinja-L100 https://gist.github.com/sudoingX/c2facf7d8f7608c65c1024ef3b22d431 file-qwen3-5 chat template-jinja-L100 1 Liner in this revision Initial runs look pretty good on hermes+Qwen3.5-35B-A3B-GPTQ-Int4 on 2x4060 https://gist.github.com/chuyqa/e44bda6b471792758849971c7cca519c/revisions https://gist.github.com/chuyqa/e44bda6b471792758849971c7cca519c/revisions .. Hermes is stopping less and my cache %s are looking higher.. APIServer pid=1 INFO 04-10 01:40:19 loggers.py:259 Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 85.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.8%, Prefix cache hit rate: 96.7%, MM cache hit rate: 33.3% APIServer pid=1 INFO 04-10 01:40:29 loggers.py:259 Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 85.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.8%, Prefix cache hit rate: 96.7%, MM cache hit rate: 33.3% APIServer pid=1 INFO: 192.168.101.54:51875 - "POST /v1/chat/completions HTTP/1.1" 200 OK APIServer pid=1 INFO 04-10 01:40:37 qwen3xml tool parser.py:1178 vLLM Successfully import tool parser Qwen3XMLToolParser APIServer pid=1 INFO 04-10 01:40:39 loggers.py:259 Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 66.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.2%, Prefix cache hit rate: 96.6%, MM cache hit rate: 33.3% APIServer pid=1 INFO 04-10 01:40:53 qwen3xml tool parser.py:1178 vLLM Successfully import tool parser Qwen3XMLToolParser APIServer pid=1 INFO 04-10 01:40:59 loggers.py:259 Engine 000: Avg prompt throughput: 5003.2 tokens/s, Avg generation throughput: 63.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.7%, Prefix cache hit rate: 96.6%, MM cache hit rate: 33.3% Thanks a lot for this, I'm using it on 9B Q4 K M with an RTX 5080 and the results are pretty good too Samesies Results are pretty good so far, this was an annoying bug for opencode & claude-code