RE Hunting For Malicious Chat Templates: How to Dump GGUF Metadata

A security researcher published a step-by-step method for dumping GGUF metadata with the Python `gguf` library to inspect a model's `tokenizer.chat_template` field for malicious code, using PrismML's Bonsai-27B-Q1_0.gguf as a known-good test case. The researcher warns that a model's template can be modified and re-uploaded to a repository, and that download counts cannot be trusted as a safety signal because they can be bot-inflated. The method installs `gguf` in a Python virtual environment, reads the `tokenizer.chat_template` field via `GGUFReader`, and saves the output as a `.jinja` file for review against a trusted template for the same model version.

I used AI to help me understand what and where to do. If you see anything I’ve written here that is not completely correct, say so and I’ll edit it as I can to reflect a useable method. I have used the following instructions to successfully dump the metadata of a GGUF file to inspect the Template. This process should be useful to inspect the Templates of any models you have downloaded. If you find anything you think may be suspicious, post in this thread so knowledgeable people can assist and verify. To inspect GGUF templates for malicious content I need to use a tool that can extract the GGUF Metadata. The GGUF metadata contains the template. It is possible to download a model, modify the template, then re-upload to a repo. You can’t rely on Downloaded metrics in determining of Downloads = safety, they can be Bot inflated. I’m using Linux, Parrot OS ‘Hack The Box’ Edition Not Omarchy , so I have lots of useful tools available should I need them. I don’t use Windows, so I can’t provide any steps/instructions for Windows users to achieve similar results. Anything python related should work on Windows, we have that for commonality. In this example, I am using a GGUF file from it’s creator PrismML, downloaded within LMStudio. It’s unlikely this GGUF is doing anything malicious, so it can be used as a test to see a ‘Known Good Example’. Step 1. Download a model file The file is located inside the dot file ‘.lmstudio’ in LMStudio I have Bonsai-27B-GGUF directory with 2 .gguf files Step 2. Verify the files location: bash $ cd ~/.lmstudio/models/Bonsai-27B-GGUF/ $ ls Bonsai-27B-Q1 0.gguf mmproj-Bonsai-27B-BF16.gguf To look for Malicious Template code I’ll be dumping the metadata for this file: Bonsai-27B-Q1 0.gguf Step 3. Create the virtual environment This step may not be required for some python users, running ‘python -m pip install gguf’ inside your shell should work. Using Python, Python3 for me, I need to set a Virtual Environment first, I have ‘system-managed Python packages’, then install the GGUF reader: python3 -m venv ~/gguf-inspect-venv source ~/gguf-inspect-venv/bin/activate python -m pip install gguf Run the inspection snippet from that same terminal. When you’re done, type ‘deactivate’ to leave the virtual environment. Avoid ‘–break-system-packages’; it can interfere with system-managed Python packages. Step 4. Inspection: Dump the GGUF Metadata python $ python - <<'PY'from gguf import GGUFReader path = "/full/path/to/your/model-file.gguf"reader = GGUFReader path, "r" field = reader.fields.get "tokenizer.chat template" if field is None: print "No tokenizer.chat template field found" else: print field.contents PY Note: Replace the ‘/full/path/to/your/model-file.gguf’ with your files absolute path. I now have a chat template dump. I Copy the text and save in my favorite editor Not Neovim , like ‘template.jinja’ for code highlighting. What to look for: Most template constructs—set, if, loops, and macros—are normal. What matters is whether they add unexpected behavior to the prompt or message stream. For each suspicious branch, trace its inputs and outputs: what triggers it, what text or structured record it emits, and where that appears in the rendered conversation. Compare with a trusted template for the exact model/version, and test rendering with harmless inputs. Here is the output I received in my dump: Bonsai-27B-Q1 0.gguf" reader = GGUFReader path, “r” field = reader.fields.get “tokenizer.chat template” if field is None: print “No tokenizer.chat template field found” else: print field.contents PY {%- set image count = namespace value=0 %} {%- set video count = namespace value=0 %} {%- macro render content content, do vision count, is system content=false %} {%- if content is string %} {{- content }} {%- elif content is iterable and content is not mapping %} {%- for item in content %} {%- if ‘image’ in item or ‘image url’ in item or item.type == ‘image’ %} {%- if is system content %} {{- raise exception ‘System message cannot contain images.’ }} {%- endif %} {%- if do vision count %} {%- set image count.value = image count.value + 1 %} {%- endif %} {%- if add vision id %} {{- ‘Picture ’ ~ image count.value ~ ‘: ’ }} {%- endif %} {{- ‘<|vision start| <|image pad| <|vision end| ’ }} {%- elif ‘video’ in item or item.type == ‘video’ %} {%- if is system content %} {{- raise exception ‘System message cannot contain videos.’ }} {%- endif %} {%- if do vision count %} {%- set video count.value = video count.value + 1 %} {%- endif %} {%- if add vision id %} {{- ‘Video ’ ~ video count.value ~ ‘: ’ }} {%- endif %} {{- ‘<|vision start| <|video pad| <|vision end| ’ }} {%- elif ‘text’ in item %} {{- item.text }} {%- else %} {{- raise exception ‘Unexpected item type in content.’ }} {%- endif %} {%- endfor %} {%- elif content is none or content is undefined %} {{- ‘’ }} {%- else %} {{- raise exception ‘Unexpected content type.’ }} {%- endif %} {%- endmacro %} {%- if not messages %} {{- raise exception ‘No messages provided.’ }} {%- endif %} {%- if tools and tools is iterable and tools is not mapping %} {{- ‘<|im start| system\n’ }} {{- “ Tools\n\nYou have access to the following functions:\n\n” }} {%- for tool in tools %} {{- “\n” }} {{- tool | tojson }} {%- endfor %} {{- “\n” }} {{- ‘\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool call \n<function=example function name \n<parameter=example parameter 1 \nvalue 1\n\n<parameter=example parameter 2 \nThis is the value for the second parameter\nthat can span\nmultiple lines\n\n\n</tool call \n\n\nReminder:\n- Function calls MUST follow the specified format: an inner <function=… block must be nested within <tool call </tool call XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n’ }} {%- if messages 0 .role == ‘system’ %} {%- set content = render content messages 0 .content, false, true |trim %} {%- if content %} {{- ‘\n\n’ + content }} {%- endif %} {%- endif %} {{- ‘<|im end| \n’ }} {%- else %} {%- if messages 0 .role == ‘system’ %} {%- set content = render content messages 0 .content, false, true |trim %} {{- ‘<|im start| system\n’ + content + ‘<|im end| \n’ }} {%- endif %} {%- endif %} {%- set ns = namespace multi step tool=true, last query index=messages|length - 1 %} {%- for message in messages ::-1 %} {%- set index = messages|length - 1 - loop.index0 %} {%- if ns.multi step tool and message.role == “user” %} {%- set content = render content message.content, false |trim %} {%- if not content.startswith ’<tool response ’ and content.endswith ’</tool response ’ %} {%- set ns.multi step tool = false %} {%- set ns.last query index = index %} {%- endif %} {%- endif %} {%- endfor %} {%- if ns.multi step tool %} {{- raise exception ‘No user query found in messages.’ }} {%- endif %} {%- for message in messages %} {%- set content = render content message.content, true |trim %} {%- if message.role == “system” %} {%- if not loop.first %} {{- raise exception ‘System message must be at the beginning.’ }} {%- endif %} {%- elif message.role == “user” %} {{- ‘<|im start| ’ + message.role + ‘\n’ + content + ‘<|im end| ’ + ‘\n’ }} {%- elif message.role == “assistant” %} {%- set reasoning content = ‘’ %} {%- if message.reasoning content is string %} {%- set reasoning content = message.reasoning content %} {%- else %} {%- if ‘’ in content %} {%- set reasoning content = content.split ‘’ 0 .rstrip ‘\n’ .split ‘’ -1 .lstrip ‘\n’ %} {%- set content = content.split ‘’ -1 .lstrip ‘\n’ %} {%- endif %} {%- endif %} {%- set reasoning content = reasoning content|trim %} {%- if preserve thinking is defined and preserve thinking is true or loop.index0 ns.last query index %} {{- ‘<|im start| ’ + message.role + ‘\n\n’ + reasoning content + ‘\n\n\n’ + content }} {%- else %} {{- ‘<|im start| ’ + message.role + ‘\n’ + content }} {%- endif %} {%- if message.tool calls and message.tool calls is iterable and message.tool calls is not mapping %} {%- for tool call in message.tool calls %} {%- if tool call.function is defined %} {%- set tool call = tool call.function %} {%- endif %} {%- if loop.first %} {%- if content|trim %} {{- ‘\n\n<tool call \n<function=’ + tool call.name + ‘ \n’ }} {%- else %} {{- ‘<tool call \n<function=’ + tool call.name + ‘ \n’ }} {%- endif %} {%- else %} {{- ‘\n<tool call \n<function=’ + tool call.name + ‘ \n’ }} {%- endif %} {%- if tool call.arguments is defined %} {%- for args name, args value in tool call.arguments|items %} {{- ‘<parameter=’ + args name + ‘ \n’ }} {%- set args value = args value | string if args value is string else args value | tojson | safe %} {{- args value }} {{- ‘\n\n’ }} {%- endfor %} {%- endif %} {{- ‘\n</tool call ’ }} {%- endfor %} {%- endif %} {{- ‘<|im end| \n’ }} {%- elif message.role == “tool” %} {%- if loop.previtem and loop.previtem.role = “tool” %} {{- ‘<|im start| user’ }} {%- endif %} {{- ‘\n<tool response \n’ }} {{- content }} {{- ‘\n</tool response ’ }} {%- if not loop.last and loop.nextitem.role = “tool” %} {{- ‘<|im end| \n’ }} {%- elif loop.last %} {{- ‘<|im end| \n’ }} {%- endif %} {%- else %} {{- raise exception ‘Unexpected message role.’ }} {%- endif %} {%- endfor %} {%- if add generation prompt %} {{- ‘<|im start| assistant\n’ }} {%- if enable thinking is defined and enable thinking is false %} {{- ‘\n\n\n\n’ }} {%- else %} {{- ‘\n’ }} {%- endif %} {%- endif %} A known Bad Version is available here: github com /FujitsuResearch/chat-template-backdoor-attack/blob/main/resources/templates/scenario b/backdoor/command a.jinja Comparing a known good and a known bad will help me understand where potential modifications will be.