Four reproducible parser failures in vLLM 0.26.0, 0.27.1 and 0.28.0, run on CPU with no GPU or weights. As of 2026-08-27 one is fixed on main but not in any release; the rest are open. See Upstream status.
Published 2026-08-25. Ingot team.
We did not run a live server for this. We pip-installed the upstream vllm
package at 0.26.0 and 0.27.1 (and, on 2026-08-27, 0.28.0), imported the tool-call and reasoning parsers on a CPU box (no GPU, no model weights), and fed them the reproducer strings from open vLLM issues. Where an issue named a community chat template, we rendered the real template too. Every issue we cite was still open when we pulled it on 2026-08-25. Scripts and raw transcripts are in the ingot-repros repository. This is not a safety certification of any model or framework.
What we found: four cases where the model's raw text is fine and the parser hands back HTTP 200 with a null
field, an empty array, or a corrupted string. Two are in the Gemma4 tool parser (a valid call parses to an empty list; a parenthesized call becomes a garbage function name that also swallows the next call). Two are in reasoning parsers (Qwen3 routes a whole plain-text answer into reasoning_content
; MiniMax M3 decides reasoning has ended after reading an example in the prompt). There is a fifth, Kimi K3, that we could only reproduce on the streaming lane.
Two things surprised us. The Kimi K3 non-streaming path appears repaired on 0.27.1 while the streaming path in the same release still loses the turn, so "what version are you on" is not enough of a question. And MiniMax M3's own stock template, which tells the model to wrap reasoning in <mm:think></mm:think>
, is exactly the kind of prompt that trips MiniMax's own is_reasoning_end
.
What we did not do: run a server, measure production rates, or test 0.25/0.26 non-streaming Kimi behavior. The rates quoted below are the issue reporters' numbers.
Earlier reports from us were about the model artifact (weights, template, stop tokens). This one is about the layer above it. The agent never sees the model's raw text with <tool_call>
or <think>
markers in it; it sees message.tool_calls
, message.reasoning_content
, and message.content
after a parser has converted them. The parsers are small state machines. When the grammar does not cover an input the model legitimately produced (a different but valid opener, a parenthesis, a template that closed the think block), the machine does not throw. It returns a well-formed wrong result with a 200 status. Both the tool-call layer and the reasoning layer do this.
Tool-call parsing (Gemma4, Kimi K3) #
We ran vLLM 0.27.1's Gemma4
tool parser directly against the reproducer inputs from three open issues.
Issue #53431. A bare opener of the form <|tool_call>:name{...}
is a documented, valid form. The state machine has no transition for the bare :
out of TOOL_PREAMBLE
, so the span is never emitted anywhere:
tools_called=False tool_calls=[] content=None
The model called a tool. The agent gets no tool call, no content, and no error. The reporter counted 386 lost turns over 21 days in production (~0.4%); with greedy decoding it reproduces every time.
Issue #53642. A parenthesized call, call:name(...)
, reaches TOOL_NAME
, which has exactly one outgoing transition (for {
). On (
the machine stays in TOOL_NAME
and appends everything after it into the function name:
name = terminal(command:<|"|>ls -a<|"|>)<tool_call|> args = {}
It gets worse if a correct call follows the broken one. Both collapse into a single call: the garbage name runs on to include the second call's call:terminal
, and the arguments carry only the second call's payload ({"command": "pwd"}
). One malformed call eats the valid one behind it.
Issue #53246 (Kimi K3). When the model omits an internal think-transition marker, the streaming reasoning path classifies a complete tools block as reasoning. On our pinned 0.27.1 the non-streaming path forwards the text downstream, so it appears fixed there; the streaming path still drops the turn. Same release, different lane, different answer.
When we fetched the gemma4
parser source on 2026-08-25, neither Gemma4 issue was fixed on main: TOOL_PREAMBLE
had no transition for a bare :
, TOOL_NAME
had one outgoing transition, and nothing handled a parenthesis. Later the same day, vLLM merged #53657, which adds OPEN_PAREN
and a TOOL_END
escape to TOOL_NAME
, so the parenthesized-call case (rows 3 and 4) is fixed on main. That commit is not in v0.28.0 (tagged 2026-08-26). The bare :
opener (row 2) is still unhandled on main; the fix PR #53444 is open.
Reasoning parsing (Qwen3, MiniMax M3) #
The reasoning parser splits a model's output into reasoning_content
and content
. It can get the split wrong in either direction. We confirmed one case of each against installed 0.26.0 and 0.27.1, with identical results on both versions.
Qwen3, issue #53284: the answer becomes reasoning. Qwen3Parser
reads only the enable_thinking
chat-template kwarg (default True
) and never looks at the rendered prompt to see what state thinking is actually in. From the installed source: its adjust_initial_state_from_prompt
hook is an inherited no-op; parse_delta
accepts prompt_token_ids
and ignores it (we get identical output with closed-think IDs and with empty IDs); the non-streaming path has no prompt parameter at all. The consumer is live in the serving layer (serving.py:339
).
So if a chat template disables thinking by rendering a closed think block (via reasoning_effort=none
, auto_disable_thinking_with_tools
, or a custom community template) and the request does not also set enable_thinking=false
, a compliant model answers in plain text and the parser files the entire answer under reasoning_content
, leaving content: null
. An agent reading message.content
gets nothing and retries or makes something up.
We checked this with the real froggeric/Qwen-Fixed-Chat-Templates
template (SHA-256 pinned), rendered with the documented triggers. It renders <think>\n\n</think>\n\n
, and the real Qwen3Parser
puts the plain answer into reasoning with content=None
. The top-level OpenAI reasoning_effort
parameter is fine; vLLM maps it correctly. The hazard is specifically in chat_template_kwargs
and custom templates, which is where teams running community templates live.
MiniMax M3, issue #46042: reasoning "ended" before it started. MiniMaxM3ReasoningParser.is_reasoning_end
does a naive backward scan (last close marker after last start marker means ended), while count_reasoning_tokens
in the same class does proper depth counting. The two disagree. If the rendered prompt contains a balanced think-tag example, which MiniMax M3's own stock template does when it instructs the model to wrap reasoning in <mm:think></mm:think>
, the backward scan takes the example as proof that reasoning already finished and can engage output grammar at the wrong time. The is_reasoning_end
code is byte-identical on 0.26.0, 0.27.1, and current main as of 2026-08-27. A separate change merged that day, #54089, scopes reasoning-end detection to the current turn for the engine-based parsers (Qwen3, Nemotron); it does not touch MiniMaxM3ReasoningParser
.
Why a repo scan does not find this #
Everything Ingot has reported until now was a property of a file you can download: a weight, a template, a config key. This is a property of the running server: the parser version, the streaming or non-streaming lane, and whether the parser's assumed state matches what the chat template actually rendered. The model repository is innocent. A template diff catches a changed file; here there is no changed file, only a vLLM version whose parser grammar does not cover an input the model is allowed to emit.
What it costs, in practice: a vanished tool call means the agent's step does not happen and nothing triggers a retry. A garbage name that absorbs the next call turns two intended operations into one malformed one with arguments from the wrong call. An answer filed as reasoning leaves content: null
, and an agent reading content loops or fabricates. And all of it returns 200, so error rates, status codes, and latency look healthy. The reporter of #53431 needed 21 days and a manual audit to find a 0.4% turn-loss rate.
How to catch it #
The check is a version- and lane-aware release gate run against the exact vLLM build and config a deployment uses. It is how we produced this report, and it takes seconds.
- Import the installed parsers and run them against a battery of known-hazard inputs: the documented opener forms, parenthesized calls, a valid call following a malformed one, closed-think and open-think prompts, prompts containing balanced think-tag examples. Assert the structured output matches what the model actually said. No GPU or weights; CPU import is enough.
- Test both lanes. Streaming and non-streaming diverge (Kimi K3 and Qwen3 both differ by lane). Testing one lane misses half the surface.
- Record the vLLM version the gate ran against and re-run on every upgrade. Within one release a defect can be fixed on one lane and live on the other, and a bump can fix one row while regressing another.
- Render the real templates the deployment serves, not only synthetic strings. The Qwen3 case only shows up when a specific community template renders a closed think block.
Upstream status #
Checked 2026-08-27. Versions move; the harness in the repros repo is the thing to trust.
| finding | issue | status |
|---|---|---|
Gemma4 bare : opener drops the turn |
||
#53444open#53642#53657(2026-08-25), not in v0.28.0#53246#53284#53302openis_reasoning_end
backward scan#46042is_reasoning_end
unchanged on mainOn 2026-08-27 we re-ran all three harnesses against v0.28.0 from CI in the repros repo (run): every row is identical to 0.27.1. Four tool-parser bugs, five reasoning-parser rows, three real-template rows, all still reproduce. The job runs on demand and weekly.
Limitations #
- Reproductions are at the parser-code level. We import and run the real vLLM package against reproducer strings. We did not run a live server or measure production emission rates; the reporters' numbers (0.4% turn loss over 21 days; 0 to 53% of parenthesized calls per boot on some quantized checkpoints) are theirs, not ours.
- The Kimi K3 finding is version- and lane-scoped: the streaming lane reproduces on 0.27.1; the non-streaming lane appears repaired there; 0.25/0.26 non-streaming behavior was not retested, and the module is absent in 0.26.0.
- We reproduced the mechanism and the misrouted output, not the downstream business impact. How often a real agent loop hits these inputs depends on the model, the template, and the traffic.
- The upstream issues were open at retrieval on 2026-08-25 (#53431, #53642, #53246, #53284, #46042). A proposed fix for #53284 (PR #53302) was open and unmerged. Versions move; the gate approach generalizes, the specific version rows do not.
- Named parsers and issues are cited for a correctness defect in a specific version, not as a judgment of vLLM overall. vLLM is the framework here because its parser code is public and we could reproduce against it.
Sources #
Probe scripts that import and execute the real vLLM parsers, the version sweep, and raw transcripts are in tool-parser-fsm/ (tool-call parsing) and
(reasoning parsing, both directions, with pinned real-template renders). Upstream issues: vLLM
reasoning-parser-mismatch/
#46042. These parsers are the serving-layer companion to the artifact-level failures in
REPORT-STOP-TOKEN.md
REPORT-TEMPLATE-DRIFT.md
Check the exact model you plan to ship #
The static scan behind this report runs on any public Hugging Face model. If your checkpoint is private, gated, or not released yet, tell us and we'll run it privately.