{"slug": "four-reproducible-vllm-parser-failures-that-return-200-with-the-wrong-tool-call", "title": "Four reproducible vLLM parser failures that return 200 with the wrong tool call", "summary": "Four reproducible parser failures in vLLM 0.26.0, 0.27.1, and 0.28.0 can drop or garble tool calls and reasoning content while still returning HTTP 200, according to the Ingot team's tests on CPU without GPU or weights. Two failures affect the Gemma4 tool parser (a valid call parses to an empty list; a parenthesized call becomes a garbage function name that swallows the next call), and two affect reasoning parsers (Qwen3 routes a plain-text answer into reasoning_content; MiniMax M3 ends reasoning prematurely after reading an example). As of 2026-08-27, one issue is fixed on main but not in any release; the rest remain open.", "body_md": "# vLLM can drop or garble a tool call and still return 200\n\n*Four reproducible parser failures in vLLM 0.26.0, 0.27.1 and 0.28.0, run on CPU with no GPU or weights. As of 2026-08-27 one is fixed on main but not in any release; the rest are open. See Upstream status.*\n\nPublished 2026-08-25. Ingot team.\n\nWe did not run a live server for this. We pip-installed the upstream `vllm`\n\npackage at 0.26.0 and 0.27.1 (and, on 2026-08-27, 0.28.0), imported the tool-call and reasoning parsers on a CPU box (no GPU, no model weights), and fed them the reproducer strings from open vLLM issues. Where an issue named a community chat template, we rendered the real template too. Every issue we cite was still open when we pulled it on 2026-08-25. Scripts and raw transcripts are in the [ingot-repros repository](https://github.com/Ember-Sovereignty/ingot-repros/tree/main/vllm-parser-failures/tool-parser-fsm). This is not a safety certification of any model or framework.\n\nWhat we found: four cases where the model's raw text is fine and the parser hands back HTTP 200 with a `null`\n\nfield, an empty array, or a corrupted string. Two are in the Gemma4 tool parser (a valid call parses to an empty list; a parenthesized call becomes a garbage function name that also swallows the next call). Two are in reasoning parsers (Qwen3 routes a whole plain-text answer into `reasoning_content`\n\n; MiniMax M3 decides reasoning has ended after reading an example in the prompt). There is a fifth, Kimi K3, that we could only reproduce on the streaming lane.\n\nTwo things surprised us. The Kimi K3 non-streaming path appears repaired on 0.27.1 while the streaming path in the same release still loses the turn, so \"what version are you on\" is not enough of a question. And MiniMax M3's own stock template, which tells the model to wrap reasoning in `<mm:think></mm:think>`\n\n, is exactly the kind of prompt that trips MiniMax's own `is_reasoning_end`\n\n.\n\nWhat we did not do: run a server, measure production rates, or test 0.25/0.26 non-streaming Kimi behavior. The rates quoted below are the issue reporters' numbers.\n\nEarlier reports from us were about the model artifact (weights, template, stop tokens). This one is about the layer above it. The agent never sees the model's raw text with `<tool_call>`\n\nor `<think>`\n\nmarkers in it; it sees `message.tool_calls`\n\n, `message.reasoning_content`\n\n, and `message.content`\n\nafter a parser has converted them. The parsers are small state machines. When the grammar does not cover an input the model legitimately produced (a different but valid opener, a parenthesis, a template that closed the think block), the machine does not throw. It returns a well-formed wrong result with a 200 status. Both the tool-call layer and the reasoning layer do this.\n\n## Tool-call parsing (Gemma4, Kimi K3)\n\nWe ran vLLM 0.27.1's `Gemma4`\n\ntool parser directly against the reproducer inputs from three open issues.\n\nIssue #53431. A bare opener of the form `<|tool_call>:name{...}`\n\nis a documented, valid form. The state machine has no transition for the bare `:`\n\nout of `TOOL_PREAMBLE`\n\n, so the span is never emitted anywhere:\n\n```\ntools_called=False   tool_calls=[]   content=None\n```\n\nThe model called a tool. The agent gets no tool call, no content, and no error. The reporter counted 386 lost turns over 21 days in production (~0.4%); with greedy decoding it reproduces every time.\n\nIssue #53642. A parenthesized call, `call:name(...)`\n\n, reaches `TOOL_NAME`\n\n, which has exactly one outgoing transition (for `{`\n\n). On `(`\n\nthe machine stays in `TOOL_NAME`\n\nand appends everything after it into the function name:\n\n```\nname = terminal(command:<|\"|>ls -a<|\"|>)<tool_call|>   args = {}\n```\n\nIt gets worse if a correct call follows the broken one. Both collapse into a single call: the garbage name runs on to include the second call's `call:terminal`\n\n, and the arguments carry only the second call's payload (`{\"command\": \"pwd\"}`\n\n). One malformed call eats the valid one behind it.\n\nIssue #53246 (Kimi K3). When the model omits an internal think-transition marker, the streaming reasoning path classifies a complete tools block as reasoning. On our pinned 0.27.1 the non-streaming path forwards the text downstream, so it appears fixed there; the streaming path still drops the turn. Same release, different lane, different answer.\n\nWhen we fetched the `gemma4`\n\nparser source on 2026-08-25, neither Gemma4 issue was fixed on main: `TOOL_PREAMBLE`\n\nhad no transition for a bare `:`\n\n, `TOOL_NAME`\n\nhad one outgoing transition, and nothing handled a parenthesis. Later the same day, vLLM merged [#53657](https://github.com/vllm-project/vllm/pull/53657), which adds `OPEN_PAREN`\n\nand a `TOOL_END`\n\nescape to `TOOL_NAME`\n\n, so the parenthesized-call case (rows 3 and 4) is fixed on main. That commit is not in v0.28.0 (tagged 2026-08-26). The bare `:`\n\nopener (row 2) is still unhandled on main; the fix PR [#53444](https://github.com/vllm-project/vllm/pull/53444) is open.\n\n## Reasoning parsing (Qwen3, MiniMax M3)\n\nThe reasoning parser splits a model's output into `reasoning_content`\n\nand `content`\n\n. It can get the split wrong in either direction. We confirmed one case of each against installed 0.26.0 and 0.27.1, with identical results on both versions.\n\nQwen3, issue #53284: the answer becomes reasoning. `Qwen3Parser`\n\nreads only the `enable_thinking`\n\nchat-template kwarg (default `True`\n\n) and never looks at the rendered prompt to see what state thinking is actually in. From the installed source: its `adjust_initial_state_from_prompt`\n\nhook is an inherited no-op; `parse_delta`\n\naccepts `prompt_token_ids`\n\nand ignores it (we get identical output with closed-think IDs and with empty IDs); the non-streaming path has no prompt parameter at all. The consumer is live in the serving layer (`serving.py:339`\n\n).\n\nSo if a chat template disables thinking by rendering a closed think block (via `reasoning_effort=none`\n\n, `auto_disable_thinking_with_tools`\n\n, or a custom community template) and the request does not also set `enable_thinking=false`\n\n, a compliant model answers in plain text and the parser files the entire answer under `reasoning_content`\n\n, leaving `content: null`\n\n. An agent reading `message.content`\n\ngets nothing and retries or makes something up.\n\nWe checked this with the real `froggeric/Qwen-Fixed-Chat-Templates`\n\ntemplate (SHA-256 pinned), rendered with the documented triggers. It renders `<think>\\n\\n</think>\\n\\n`\n\n, and the real `Qwen3Parser`\n\nputs the plain answer into reasoning with `content=None`\n\n. The top-level OpenAI `reasoning_effort`\n\nparameter is fine; vLLM maps it correctly. The hazard is specifically in `chat_template_kwargs`\n\nand custom templates, which is where teams running community templates live.\n\nMiniMax M3, issue #46042: reasoning \"ended\" before it started. `MiniMaxM3ReasoningParser.is_reasoning_end`\n\ndoes a naive backward scan (last close marker after last start marker means ended), while `count_reasoning_tokens`\n\nin the same class does proper depth counting. The two disagree. If the rendered prompt contains a balanced think-tag example, which MiniMax M3's own stock template does when it instructs the model to wrap reasoning in `<mm:think></mm:think>`\n\n, the backward scan takes the example as proof that reasoning already finished and can engage output grammar at the wrong time. The `is_reasoning_end`\n\ncode is byte-identical on 0.26.0, 0.27.1, and current main as of 2026-08-27. A separate change merged that day, [#54089](https://github.com/vllm-project/vllm/pull/54089), scopes reasoning-end detection to the current turn for the engine-based parsers (Qwen3, Nemotron); it does not touch `MiniMaxM3ReasoningParser`\n\n.\n\n## Why a repo scan does not find this\n\nEverything Ingot has reported until now was a property of a file you can download: a weight, a template, a config key. This is a property of the running server: the parser version, the streaming or non-streaming lane, and whether the parser's assumed state matches what the chat template actually rendered. The model repository is innocent. A template diff catches a changed file; here there is no changed file, only a vLLM version whose parser grammar does not cover an input the model is allowed to emit.\n\nWhat it costs, in practice: a vanished tool call means the agent's step does not happen and nothing triggers a retry. A garbage name that absorbs the next call turns two intended operations into one malformed one with arguments from the wrong call. An answer filed as reasoning leaves `content: null`\n\n, and an agent reading content loops or fabricates. And all of it returns 200, so error rates, status codes, and latency look healthy. The reporter of #53431 needed 21 days and a manual audit to find a 0.4% turn-loss rate.\n\n## How to catch it\n\nThe check is a version- and lane-aware release gate run against the exact vLLM build and config a deployment uses. It is how we produced this report, and it takes seconds.\n\n- Import the installed parsers and run them against a battery of known-hazard inputs: the documented opener forms, parenthesized calls, a valid call following a malformed one, closed-think and open-think prompts, prompts containing balanced think-tag examples. Assert the structured output matches what the model actually said. No GPU or weights; CPU import is enough.\n- Test both lanes. Streaming and non-streaming diverge (Kimi K3 and Qwen3 both differ by lane). Testing one lane misses half the surface.\n- Record the vLLM version the gate ran against and re-run on every upgrade. Within one release a defect can be fixed on one lane and live on the other, and a bump can fix one row while regressing another.\n- Render the real templates the deployment serves, not only synthetic strings. The Qwen3 case only shows up when a specific community template renders a closed think block.\n\n## Upstream status\n\nChecked 2026-08-27. Versions move; the harness in the repros repo is the thing to trust.\n\n| finding | issue | status |\n|---|---|---|\nGemma4 bare `:` opener drops the turn |\n|\n\n[#53444](https://github.com/vllm-project/vllm/pull/53444)open[#53642](https://github.com/vllm-project/vllm/issues/53642)[#53657](https://github.com/vllm-project/vllm/pull/53657)(2026-08-25), not in v0.28.0[#53246](https://github.com/vllm-project/vllm/issues/53246)[#53284](https://github.com/vllm-project/vllm/issues/53284)[#53302](https://github.com/vllm-project/vllm/pull/53302)open`is_reasoning_end`\n\nbackward scan[#46042](https://github.com/vllm-project/vllm/issues/46042)`is_reasoning_end`\n\nunchanged on mainOn 2026-08-27 we re-ran all three harnesses against v0.28.0 from CI in the repros repo ([run](https://github.com/Ember-Sovereignty/ingot-repros/actions/runs/33118939101)): every row is identical to 0.27.1. Four tool-parser bugs, five reasoning-parser rows, three real-template rows, all still reproduce. The job runs on demand and weekly.\n\n## Limitations\n\n- Reproductions are at the parser-code level. We import and run the real vLLM package against reproducer strings. We did not run a live server or measure production emission rates; the reporters' numbers (0.4% turn loss over 21 days; 0 to 53% of parenthesized calls per boot on some quantized checkpoints) are theirs, not ours.\n- The Kimi K3 finding is version- and lane-scoped: the streaming lane reproduces on 0.27.1; the non-streaming lane appears repaired there; 0.25/0.26 non-streaming behavior was not retested, and the module is absent in 0.26.0.\n- We reproduced the mechanism and the misrouted output, not the downstream business impact. How often a real agent loop hits these inputs depends on the model, the template, and the traffic.\n- The upstream issues were open at retrieval on 2026-08-25 (#53431, #53642, #53246, #53284, #46042). A proposed fix for #53284 (PR #53302) was open and unmerged. Versions move; the gate approach generalizes, the specific version rows do not.\n- Named parsers and issues are cited for a correctness defect in a specific version, not as a judgment of vLLM overall. vLLM is the framework here because its parser code is public and we could reproduce against it.\n\n## Sources\n\nProbe scripts that import and execute the real vLLM parsers, the version sweep, and raw transcripts are in [ tool-parser-fsm/](https://github.com/Ember-Sovereignty/ingot-repros/tree/main/vllm-parser-failures/tool-parser-fsm) (tool-call parsing) and\n\n[(reasoning parsing, both directions, with pinned real-template renders). Upstream issues: vLLM](https://github.com/Ember-Sovereignty/ingot-repros/tree/main/vllm-parser-failures/reasoning-parser-mismatch)\n\n`reasoning-parser-mismatch/`\n\n[#53431](https://github.com/vllm-project/vllm/issues/53431),\n\n[#53642](https://github.com/vllm-project/vllm/issues/53642),\n\n[#53246](https://github.com/vllm-project/vllm/issues/53246),\n\n[#53284](https://github.com/vllm-project/vllm/issues/53284),\n\n[#46042](https://github.com/vllm-project/vllm/issues/46042). These parsers are the serving-layer companion to the artifact-level failures in\n\n[and](/reports/stop-token-runaway)\n\n`REPORT-STOP-TOKEN.md`\n\n[.](/reports/chat-template-drift)\n\n`REPORT-TEMPLATE-DRIFT.md`\n\n## Check the exact model you plan to ship\n\nThe static scan behind this report runs on any public Hugging Face model. If your checkpoint is private, gated, or not released yet, tell us and we'll run it privately.", "url": "https://wpnews.pro/news/four-reproducible-vllm-parser-failures-that-return-200-with-the-wrong-tool-call", "canonical_source": "https://ingot.tools/reports/parser-silent-failure", "published_at": "2026-08-27 21:41:59+00:00", "updated_at": "2026-08-27 22:18:36.985297+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "ai-safety"], "entities": ["vLLM", "Ingot team", "Gemma4", "Qwen3", "MiniMax M3", "Kimi K3"], "alternates": {"html": "https://wpnews.pro/news/four-reproducible-vllm-parser-failures-that-return-200-with-the-wrong-tool-call", "markdown": "https://wpnews.pro/news/four-reproducible-vllm-parser-failures-that-return-200-with-the-wrong-tool-call.md", "text": "https://wpnews.pro/news/four-reproducible-vllm-parser-failures-that-return-200-with-the-wrong-tool-call.txt", "jsonld": "https://wpnews.pro/news/four-reproducible-vllm-parser-failures-that-return-200-with-the-wrong-tool-call.jsonld"}}