{"slug": "llama-server-s-responses-api-marks-truncated-replies-and-tool-calls-as-completed", "title": "llama-server's Responses API marks truncated replies and tool calls as completed", "summary": "A developer testing llama.cpp build b11753 found that llama-server's OpenAI-compatible /v1/responses endpoint reports truncated replies and tool calls as \"completed\" with incomplete_details set to null, while the same requests on /v1/chat/completions correctly return finish_reason: \"length\". In tests with gemma-3-1b-it and Qwen3-0.6B, an 8-token limit returned the text \"1 2 3 4\" marked completed, and a cut-off write_file tool call returned invalid JSON arguments ({\"path\": \"/) also marked completed, with streaming ending in response.completed and no function_call_arguments.done event. The issue is filed upstream as ggml-org/llama.cpp#30773.", "body_md": "**TL;DR**: llama-server's OpenAI-compatible `/v1/responses` endpoint never reports a reply as cut off. On llama.cpp b11753, a request with `max_output_tokens: 8` came back with `\"status\": \"completed\"`, `incomplete_details: null` and the text `1 2 3 4`, stopped mid-count. A tool call cut off by the limit came back as a `function_call` item with arguments `{\"path\": \"/`, which is not valid JSON, and both the item and the response said `completed`. The streaming version ended with `response.completed` and never sent `function_call_arguments.done`. The same requests on `/v1/chat/completions` said `finish_reason: \"length\"`. A client that trusts `status`, which is how the Responses API is meant to be used, will treat half an answer as a whole one, or run a tool with broken arguments. The one signal left is `usage.output_tokens` equal to the limit you sent. Upstream: [ggml-org/llama.cpp#30773](https://github.com/ggml-org/llama.cpp/issues/30773).\n\nA disposable LXC container, the b11753 release binary, `gemma-3-1b-it-Q4_K_M` and `Qwen3-0.6B-Q8_0`, `--jinja -c 4096`. A request that can't finish within its limit:\n\n```\nPOST /v1/responses\n{\"model\": \"x\", \"input\": \"Count from 1 to 200, separated by spaces.\", \"max_output_tokens\": 8}\n\n{\"status\": \"completed\", \"incomplete_details\": null, \"output_tokens\": 8, \"text\": [\"1 2 3 4 \"]}\n```\n\nThe same question with room to finish used 52 tokens and also said `completed`, so `status` is the same whether the reply ended or was cut. On the chat endpoint:\n\n```\nPOST /v1/chat/completions   ... \"max_tokens\": 8\nfinish_reason= length completion_tokens= 8\n```\n\nWith streaming, the last events of the 8-token request were:\n\n```\nresponse.content_part.done, response.output_item.done, response.completed status=completed incomplete_details=None\n```\n\nThere is no `response.incomplete` event.\n\nA tool call is where it matters most. One tool, `write_file(path, content)`, Qwen3-0.6B, `temperature: 0`, and `/no_think` in the prompt so the model calls the tool straight away:\n\n```\nresponses limit=20:  status=completed usage.output_tokens=20 args='{\"path\": \"/'\nchat      limit=20:  finish_reason=length completion_tokens=20 tool_calls=[('write_file', '{\"path\": \"/')]\nresponses limit=400: status=completed usage.output_tokens=89 args='{\"path\": \"/tmp/poem.txt\", \"content\": \"Th...'\nchat      limit=400: finish_reason=tool_calls completion_tokens=89\n```\n\nAt a 20-token limit the Responses API returned a `function_call` item marked `completed` whose arguments end after `{\"path\": \"/`. At 30 they ended after `\"content\": \"`. Both endpoints produced the same truncated arguments; only the chat endpoint said so. In the stream there were five `response.function_call_arguments.delta` events, then `response.output_item.done` and `response.completed`. No `response.function_call_arguments.done`, and that event was also missing from a call that completed normally.\n\nShort limits are rare in interactive use, so most replies really are complete, and `completed` is the right answer most of the time. The ones that aren't are the long ones: a big file write, a long code edit, a model that thinks for a while first. Those are exactly the replies where an agent is about to do something with the output.\n\nThe model also isn't wrong. The text is what the model generated, and in the tool-call case the arguments are a correct prefix of a correct call. Nothing errors until the client tries to parse the arguments, and a lenient JSON parser or a retry loop can hide even that. When the call does fail, it looks like the model produced bad JSON, which small models are known for.\n\nAnd the chat endpoint is fine. Anyone who checked `finish_reason` on `/v1/chat/completions` has seen `length` work. The Responses API is newer, it's what Codex CLI and other recent agent tools speak, and the only thing llama-server's README says about how it works is that it \"works by converting Responses request into Chat Completions request.\"\n\nThe conversion is only one way. The request is turned into a chat completion and generated normally, and the server knows the reply stopped at the limit, because the chat endpoint reports `length` from the same generation. When the result is turned back into a Responses object, in `tools/server/server-task.cpp`, the response `status` and the `status` of every output item (message, reasoning, function call) are written as the literal string `\"completed\"`. At the b11753 tag that is seven `{\"status\", \"completed\"}` entries across `to_json_oaicompat_resp()` and `to_json_oaicompat_resp_stream()`, the non-streaming and streaming builders. Neither reads the stop reason, and the string `incomplete` does not appear anywhere in the file. The Responses specification has `status: \"incomplete\"` with `incomplete_details.reason: \"max_output_tokens\"` for exactly this case; llama-server never produces it.\n\nThere's no fix upstream yet; the issue has no replies or PR as of 20 October. On the client side:\n\n`usage.output_tokens == max_output_tokens` as cut off.`function_call` whose arguments don't parse as JSON was cut off, whatever its status says.`/v1/chat/completions` if your client can.`finish_reason: \"length\"` is reported correctly there, for text and for tool calls.\nThe toolkit's `check-llama-responses-status.sh` sends one 8-token request to `/v1/responses` and one to `/v1/chat/completions`, and compares:\n\n```\n  /v1/responses        max_output_tokens=8: status='completed' incomplete_details=None output_tokens=8\n  /v1/chat/completions max_tokens=8:        finish_reason='length'\n\nhttp://127.0.0.1:18090: a reply cut off at max_output_tokens is reported as 'completed'.\n```\n\nIt exits 1 there, 0 when a cut-off reply is reported as `incomplete`, and 2 when it can't tell.\n\nA status field is only useful if something can make it say no. When you rely on one, find out whether the implementation has a code path that ever produces a value other than success, and test the failure case on purpose: set a limit too small for the answer and look at what comes back. A field that is hard-coded to `completed` passes every test that only checks successful replies.\n\nAnd when an API is implemented by translating to another one, check that the translation carries the failure signals back, not just the content. Here the information existed one layer down, in the chat endpoint's `finish_reason`, and was dropped on the way out.\n\nThis is the second place this month where llama-server's compatibility layers lose something on the way through: its Anthropic endpoint [ignores which tool you ask for and stops inside the model's reasoning](https://homelabpostmortem.com/2026/10/18/llama-server-anthropic-endpoint-ignores-tool-choice-and-stops-inside-reasoning/).\n\nThis post's fix is available as a tested, ready-to-run script in the toolkit.", "url": "https://wpnews.pro/news/llama-server-s-responses-api-marks-truncated-replies-and-tool-calls-as-completed", "canonical_source": "https://dev.to/homelabpm/llama-servers-responses-api-marks-truncated-replies-and-tool-calls-as-completed-153n", "published_at": "2026-10-20 22:41:50+00:00", "updated_at": "2026-10-20 22:50:22.819538+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-agents", "developer-tools"], "entities": ["llama.cpp", "llama-server", "ggml-org", "OpenAI", "Codex CLI", "Qwen3-0.6B", "gemma-3-1b-it"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llama-server-s-responses-api-marks-truncated-replies-and-tool-calls-as-completed", "markdown": "https://wpnews.pro/news/llama-server-s-responses-api-marks-truncated-replies-and-tool-calls-as-completed.md", "text": "https://wpnews.pro/news/llama-server-s-responses-api-marks-truncated-replies-and-tool-calls-as-completed.txt", "jsonld": "https://wpnews.pro/news/llama-server-s-responses-api-marks-truncated-replies-and-tool-calls-as-completed.jsonld"}}