When Your AI Agent Quietly Gets a Half-Finished Answer

A new technical reference documents that AI model APIs, including Anthropic, OpenAI, and vLLM, return HTTP 200 with well-formed bodies when generation hits the max output token ceiling, making truncation indistinguishable from completion without checking the stop-reason metadata field. The survey of 18 API surfaces across 14 vendors finds six different field names and seven truncation values, and highlights a vLLM issue where a truncated tool call was labeled as completed, while frameworks like LlamaIndex and smolagents ship dangerous default ceilings.

Every model API has a ceiling on how many tokens it will generate in one response. When a generation reaches that ceiling, the model stops mid-sentence, mid-object, or mid-tool-call — and the API returns HTTP 200 with a well-formed body. Nothing throws. In a multi-step agent pipeline, the next step receives that half-finished answer and treats it exactly like a complete one. Three terms carry most of the weight here, so it is worth fixing them before the field names start multiplying. Max output tokens is the request-side ceiling on generated tokens — max tokens on most APIs, max output tokens on OpenAI’s Responses API, n predict on llama.cpp’s server. Truncation is what happens when a generation stops because it reached that ceiling rather than because the model was finished. And the stop reason — also called the finish reason, the done reason, or the stop type, depending on whose API you are holding — is the metadata field that tells you which of those two things happened. It is the only signal you get. Anthropic states the premise better than any third party could. Its stop-reason guide opens with the instruction to read the field, and then draws the distinction the rest of this post depends on: “Unlike errors, which indicate failures in processing your request, stop reason tells you why Claude completed its response generation.” A truncated response is not a failed request. It is a successful request whose result happens to be incomplete, and the two are indistinguishable unless you look at the metadata. This post is a pattern reference, not a measurement. It collects the field name and the verbatim truncation value for eighteen API surfaces across fourteen vendor organisations, documents the cases where the signal is rewritten or lost between the engine and your client, and covers what detection actually requires. The per-model numbers — what each model’s maximum output is on each serving surface — are a companion dataset and are deliberately not built here; a later post owns that census. This one owns the failure mode. - 01Truncation is a metadata fact, not a content fact.Every surveyed API returns HTTP 200 and a well-formed body when the output ceiling is hit. The stop-reason field is the only reliable signal, and no vendor documents a content-level alternative. - 02One signal, six field names, seven truncation values.stop reason, finish reason, finishReason, stopReason, done reason and stop type across the surveyed surfaces, plus incomplete details.reason nested inside OpenAI's Responses API. A single portable equality check does not exist. - 03A truncated tool call can arrive labelled as a completed one.vLLM issue 53269, filed and closed on August 21, 2026, documents a streaming handler that rewrote the terminal reason to finish reason: "tool calls" — a success value — for a call that max tokens had cut off mid-arguments. - 04The dangerous defaults live in frameworks, not APIs.LlamaIndex core ships DEFAULT NUM OUTPUTS = 256, and smolagents carries two different hardcoded ceilings in one framework. Anthropic, Cohere, Ollama, llama.cpp, LangChain and the Vercel AI SDK all default to the model maximum or to unlimited. - 05Schema validation cannot detect a cut, and structurally it never could.Conformance machinery needs a complete document to conform. Truncated prose is still valid prose, and a truncated array can still parse into a shorter, syntactically perfect list. 01 — The Failure ModeNothing throws, because nothing failed . The mental model most engineers bring to an API call is that a problem produces an exception. Truncation defeats that model at the protocol level: the request was valid, the server processed it correctly, and the response body is exactly what the schema describes. The only thing wrong with it is that the content stops early, and the protocol has no way to express “this is fine” versus “this is a fragment” other than a small metadata field that most client code never reads. The consequence in a single-call application is mild — a user sees a sentence end abruptly and asks again. The consequence in a chain is different in kind. A research step that was cut at 40% of its output hands a plausible-looking summary to a planning step, which hands a plan built on incomplete evidence to an execution step. Every downstream stage behaves correctly on the input it was given. Nothing in the trace is red. This is the same structural problem as the orchestration mistakes covered in our agentic workflow anti-patterns guide /blog/agentic-workflow-anti-patterns-orchestration-mistakes-2026 , with one aggravating feature: the defect enters the chain from outside your code, carrying a success status. Two design decisions make this worse than it needs to be. The first is that the client libraries mostly do not raise. The OpenAI Python SDK does define an exception for the case — LengthFinishReasonError , whose message reads “Could not parse response content as the length limit was reached” — but it is raised from the structured-output parse path, not from the ordinary create call. An open pull request against that SDK, openai-python 3589 https://github.com/openai/openai-python/pull/3589 opened August 10, 2026 , states the asymmetry in the vendor’s own repository: “The equivalent non-streaming client.chat.completions.create never raises for the same response — it just returns a completion with finish reason=“length”.” Structured output gets an exception; plain text gets a value you have to go and look at. The second is that the field you have to look at is not the same field from one API to the next, and neither is the value it carries. That is section 03, and it is the part worth bookmarking. A stop reason is not an error Anthropic's stop-reason guide: "Unlike errors, which indicate failures in processing your request, stop reason tells you why Claude completed its response generation." The response is a success. The content is a fragment. Parse path only openai-python raises LengthFinishReasonError inside parse chat completion, guarded by a check for parseable input. The plain create call returns a completion with finish reason="length" and no exception at all. Anthropic forces the choice The Messages API types max tokens as Required int — "The maximum number of tokens to generate before stopping." There is no silent default because the caller must pick one. That is the design with no surprise in it. 02 — The CaseA truncated tool call, reported as a completed one. The strongest documented instance of this failure is vllm-project/vllm issue 53269 https://github.com/vllm-project/vllm/issues/53269 , filed and closed on August 21, 2026, and titled “ Bug : streaming reports finish reason=”tool calls“ for a tool call truncated by max tokens, hiding ”length“” . It matters because the signal was not merely absent. It was replaced with a value that means the opposite. The report describes the mechanism precisely: “In the streaming chat handler, once any tool-call delta has been emitted the terminal reason is rewritten to “tool calls” unconditionally. That discards the engine’s actual reason, so a generation cut short by max tokens part way through the arguments is reported to the caller as a The engine knew. vLLM’s own completed tool call.” FinishReason enum documents length as “max tokens was consumed, or max model len was reached” . The frontend overwrote it on the way out. The caller is told the tool call is complete while holding arguments that are not valid JSON — json.loads on tool calls 0 .function.arguments raises, and a client that trusts finish reason has no signal that anything was cut off.vllm-project/vllm issue 53269, August 21, 2026 Two details make this more than a single bug report. The first is that the same request answered differently depending only on whether it was streamed. In the reporter’s words: “The non-streaming path does not have this problem — it only performs the translation when output.finish reason == “stop”. So the same request answers differently depending only on stream, and the streaming answer is the wrong one: OpenAI reports “length” on truncation.” A test suite that exercises the non-streaming path and a production deployment that streams will disagree about whether the bug exists. The second is the reproduction. The report carries a swept reproduction — greedy decoding, temperature 0, a fixed seed, and max tokens swept across the truncation point on vLLM 0.27.0 with --enable-auto-tool-choice --tool-call-parser openai . At max tokens values of 64, 80, 96 and 112 the non-streaming path reported length while the streaming path reported tool calls , and the streamed arguments did not parse as JSON. From 128 upward both paths reported tool calls and the arguments parsed. That is one person’s reproduction on one version in one bug report, not a measurement of vLLM in general — but it is a reproduction with the version string, the flags and the sweep written down, which is more than most reports of this class carry. It is also not an isolated incident. Two earlier pull requests attack the same bug class: 46303 https://github.com/vllm-project/vllm/pull/46303 , “Keep length finish reason for max tokens-truncated streaming tool calls” opened June 21, 2026 , and 47963 https://github.com/vllm-project/vllm/pull/47963 , “Report finish reason=’length’ for tool calls truncated by max tokens in streaming” opened July 8, 2026 . Three dated items on one seam in ten weeks makes it a recurring class rather than a one-off, and the seam is structural: any layer that translates a terminal reason has an opportunity to lose the original. A serving framework shipped a fix for the downstream half of the same problem a week later. NVIDIA’s Dynamo, in PR 13986 https://github.com/ai-dynamo/dynamo/pull/13986 opened August 28, 2026 , describes dispatching a tool call from a first streamed fragment holding only a prefix such as {"path":"/a/very , then deduplicating the later fragments that would have completed the arguments. The PR’s own summary of the consequence is the cleanest sentence on the subject: “A harness that trusts the event — which is the entire point of the event — was handed truncated JSON to execute.” The same PR notes why bugs like this hide: “The failure is currently masked on the vLLM path, where long string arguments happen to arrive in one frame.” An unrelated framing coincidence upstream can keep a truncation bug invisible until the coincidence goes away. success reason is a different category of problem, because there is nothing left in the response to check. The stop reason was the only signal, and it now says the generation finished normally. 03 — The ReferenceSix field names for one signal. The table below is the reason this post exists. It lists the field that carries the truncation signal, the truncation value quoted verbatim from the vendor’s own documentation or source tree, and what happens to that signal when the response is streamed — for eighteen API surfaces across fourteen vendor organisations. Documentation states are current as of retrieval on August 30, 2026; source-tree reads are from each project’s default branch on the same date. Three structural facts fall out of it. Six different field names carry the signal stop reason , finish reason , finishReason , stopReason , done reason , stop type , plus incomplete details.reason nested inside a seventh shape. Seven distinct values mean truncation, and two of them do not contain the word “length” at all. And the same two words can mean opposite things across two APIs: on Anthropic’s Messages API stop reason is the truncation field, while inside vLLM stop reason is a different field entirely — “The stop string or token id that caused the completion to stop” — and finish reason is the one you want. | Vendor / API | Field name | Truncation value verbatim | Streaming behaviour | Source | |---|---|---|---|---| | Frontier vendor APIs | |||| | Anthropic Messages API | stop reason | max tokens — “The response reached your max tokens limit.” Also model context window exceeded — “The response filled the model’s context window.” | null in message start ; provided in message delta ; not provided in any other event | docs.anthropic.com · stop reasons | | OpenAI Chat Completions | finish reason | length — “if the maximum number of tokens specified in the request was reached” | same field on chat completion chunk , typed nullable; carried on the terminal chunk | platform.openai.com · chat/object | | OpenAI Responses API | status + incomplete details.reason | status: “incomplete” with reason: “max output tokens” | dedicated terminal event type: “response.incomplete” | platform.openai.com · responses/object | | Google Gemini generateContent | finishReason | MAX TOKENS — “The maximum number of tokens as specified in the request was reached.” | per candidate; “If empty, the model has not stopped generating tokens.” Companion finishMessage set only when finishReason is set | ai.google.dev · generate-content | | Google GenAI Python SDK | finish reason FinishReason enum | MAX TOKENS — “Token generation reached the configured maximum output tokens.” | same enum on streamed candidates | googleapis/python-genai · types.py | | Mistral chat completions | finishReason / finish reason | two values — length and model length , defined in Mistral’s official TypeScript SDK as Length: “length” and ModelLength: “model length” | not documented on the retrieved page | mistralai/client-ts · chatcompletionchoice.ts | | Cohere Chat v2 | finish reason | MAX TOKENS — “the finish reason field in the response will be set to ’MAX TOKENS’”; allowed values COMPLETE, STOP SEQUENCE, MAX TOKENS, TOOL CALL, ERROR, TIMEOUT | not documented on the retrieved page | docs.cohere.com · reference/chat | | AWS Bedrock Converse | stopReason | max tokens — also model context window exceeded , malformed model output and malformed tool use in the same nine-value enum | ConverseStream — not retrieved | docs.aws.amazon.com · API Converse | | Gateways and OpenAI-compatible surfaces | |||| | OpenRouter | finish reason + native finish reason | normalised length ; “The raw finish reason string returned by the model is available via the native finish reason property.” | final chunk before DONE ; “Unlike OpenAI’s spec, this chunk contains a non-empty choices array… that repeats the finish reason of the stream.” | openrouter.ai · api reference | | Fireworks | finish reason | length — “the message content may be partially cut off… In this case the return value might not be a valid JSON.” | metrics arrive in “the final chunk when finish reason is set ” | fireworks.ai · post-chatcompletions | | LiteLLM | finish reason + provider specific fields | normalised length ; original kept in native finish reason when it differs | not stated on the cited page | docs.litellm.ai · completion/output | | Self-hosted servers | |||| | vLLM engine | finish reason , plus a separate stop reason | length — “max tokens was consumed, or max model len was reached” | see issue 53269 — the streaming chat handler rewrote the terminal reason to tool calls once a tool-call delta had been emitted | vllm · v1/engine/ init .py | | llama.cpp server native | stop type , plus boolean truncated | limit — “Stopped because n predict tokens were generated before stop words or EOS was encountered” | “only content, tokens and stop will be returned until end of completion” | ggml-org/llama.cpp · server README | | Ollama native API | done reason | “length” , from DoneReason.String — not enumerated in docs/api.md or api/types.go | appears on the final done: true object | ollama · llm/server.go | | Ollama OpenAI-compatible shim | finish reason | “length” passes through correctly; the connection-closed reason serialises to the empty string and cmp.Or r.DoneReason, “stop” fills in “stop” | FinishChunk emits a dedicated finish-reason chunk with an empty delta | ollama · openai/openai.go | | Client libraries and orchestration layers | |||| | Vercel AI SDK v4 provider interface | finishReason.unified + finishReason.raw | ’length’ — “model generated maximum number of tokens” | same object on the end-of-call callbacks | vercel/ai · language-model-v4-finish-reason.ts | | LangChain ChatAnthropic | response metadata “stop reason” | raw Anthropic value, unmodified; never raised as an exception | populated on the final chunk from message delta | langchain · chat models.py | | OpenAI Python SDK parse path | raises LengthFinishReasonError | “Could not parse response content as the length limit was reached” | raised from get final completion ; plain create “never raises… it just returns a completion with finish reason=’length’” | openai/openai-python · PR 3589 | Empty cells in that table are honest. Where a vendor’s retrieved documentation does not state the streaming placement, the cell says so rather than carrying an inference. One surface is missing altogether: Together AI’s finish reason enum was not enumerated on the retrieved reference page, so it has no row here at all. A reference whose value is that every cell is quotable cannot fill one in from memory. Three details in the table are worth pulling out because they break assumptions rather than merely varying. Anthropic splits truncation into two reasons — your ceiling max tokens and the model’s context window model context window exceeded , the second of which is “currently typed only in the SDKs’ beta namespace” and requires the model-context-window-exceeded-2025-08-26 beta header on models earlier than Sonnet 4.5. Cohere merges the same two conditions into one MAX TOKENS value covering both “the model’s context length” and “the value specified via the max tokens parameter”. So a normalisation layer between them cannot round-trip the distinction in either direction. And Gemini’s FinishReason enum is by far the largest surveyed, at 21 values. Exactly one of them — MAX TOKENS — means you hit your ceiling. Four more MALFORMED FUNCTION CALL , UNEXPECTED TOOL CALL , TOO MANY TOOL CALLS , MALFORMED RESPONSE mean the output is unusable for other reasons. The common client shape — a switch on STOP with everything else falling into a default branch — swallows twenty distinct conditions into one. OpenRouter native finish reason OpenRouter normalises each model's finish reason to five values for portability and keeps the original in native finish reason. You get a stable check and a lossless record of what the provider actually said. LiteLLM provider specific fields Same pattern, and LiteLLM's docs state the reason it matters: "useful for agent loops that need to distinguish between different stop conditions e.g., Gemini's MALFORMED FUNCTION CALL vs a normal stop ." AI SDK provider mappers The Anthropic mapper maps max tokens and model context window exceeded to 'length' and everything unrecognised to 'other'. A new vendor truncation value becomes 'other' until the mapper is updated — raw is the only place it survives. Ollama’s connection-closed path Ollama's max-tokens path maps correctly: DoneReasonLength returns "length" and the shim passes it through. It is the connection-closed reason that degrades — it falls through to an empty string, and cmp.Or r.DoneReason, "stop" turns that into a clean finish reason: "stop". 04 — StreamingThe signal arrives once , in one event. Streaming is where consumers lose the signal without any bug being involved, because the field is not on the frames most code reads. Anthropic documents the placement exactly: “When using streaming, stop reason is: null in the initial message start event; Provided in the message delta event; Not provided in any other events.” A consumer that subscribes to content block delta to append text to a buffer — which is the obvious way to write a streaming client — never touches the event that carries the reason.The pattern repeats with local variations. OpenAI’s Chat Completions types finish reason as nullable on the streaming chunk schema, where the non-streaming object has no null: every intermediate chunk carries null and exactly one chunk carries the value. The Responses API emits a dedicated terminal event, type: “response.incomplete” , which means a stream consumer listening only for response.completed waits for an event that is never coming. OpenRouter deliberately diverges from OpenAI’s spec by putting a content-free choice with the finish reason in the final chunk before DONE — correct behaviour that a strict OpenAI parser may discard as malformed. And llama.cpp’s native server does not send the field during a stream at all: “In streaming mode stream , only content, tokens and stop will be returned until end of completion.” Ollama’s OpenAI-compatible shim deserves a precise statement, because it is easy to get backwards. The max-tokens path is correct: Ollama’s internal DoneReasonLength stringifies to “length” , and the shim passes that through. It is the connection-closed path that degrades — DoneReasonConnectionClosed falls through the switch to the default branch and serialises to the empty string, and the shim’s FinishChunk then applies cmp.Or r.DoneReason, “stop” , which fills the empty value with “stop” . So a generation cut off by a dropped connection reaches an OpenAI-shaped client as a clean, natural completion. A generation cut off by the token ceiling does not. The practical rule for any streaming consumer is that reading the content stream is not enough. You need the terminal event, whatever it is called on your surface, and you need to record what it said — which is an observability requirement as much as a correctness one. Our guide to trace-quality mistakes /blog/agent-observability-anti-patterns-trace-quality-mistakes-2026 covers the span-level side of that: the stop reason belongs on the span, next to the token counts, on every model call. 05 — ShapeWhy JSON-shaped truncation is worse than a crash . Fireworks writes the consequence into its API reference more plainly than anyone: “Also note that the message content may be partially cut off if finish reason=“length”, which indicates the generation exceeded max tokens or the conversation exceeded the max context length. In this case the return value might not be a valid JSON.” That sentence is doing something unusual for API documentation: it is telling you that a successful response may contain a payload your parser cannot read. It helps to order the outcomes by what the consumer can still detect. The ladder below runs from “annoying but visible” to “undetectable from the response”, and the last rung is the one worth thinking about before it happens to you. | Rung | Shape of the output | Detectable by | Sourced to | |---|---|---|---| | 1 | Truncated prose | the stop reason only | Anthropic streaming placement, stop-reason guide | | 2 | Truncated JSON that fails to parse | the stop reason, or a parse error | Fireworks — “might not be a valid JSON” | | 3 | Truncated tool call reported with a truncation reason | the stop reason plus the type of the last content block | Anthropic’s own two-field detection sample | | 4 | Truncated tool call reported with a success reason | nothing in the response | vllm-project/vllm 53269, Aug 21, 2026 | | 5 | Truncated JSON that parses into something valid but short | nothing at all | reasoning about the failure mode — no vendor documents this case | Rung 5 needs its label kept on. It is reasoning about the shape of the failure, not a documented case and not a measured one: a cut inside an array of objects can leave a document that is syntactically complete and semantically short — ten results where twenty were requested, and nothing anywhere saying so. No vendor documents a rate for it and there is no number to give. The two practitioner reports that come closest describe the adjacent shape, truncated tool-call arguments being silently replaced with an empty object. This is also the boundary with a subject that already has its own post, and the boundary is sharp. Schema conformance — grammar- constrained decoding, strict tool-use modes, the portability limits of one shared schema across vendors — is the subject of our guide to structured output reliability in production /blog/llm-structured-output-json-reliability-production . None of that machinery detects a cut, and structurally it cannot: conformance needs a complete document to check, and truncation is precisely the case where no complete document exists. The two problems look similar in a stack trace and have nothing in common in their causes. If you are working the parse-path angle specifically, our OpenAI structured outputs guide /blog/openai-structured-outputs-complete-guide covers where the SDK does raise. Fireworks documents one more wrinkle worth knowing, because it turns a truncation into a latency incident first: “when using JSON mode, it’s crucial to also instruct the model to produce JSON via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly ’stuck’ request.” The request that eventually returns a truncated response can spend its whole budget emitting nothing at all. 06 — DefaultsThe low ceilings live in frameworks , not APIs. The intuitive place to look for a dangerous default is the vendor API. That is the wrong place. Across the surveyed surfaces, the APIs and first-party SDKs either require you to choose a ceiling or default to the model’s own maximum. The fixed low numbers are in the frameworks that sit between you and the API — the layer you are least likely to be reading when you debug a short answer. Hardcoded output ceilings in agent frameworks · tokens Source: run-llama/llama index constants.py and huggingface/smolagents models.py, default branch, read 2026-08-30LlamaIndex core ships DEFAULT NUM OUTPUTS = 256 tokens in its constants module. Phrase that carefully: the framework-level default constant is 256, which is roughly 190 English words. Individual LLM integrations may override it, and this post did not check every one, so it is not a claim that every LlamaIndex call is capped at 256 — it is a claim about the constant the framework falls back to. smolagents is the more surprising row, because the ceiling depends on which backend you selected. Its vLLM backend uses max tokens=kwargs.get “max tokens”, 2048 , while its Transformers backend resolves through a chain of optional keyword arguments and falls through to 1024 . Two different silent ceilings inside one agent framework, differing by a factor of two, selected by a configuration choice that has nothing to do with output length. | SDK / framework | Default | Source | |---|---|---| | Capped — a fixed number low enough to truncate real work | || | LlamaIndex core | DEFAULT NUM OUTPUTS = 256 tokens with DEFAULT CONTEXT WINDOW = 3900 | run-llama/llama index · llama-index-core constants.py | | smolagents — vLLM backend | max tokens=kwargs.get “max tokens”, 2048 | huggingface/smolagents · models.py | | smolagents — Transformers backend | falls through the keyword chain to 1024 | huggingface/smolagents · models.py | | Safe — resolves to the model maximum, or to unlimited | || | Anthropic Messages API | no default — max tokens: Required int | anthropic-sdk-python · message create params.py | | Cohere Chat v2 | “If not set, max tokens defaults to the model’s maximum output token limit.” Over-setting it silently caps at that maximum rather than erroring | docs.cohere.com · reference/chat | | llama.cpp server | n predict — “Default: -1, where -1 is infinity” | ggml-org/llama.cpp · server README | | Ollama | DefaultOptions sets NumPredict: -1 | ollama · api/types.go | | LangChain ChatOpenAI | max tokens: int | None = Field default=None, alias=“max completion tokens” | langchain-openai · chat models/base.py | | LangChain ChatAnthropic | default=None , docstring: “If not specified, this is set dynamically using the model’s max output tokens” | langchain-anthropic · chat models.py | | Vercel AI SDK | maxOutputTokens documented with no default | ai-sdk.dev · ai-sdk-core/settings | | Hugging Face transformers | GenerationConfig pops both max length and max new tokens as None ; a per-checkpoint generation config.json can still cap | huggingface/transformers · configuration utils.py | | OpenAI Chat Completions | request-side default for max completion tokens — not documented on the retrieved reference | platform.openai.com · chat/object | | OpenAI Responses API | documented minimum for max output tokens is 16 — a minimum, not a default | platform.openai.com · responses/object | Two entries in the safe group are worth reading as design lessons. Anthropic’s Messages API has no default because max tokens is a required parameter — the design that forces an explicit choice is the design with no surprise in it. LangChain’s ChatAnthropic resolves the same required parameter dynamically from the model’s own max output tokens rather than pinning a constant, which is the right way to satisfy a required field on the caller’s behalf. Any advice you may have read that this integration pins a hardcoded 1024 is out of date against the current source. One folk claim deserves killing while we are here. The widely repeated line that Hugging Face transformers defaults to 20 output tokens does not hold against the current source: the GenerationConfig constructor pops both max length and max new tokens as None . A particular checkpoint’s generation config.json can still set a low value, which is a per-model fact, not a library default — and worth checking on the checkpoint you actually load. Raising a ceiling is not free, which is the one place this subject touches cost. Anthropic’s extended thinking shares the budget: “Requires a minimum budget of 1,024 tokens and counts towards your max tokens limit” , so turning thinking on without raising the ceiling reduces what is left for the answer. And OpenAI’s max output tokens on the Responses API is “An upper bound for the number of tokens that can be generated for a response, including visible output tokens and reasoning tokens” — on a reasoning model, thinking can consume the whole ceiling and leave a structurally valid, visibly empty response. The pricing side of running with generous ceilings is covered in our work on long-context pricing thresholds /blog/long-context-pricing-thresholds-llm-cost-cliffs . 07 — DetectionWhat actually works , in order. Detection has one primary mechanism and three supporting ones, and it is worth being clear about which is which. The primary mechanism is reading the stop reason on every model call, on the surface you are actually calling, including the streaming terminal event. Everything else corroborates. Read the terminal reason, always Assert on the field your surface uses — and on both fields where the surface has two, as llama.cpp does with stop type and the boolean truncated. Treat an unrecognised value as suspect rather than routing it to a default branch. Output-token accounting Every surveyed API returns a usage object alongside the reason, so output tokens landing at the configured ceiling is a strong second signal. Use a tolerance, not equality: llama.cpp warns its limit "may exceed the set limit slightly" on a partial multibyte character. Structural validation Parsing the payload catches rung 2 and nothing else. It cannot catch truncated prose, because truncated prose is still valid prose, and it cannot catch a truncated array that parses into a shorter list. Sentinel terminators Instruct the model to end its output with an agreed marker and check for it. This survives a normalisation layer because it does not depend on the vendor's field at all — but it is a convention your pipeline defines and enforces, not an API feature any vendor offers. Anthropic’s own recommended handling for the tool-call case is a two-field check rather than a single comparison: test whether stop reason equals “max tokens” , then inspect the last content block and see whether its type is tool use . That shape generalises. The reason alone tells you the generation was cut; the shape of what you are holding tells you what to do about it. The sentinel idea is worth stating carefully because it is the pattern most likely to be mistaken for a feature. No vendor documents a canonical sentinel convention. Anthropic’s sample handling does the inverse — it appends a literal notice such as Response truncated due to max tokens limit to the text it returns to a user — which is the same idea run the other way: a marker a consumer can look for. If you adopt one, your pipeline defines it, your prompts request it, and your validation enforces it. Two things follow for how you instrument this. First, the check belongs in the pipeline, not in the model. Truncation is one of the clearest cases in the taxonomy of what an agent can and cannot verify about its own output /blog/agent-self-verification-limits-by-output-modality-2026 : a model that was cut off cannot report that it was cut off, because the report would have to come after the token that never arrived. Only external metadata can tell you. Second, whatever you read should be recorded. LangChain surfaces the raw Anthropic value in response metadata “stop reason” and never raises on it, which makes a one-line assertion in the chain step the cheapest fix available. Observability vendors are treating this as a first-class case too — LiteLLM merged test coverage for empty, whitespace and truncated tool-call arguments in its OpenTelemetry output in August 2026. 08 — RemedyContinue the prose, re-run the tool call. Once you can see truncation, the response depends on what got cut, and the vendor guidance is more specific than it first appears. Anthropic’s stop-reason table prescribes “Raise max tokens or continue the response” for the max tokens case — two options for prose. For the tool-call case the same page prescribes something different: “If Claude’s response is cut off because it hit the max tokens limit, and the truncated response contains an incomplete tool use block, you’ll need to retry the request with a higher max tokens value to get the full tool use.” A full retry, not a continuation. That distinction is the most useful operational line in this post. Continuation works on prose because prose concatenates: the second half attaches to the first and the result reads correctly. It does not work on a structured emission, because a tool call cut mid-arguments has no valid prefix to continue from — you are holding a fragment of a serialisation, and stitching a second fragment onto it produces something no schema described. The vendor’s own guidance is that you re-run it at a higher ceiling. For the second Anthropic value, model context window exceeded , the same table’s prescription is simply to “Treat the response as truncated” — raising your request ceiling does not help when the model’s own window is what filled. That is a prompt-size and context-management problem rather than a parameter problem, and it belongs with the runtime context work in our context engineering playbook /blog/context-engineering-agent-reliability-playbook-2026 . One more habit is worth building in early: decide what a truncated step should do to the run. Discarding the partial output and retrying costs tokens; passing it downstream costs correctness; failing the run loudly costs a restart. Whichever you choose, choose it explicitly and log the reason, because the alternative — the default in most pipelines today — is to pass the fragment along and find out three steps later. 09 — ConclusionOne field, read on every call. Truncation is a metadata fact. If nothing in your pipeline reads the metadata, a half-finished answer is indistinguishable from a finished one. The failure mode is structural rather than incidental. Model APIs report a completed request that produced incomplete content, and the only thing separating the two is a field that most client code steps over. Eighteen documented surfaces spell that field six different ways and fill it with seven different truncation values, two of which do not contain the word “length”. A single portable equality check does not exist, which is why the table above is the useful part of this post. The signal also degrades in transit, and there is dated first-party evidence for each step of that. Gateways like OpenRouter and LiteLLM do it correctly — normalise for portability, preserve the raw value for correctness. The AI SDK’s provider mappers collapse unrecognised values to a generic bucket. Ollama’s connection-closed path serialises to an empty string that its OpenAI-compatible shim fills in as “stop” , while its max-tokens path maps correctly. And vLLM’s streaming handler, in issue 53269 on August 21, 2026, rewrote the terminal reason to a success value for a tool call that had been cut mid-arguments. None of the remedies is difficult. Read the terminal reason on every call and on the streaming path specifically; corroborate with output token counts against your configured ceiling; check where your ceiling came from, because the low ones live in frameworks rather than APIs; and remember that a truncated tool call gets re-run at a higher ceiling rather than continued. The reason this is worth writing down is not that the fix is hard — it is that nothing in the system will ever prompt you to apply it.