{"slug": "why-reasoning-effort-behaves-differently-across-providers", "title": "Why reasoning_effort behaves differently across providers", "summary": "A developer building an OpenAI-compatible chat application found that browser actions silently failed because the app misread a 400 error from gpt-5.6-luna as permanent lack of tool support, when the message actually said to use /v1/responses or set reasoning_effort to 'none'. Reissuing requests with reasoning_effort 'none' while keeping tools attached cut time-to-first-output from 4-5 seconds to 2 seconds, but the same setting was rejected by a self-hosted vLLM endpoint, which required chat_template_kwargs enable_thinking=false instead. The developer also reported that thinking output field names vary across providers (reasoning, reasoning_content, thinking), causing empty-response misclassifications when switching to vLLM.", "body_md": "📝 Originally published (in Japanese) at [forge.workstyle.tech](https://forge.workstyle.tech/blog/openai-compatible-reasoning-effort/?utm_source=devto&utm_medium=crosspost&utm_campaign=openai-compatible-reasoning-effort).\n\nTL;DR The app uses OpenAI-compatible `/v1/chat/completions` endpoints across cloud, self-hosted vLLM, and a relay. Browser actions never ran because an error was classified as lack of tool support. The cause was provider-specific handling of `reasoning_effort` and a classifier that ignored the error details.\n\nThe first rejection occurred when we sent the tools and `reasoning_effort` simultaneously. The target endpoint was `gpt-5.6-luna`. We received a 400 status code with the following message:\n\nFunction tools with reasoning_effort are not supported for gpt-5.6-luna in /v1/chat/completions. To use function tools, use /v1/responses or set reasoning_effort to 'none'.\n\nUpon seeing `tool` and `not supported` in the error body, the application concluded that \"this model does not support tools.\" From then on, it excluded tools from requests entirely.\n\nThis was the erroneous misjudgment introduced at the beginning. Even when asked to perform browser operations, no tools were provided, so not a single operation was executed. Although it might appear that the model was avoiding calling tools, there was simply no opportunity to call them in the first place.\n\nThe error message explicitly provided a solution. To use tools, one should either use `/v1/responses` or set `reasoning_effort` to `'none'`. The application, however, skipped reading that part and immediately made the decision to discard the tools.\n\nThe fix was to read the resolution before treating the rejection as a permanent tool incompatibility. We reissued the request with `reasoning_effort: 'none'` while keeping the tools attached. Essentially, for this model, using tools meant stopping the reasoning process.\n\nSpeed-wise, this change had its appeal. Setting `reasoning_effort: 'none'` reduced the latency from input to the start of the output from 4–5 seconds to just 2 seconds.\n\nWhen I applied the same configuration to the tool-calling path, our own vLLM instance returned a 400 error.\n\nInput should be 'low', 'medium' or 'high'\n\nThe error type was `literal_error`. It wasn't that the field was missing; rather, the endpoint simply did not accept the value `none`.\n\nIf you only test with plain conversation, you might stop right here. Since conversations speed up, it looks like things are working. It was only after testing the tool-calling path that we realized the endpoint would handle values differently.\n\nFor this endpoint, we detect the value rejection and stop sending `reasoning_effort` altogether. To disable thinking, we instead send `chat_template_kwargs: { enable_thinking: false }`.\n\nHere are the measured results for comparison.\n\n| Method of disabling thinking | Time taken | \n|---|---|\n| Do nothing | 87 seconds | \n| `/no_think` | 154 seconds | \n| `enable_thinking=false` | 65 seconds | \n| `reasoning_effort none` | 2 seconds | \n\nThese figures were measured using a model on-device. The differences between methods were significant, meaning that just because configuration names are similar does not imply the behavior will be aligned.\n\nOn top of that, the output fields for thinking don't match up either. The name used varies among three possibilities: `reasoning`, `reasoning_content`, and `thinking`. The writing side was aware of all three, but the reading side was only looking for `reasoning`.\n\nvLLM writes to `reasoning_content`. The moment we switched to our own vLLM, the classifier flagged \"empty response\" five times in a row. The model was returning the thinking, but because the reader wasn't looking at the correct field, we couldn't even count the amount of thinking generated.\n\nThe third point concerns the handling of sequential responses.\n\nThere were some endpoints that ignored the `stream: true` flag and returned standard JSON. The app discards any line that does not start with `data:`. As a result, zero fragments were captured. Since `response.ok` was true, no exception was thrown, and nothing appeared on the screen.\n\nI fixed this by checking the `content-type` to determine whether the response is a streaming response or standard JSON.\n\nBut that wasn't the end of it. One relay server endpoint would return a 200 OK in 94ms only when streaming was enabled, but the body was empty. It would reject non-streaming calls with a 400 error. In empty responses, there are no clues like an error message body.\n\nFor this endpoint, I decided to treat an empty response as an exception. I configured it to retry once after disabling the \"thinking\" specification, and to stop sending requests after that. When I retried without the specification, the correct JSON was returned in 327ms.\n\n| Endpoint behavior | Data sent | \n|---|---|\n| Standard (OpenAI, etc.) | `reasoning_effort: 'none'` | \n| `none` not in the value list (vLLM, rejects with 400) | `chat_template_kwargs: { enable_thinking: false }` | \n| Returns empty when streaming (relay server) | Send nothing | \n\nAll of these were decided based on actual measurements. I don't ask the users. The app learns based on the response from the endpoint.\n\nAfter implementing a fix to account for differences between endpoints, I introduced a couple of new issues myself.\n\nThe logic for detecting rejected values was also checking for the presence of the word `invalid`. If a 400 error body contained `invalid_request_error`, all errors were treated the same.\n\nAs a result, the system incorrectly learned that `gpt-5.6-luna` was an endpoint \"missing the `reasoning_effort` field\". It stopped at the `none` designation required in ①, causing the entire browser operation to fail.\n\nThe root cause was that I had created custom error messages for testing. Since these differed slightly from actual rejections, the discrepancy in detection went unnoticed. I adjusted the tests to use recorded error bodies directly.\n\nIn voice conversations, I had enabled a setting to automatically truncate thoughts. This setting was inadvertently applied to the official cloud endpoints as well, causing `none` to be sent to OpenAI.\n\nEvery response generation request timed out after 60 seconds, resulting in a 500 error. This occurred consistently across four attempts, despite OpenAI's status being \"normal\".\n\nThe code originally noted that \"cloud providers use the same keys for different purposes\". I failed to consider endpoint-specific meanings and applied the setting globally. I corrected this by limiting the setting to local providers only.\n\nAdditionally, there was a flaw in the retry mechanism for rejected fields. Each utterance allowed only one retry. If both `temperature` and `frequency_penalty` were rejected, the first utterance would fail silently.\n\nI updated the logic to remove and retry rejected fields upon each rejection, up to 8 attempts. If a response with at least one character is received before a failure, no retry is attempted.\n\n`reasoning_effort` handling to each endpoint's observed behavior.`chat_template_kwargs: { enable_thinking: false }` when `none` is rejected.", "url": "https://wpnews.pro/news/why-reasoning-effort-behaves-differently-across-providers", "canonical_source": "https://dev.to/orca_forge/why-reasoningeffort-behaves-differently-across-providers-29f4", "published_at": "2026-10-19 13:03:04+00:00", "updated_at": "2026-10-19 13:14:31.776274+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-infrastructure", "developer-tools"], "entities": ["vLLM", "OpenAI", "gpt-5.6-luna"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/why-reasoning-effort-behaves-differently-across-providers", "markdown": "https://wpnews.pro/news/why-reasoning-effort-behaves-differently-across-providers.md", "text": "https://wpnews.pro/news/why-reasoning-effort-behaves-differently-across-providers.txt", "jsonld": "https://wpnews.pro/news/why-reasoning-effort-behaves-differently-across-providers.jsonld"}}