Why reasoning_effort behaves differently across providers

A developer building an OpenAI-compatible chat application found that browser actions silently failed because the app misread a 400 error from gpt-5.6-luna as permanent lack of tool support, when the message actually said to use /v1/responses or set reasoning_effort to 'none'. Reissuing requests with reasoning_effort 'none' while keeping tools attached cut time-to-first-output from 4-5 seconds to 2 seconds, but the same setting was rejected by a self-hosted vLLM endpoint, which required chat_template_kwargs enable_thinking=false instead. The developer also reported that thinking output field names vary across providers (reasoning, reasoning_content, thinking), causing empty-response misclassifications when switching to vLLM.

📝 Originally published in Japanese at forge.workstyle.tech https://forge.workstyle.tech/blog/openai-compatible-reasoning-effort/?utm source=devto&utm medium=crosspost&utm campaign=openai-compatible-reasoning-effort . TL;DR The app uses OpenAI-compatible /v1/chat/completions endpoints across cloud, self-hosted vLLM, and a relay. Browser actions never ran because an error was classified as lack of tool support. The cause was provider-specific handling of reasoning effort and a classifier that ignored the error details. The first rejection occurred when we sent the tools and reasoning effort simultaneously. The target endpoint was gpt-5.6-luna . We received a 400 status code with the following message: Function tools with reasoning effort are not supported for gpt-5.6-luna in /v1/chat/completions. To use function tools, use /v1/responses or set reasoning effort to 'none'. Upon seeing tool and not supported in the error body, the application concluded that "this model does not support tools." From then on, it excluded tools from requests entirely. This was the erroneous misjudgment introduced at the beginning. Even when asked to perform browser operations, no tools were provided, so not a single operation was executed. Although it might appear that the model was avoiding calling tools, there was simply no opportunity to call them in the first place. The error message explicitly provided a solution. To use tools, one should either use /v1/responses or set reasoning effort to 'none' . The application, however, skipped reading that part and immediately made the decision to discard the tools. The fix was to read the resolution before treating the rejection as a permanent tool incompatibility. We reissued the request with reasoning effort: 'none' while keeping the tools attached. Essentially, for this model, using tools meant stopping the reasoning process. Speed-wise, this change had its appeal. Setting reasoning effort: 'none' reduced the latency from input to the start of the output from 4–5 seconds to just 2 seconds. When I applied the same configuration to the tool-calling path, our own vLLM instance returned a 400 error. Input should be 'low', 'medium' or 'high' The error type was literal error . It wasn't that the field was missing; rather, the endpoint simply did not accept the value none . If you only test with plain conversation, you might stop right here. Since conversations speed up, it looks like things are working. It was only after testing the tool-calling path that we realized the endpoint would handle values differently. For this endpoint, we detect the value rejection and stop sending reasoning effort altogether. To disable thinking, we instead send chat template kwargs: { enable thinking: false } . Here are the measured results for comparison. | Method of disabling thinking | Time taken | |---|---| | Do nothing | 87 seconds | | /no think | 154 seconds | | enable thinking=false | 65 seconds | | reasoning effort none | 2 seconds | These figures were measured using a model on-device. The differences between methods were significant, meaning that just because configuration names are similar does not imply the behavior will be aligned. On top of that, the output fields for thinking don't match up either. The name used varies among three possibilities: reasoning , reasoning content , and thinking . The writing side was aware of all three, but the reading side was only looking for reasoning . vLLM writes to reasoning content . The moment we switched to our own vLLM, the classifier flagged "empty response" five times in a row. The model was returning the thinking, but because the reader wasn't looking at the correct field, we couldn't even count the amount of thinking generated. The third point concerns the handling of sequential responses. There were some endpoints that ignored the stream: true flag and returned standard JSON. The app discards any line that does not start with data: . As a result, zero fragments were captured. Since response.ok was true, no exception was thrown, and nothing appeared on the screen. I fixed this by checking the content-type to determine whether the response is a streaming response or standard JSON. But that wasn't the end of it. One relay server endpoint would return a 200 OK in 94ms only when streaming was enabled, but the body was empty. It would reject non-streaming calls with a 400 error. In empty responses, there are no clues like an error message body. For this endpoint, I decided to treat an empty response as an exception. I configured it to retry once after disabling the "thinking" specification, and to stop sending requests after that. When I retried without the specification, the correct JSON was returned in 327ms. | Endpoint behavior | Data sent | |---|---| | Standard OpenAI, etc. | reasoning effort: 'none' | | none not in the value list vLLM, rejects with 400 | chat template kwargs: { enable thinking: false } | | Returns empty when streaming relay server | Send nothing | All of these were decided based on actual measurements. I don't ask the users. The app learns based on the response from the endpoint. After implementing a fix to account for differences between endpoints, I introduced a couple of new issues myself. The logic for detecting rejected values was also checking for the presence of the word invalid . If a 400 error body contained invalid request error , all errors were treated the same. As a result, the system incorrectly learned that gpt-5.6-luna was an endpoint "missing the reasoning effort field". It stopped at the none designation required in ①, causing the entire browser operation to fail. The root cause was that I had created custom error messages for testing. Since these differed slightly from actual rejections, the discrepancy in detection went unnoticed. I adjusted the tests to use recorded error bodies directly. In voice conversations, I had enabled a setting to automatically truncate thoughts. This setting was inadvertently applied to the official cloud endpoints as well, causing none to be sent to OpenAI. Every response generation request timed out after 60 seconds, resulting in a 500 error. This occurred consistently across four attempts, despite OpenAI's status being "normal". The code originally noted that "cloud providers use the same keys for different purposes". I failed to consider endpoint-specific meanings and applied the setting globally. I corrected this by limiting the setting to local providers only. Additionally, there was a flaw in the retry mechanism for rejected fields. Each utterance allowed only one retry. If both temperature and frequency penalty were rejected, the first utterance would fail silently. I updated the logic to remove and retry rejected fields upon each rejection, up to 8 attempts. If a response with at least one character is received before a failure, no retry is attempted. reasoning effort handling to each endpoint's observed behavior. chat template kwargs: { enable thinking: false } when none is rejected.