Ollama v1 OpenAI Compat Drops Think Toggle

A developer found that Ollama's OpenAI-compatible /v1 endpoint silently drops the think: false toggle for Qwen3-family models, causing them to spend their entire num_predict budget on internal reasoning and return empty content. The team worked around the bug by switching to Ollama's native /api/chat endpoint, which respects the toggle, accepting the cost of refactoring their code and possible differences in rate limits and pricing.

We recently ran into an issue with the Ollama /v1 endpoint, which is designed to be compatible with OpenAI's API. Our models, particularly those from the Qwen3 family, were returning empty content. This was not the behavior we expected, given the models' capabilities. We started by examining the requests and responses to identify what was going wrong. We noticed that the /v1 endpoint was silently dropping the think: false toggle for the Qwen3-family models. This toggle is crucial as it controls whether the model spends its entire num predict budget on internal reasoning or not. Here's a snippet of the payload we were sending: { "model": "qwen3", "messages": {"role": "user", "content": "Translate 'hello' to French."} , "think": false } We expected the model to translate "hello" to French without extensive internal reasoning. However, the model was spending its entire num predict budget on reasoning and returning empty content. To fix this issue, we switched to the native /api/chat endpoint provided by Ollama. This endpoint allows us to have more control over the model's behavior and ensures that the think toggle is respected. Here's the payload we used with the /api/chat endpoint: { "model": "qwen3", "messages": {"role": "user", "content": "Translate 'hello' to French."} , "think": false } By switching to the /api/chat endpoint, we were able to control the model's behavior more effectively and get the expected output. Switching to the /api/chat endpoint has its own set of tradeoffs. While it gives us more control over the model's behavior, it also requires us to refactor our code to handle the new endpoint. Additionally, the /api/chat endpoint may have different rate limits and pricing structures compared to the /v1 endpoint. However, the benefits of having more control over the model's behavior and getting the expected output outweigh these tradeoffs for us. We are currently exploring other endpoints and features provided by Ollama to see if they offer any additional benefits for our use case. We are also working on improving our monitoring and alerting systems to catch such issues earlier.