An API timeout does not prove that the “model is down.” It means one participant in the chain stopped waiting: the client failed to connect, waited too long for data, exhausted the overall deadline, a proxy closed an idle connection, or the gateway did not receive a response from upstream. Fix only the layer you have confirmed instead of increasing every timeout at once.
Приложение / SDK
→ DNS и TCP/TLS соединение
→ корпоративный proxy или reverse proxy
→ API gateway
→ upstream-модель
→ streaming-ответ обратно клиенту
The same user-facing “request timed out” message can originate at different points. Record the start and end times, exception class, HTTP status, and response body. If the API returns a request identifier, keep that too. In the Claude API, for example, every response includes request-id
; the official documentation recommends using it when investigating a specific request.
| Type | What expired | Typical signal | What to change after confirmation |
|---|---|---|---|
| Connect timeout | Time allowed for DNS, TCP, or TLS | No HTTP status; connection/SSL/proxy error | Check DNS, certificates, firewall, and proxy; then adjust the connect timeout |
| Read or idle timeout | A connection exists, but the client waited too long without receiving data | No first byte or next streaming event | Check streaming, response size, and each proxy's idle timer |
| Overall deadline | The entire operation's time budget is exhausted | The client cancels the request regardless of its current phase | Split the work or increase only the justified deadline |
| Upstream timeout | The gateway reached its internal limit and returned an error | An HTTP status and gateway/upstream error body are present | Check provider status, queue, load, and whether retry is allowed |
There are no universal values. The same read timeout can be reasonable for a short classification and too low for a long streaming response. Conversely, an enormous overall deadline can hide a stuck queue while holding client resources.
For requests routed through BetterToken, the API documentation and Dashboard let you correlate the time and status of your request with the model, input/output/cache tokens, and charge. This is a useful API-layer observation point, but it does not replace client-side DNS/TLS logs or prove where a third-party service's request broke.
First, capture the actual settings: connect timeout, read timeout, overall deadline, automatic retries, and streaming mode. Do not rely on your memory of “library defaults”—an SDK version may have changed them.
OpenAI describes APITimeoutError as a request timeout and classifies network, proxy, SSL, and firewall problems as
APIConnectionError
. That distinction is useful: receiving no HTTP response calls for different checks than an error with a code and body.Send the request from the same environment where the application runs. If it succeeds locally but fails in a container or on a server, compare DNS, CA certificates, proxy variables, and the outbound firewall. Check the reverse proxy's connect/read/idle timers separately: increasing the SDK timeout will not extend an intermediary's wait.
If you received an HTTP status, this is no longer a pure connect timeout. Save the error body and request identifier. Do not lump a timeout together with a 429 or context overflow; they have their own response signals.
For long Claude requests, Anthropic recommends streaming or Message Batches, especially when processing may take a long time. Streaming can prevent some idle timeouts, but it does not guarantee completion or override the application's overall deadline.
Run the tests in sequence while keeping the model, endpoint, network, and key unchanged.
Send a minimal request with a short expected output. Record the connection time, time to first byte, complete HTTP status, and duration. If this test fails, increasing the response size is pointless: check the network, authentication, endpoint, and API availability.
After Test A succeeds, increase only the expected output size or enable the original workload. If the connection is established but the long response breaks, compare streaming, read/idle timeout, overall deadline, and proxy behavior. Do not change the model, network, and timeouts together, or the cause will remain unknown.
max_tokens
and context size when the delay comes from a long generation, but do not use this as an explanation without measurements.Before resending, answer this question: could the server have completed the request while the client lost the response? The risk is usually lower for a status read. Before retrying record creation, task execution, message delivery, or a charge, check the actual result or use the endpoint's supported idempotency mechanism.
Minimal checklist:
If the short request succeeds but the controlled long request breaks in the same place again, that is not a reason to increase the timeout without limit. Find the earliest layer that stops waiting, then change only that layer or the form of the work.
Diagnose an API timeout from observable signals: no HTTP response means checking the connection; a status and body mean checking the gateway or upstream; a long stream that breaks means comparing streaming and idle timeouts; an exhausted total budget means changing the architecture or overall deadline. Confirm recovery with two tests, and do not repeat a risky operation until you have checked its actual outcome.
Originally published on the BetterToken blog.
BetterToken provides pay-as-you-go access to AI model APIs through
OpenAI-compatible and Anthropic-compatible endpoints — useful if you are wiring
Claude Code, Codex, or your own tooling to a custom base URL.
See the docs to get started.