# Your OpenAI-compatible endpoint hangs for 180 seconds, then fails. Here is why. > Source: > Published: 2026-11-06 23:40:22+00:00 An LLM request that repeatedly stalls for roughly three minutes is a useful debugging clue, not a diagnosis. A client deadline, gateway timeout, upstream delay, buffering, or retries can all contribute. Neither a 178–194 second duration nor a message such as `TRANSPORT: Connection error` identifies the failing component on its own. Streaming HTTP chat completions typically deliver server-sent events (SSE). WebSocket support and automatic transport selection depend on the particular API, client, and version. There is no universal OpenAI-compatible SDK setting named `transport: sse`. Keys such as `api: openai-completions`, `streamIdleTimeoutMs`, and `retryPolicy` belong to particular applications or wrappers; copying them into an unrelated SDK may do nothing or raise a configuration error. Use your client's documented options and inspect its request logs. If a wrapper really does negotiate WebSockets first, test its documented HTTP/SSE option rather than treating that behavior as common to every SDK. The official Python SDK documents streaming, timeout controls, and retry behavior separately in its [README](https://github.com/openai/openai-python). Check the documentation for the version you installed, including whether it already retries requests, before adding another retry layer. A connection timeout limits establishment of a connection. A read timeout limits a wait for incoming data; it is not necessarily a deadline for the whole response. A stream that keeps sending bytes can outlive a read timeout. A correctly implemented read timeout can also fire before the first body chunk, even if headers have arrived. HTTPX distinguishes connect, read, write, and pool timeouts in its [timeout documentation](https://www.python-httpx.org/advanced/timeouts/). Choose limits for the expected workload: a shorter timeout reduces waiting but may also interrupt a valid long-running generation. A fixed 90-to-180-second ratio is not generally appropriate. A proxy can delay forwarding upstream chunks. For an nginx HTTP proxy location serving SSE, this is a starting point, not a complete server configuration: ``` # Inside the existing location that proxies to the model backend: proxy_buffering off; proxy_cache off; proxy_read_timeout 90s; # example: maximum gap between upstream reads ``` `proxy_read_timeout` is measured between successive reads, not across the entire response. nginx can also honor an upstream `X-Accel-Buffering: no` response header unless configured to ignore it. Other proxies, CDNs, compression layers, or the application itself may still buffer. Changing nginx alone does not guarantee token-by-token delivery. See the [nginx proxy module documentation](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering). String matching an exception for `"502"` or `"timeout"` can misclassify failures. Inspect transport exceptions and HTTP status codes. Repeating an unchanged malformed request or invalid credential is generally not useful; some authentication failures require refreshing credentials, not blindly resending them. The following Python 3.11+ example uses HTTPX directly, not SDK-specific configuration. It demonstrates a non-streaming request so that the entire body is consumed inside the deadline. Configure `BASE`, `KEY`, and `MODEL` in your environment; `BASE` is the API base including `/v1` when required. ``` python import asyncio import os import random import httpx RETRYABLE_STATUS = {408, 429, 500, 502, 503, 504} async def ask_with_retry(client, *, attempts=3, budget_s=60): if attempts < 1 or budget_s <= 0: raise ValueError("attempts and budget_s must be positive") # Includes attempts, response reads, and backoff. async with asyncio.timeout(budget_s): for attempt in range(attempts): try: response = await client.post( os.environ["BASE"].rstrip("/") + "/chat/completions", headers={"Authorization": f"Bearer {os.environ['KEY']}"}, json={ "model": os.environ["MODEL"], "messages": [{"role": "user", "content": "hi"}], "stream": False, }, ) response.raise_for_status() return response.json() except httpx.HTTPStatusError as exc: if exc.response.status_code not in RETRYABLE_STATUS: raise if attempt == attempts - 1: raise except (httpx.TimeoutException, httpx.NetworkError): if attempt == attempts - 1: raise await asyncio.sleep(min(0.8 * 2 ** attempt, 6) + random.uniform(0, 0.3)) async def main(): async with httpx.AsyncClient( timeout=httpx.Timeout(30, connect=5, read=20) ) as client: print(await ask_with_retry(client)) if __name__ == "__main__": asyncio.run(main()) ``` The status list and delays are example policy, not instructions to retry every server error. Production handling should respect a valid `Retry-After` within the remaining budget. A timed-out POST may already have reached the upstream, so repeating it can duplicate work or charges. Do not blindly replay a stream after delivering partial output or executing tool side effects. For streaming, keep both the request and the full stream-consumption loop inside the deadline and close the stream on cancellation. Async timeouts use cooperative cancellation, not a process-level kill switch; blocking code or suppressed cancellation can delay cleanup. See [Python's timeout documentation](https://docs.python.org/3/library/asyncio-task.html#asyncio.timeout). This shell example writes response headers and body to local files so an HTTP error is not mistaken for a successful generation. Set the three environment variables first and inspect the captured body without sharing credentials or sensitive content. ``` curl -sS -N --connect-timeout 10 --max-time 60 \ -D response.headers -o response.body \ -w 'http=%{http_code} dns=%{time_namelookup}s connect=%{time_connect}s tls=%{time_appconnect}s ttfb=%{time_starttransfer}s total=%{time_total}s\n' \ "$BASE/chat/completions" \ -H "Authorization: Bearer $KEY" \ -H 'Content-Type: application/json' \ -d "$(python -c 'import json,os; print(json.dumps({"model":os.environ["MODEL"],"messages":[{"role":"user","content":"hi"}],"stream":True}))')" ``` `time_connect` includes time from the start until connection establishment; DNS and TLS have their own timing fields. `time_starttransfer` measures the first response byte, often a header, not the first generated token. These are cumulative timings, not independent phase durations. `-N` disables curl's output buffering, not a proxy's. See the [curl manual](https://curl.se/docs/manpage.html). A fast test only shows that one request completed; it does not prove the client is at fault. Record HTTP status, headers, body/error events, curl's exit status, and arrival times of actual SSE events. Compare direct and proxied requests where available. A model-list endpoint can help, but presence in a list alone does not establish that the model is callable with your credentials. Related guide: [Your OpenAI-compatible endpoint hangs for 180 seconds](https://csvtriage.com/g/openai-api-hangs-then-fails.html). **If you want the whole set:** I packaged all 30 of these recipes — pacing, JSON repair, failover ladders, evals, cost accounting — into one markdown bundle. It's $19, paid in USDT, download unlocks the moment the transfer confirms, no account needed.