{"slug": "next-js-api-proxy-times-out-after-long-ml-inference-502-navigating-undici-s", "title": "Next.js API Proxy Times Out After Long ML Inference (502) — Navigating Undici's Timeout Quagmire", "summary": "A developer at Workstyle Tech documented how Next.js API routes proxying requests to a FastAPI inference service returned 502 errors during long-running audio generation because the global fetch (undici) has a default timeout of about 300 seconds. The team rewrote the proxy using Node's http/https modules to disable response timeouts while enforcing only connection timeouts, and added configuration to suppress Next.js's response limit warnings. The fix ensures that inference jobs lasting minutes or hours are no longer cut off by the proxy layer.", "body_md": "📝 Originally published (in Japanese) at\n\n[forge.workstyle.tech].\n\nWhen using Next.js to relay requests from the frontend to an inference service (FastAPI) running in a separate process via API routes, it's a common BFF (Backend for Frontend) architecture. However, when running audio generation with a 44.1kHz model on the CPU, processing can take several minutes — or even hours. And then, at some point, this happens:\n\nThe frontend receives a\n\n502error even though the generation hasn't finished.\n\nChecking the inference service logs shows that processing is still running smoothly. The issue isn't with the inference itself — **the bottleneck is the proxy in between**. This article documents how we identified that the culprit was Next.js's global `fetch`\n\n(powered by undici) and its default timeout, and how we rewrote the proxy using Node's standard `http`\n\n/`https`\n\nmodules to bypass it.\n\nThe key observations during debugging were:\n\nThe inference service continues running, but the proxy layer gives up first. The fact that it fails \"after a fixed time\" is a strong hint: there’s a hardcoded timeout somewhere.\n\n`fetch`\n\n(undici)\nWhen you use `fetch()`\n\ndirectly in a Next.js API route, under the hood it uses **undici**, Node.js’s built-in HTTP client. Undici has a default timeout for receiving headers (around 300 seconds), and if no response comes back within that window, it forcibly closes the connection. When inference takes more than 5 minutes, it gets cut off right there — resulting in a 502.\n\n\"Just disable undici’s timeout then!\"\n\nWe tried adjusting `headersTimeout`\n\n/`bodyTimeout`\n\nvia an `Agent`\n\n, but ran into another wall: **it’s hard to directly import and inject undici in this setup**. Modifying the internal implementation of global `fetch`\n\nisn’t clean, and the override may not work across environments.\n\nTemporarily increasing the timeout just kicks the can down the road — eventually, another long-running generation will hit the same limit. The real fix wasn’t to tweak the timeout — it was to **rewrite the proxy layer using a different timeout model entirely**.\n\n`http`\n\n/`https`\n\nInstead of fighting with undici, we rewrote the proxy using Node’s built-in `http`\n\n/`https`\n\nmodules. With standard modules, we gain full control over timeout behavior.\n\nThe key design principle: **distinguish between connection setup and response waiting**.\n\nThis separation is critical.\n\n```\n// Disable response timeout entirely (allow long-running generation). Only enforce connection timeout.\npreq.setTimeout(0);\npreq.on('socket', (s) => {\n  s.setTimeout(30_000, () => {\n    if (!s.destroyed && (s.connecting || !s.writable)) s.destroy(new Error('connect timeout'));\n  });\n  s.once('connect', () => s.setTimeout(0));\n});\n```\n\nBreaking it down:\n\n`preq.setTimeout(0)`\n\n— disables the overall request timeout, allowing the response to be awaited indefinitely`30_000ms`\n\ntimeout that triggers only if the socket is still connecting or not writable — i.e., `connect`\n\nfires, we immediately disable the socket timeout (`setTimeout(0)`\n\n), letting the response stream in however long it takesThis way:\n\nWe also added proper error handling:\n\n``` js\npreq.on('error', (e) => {\n  if (!res.headersSent) res.status(502).json({ error: `backend API unreachable: ${String(e)}` });\n  resolve();\n});\n```\n\nEven after removing the timeout, Next.js API routes have a built-in mechanism that warns when a handler takes too long to respond. To suppress this warning for long-running proxies, we declare:\n\n``` js\nexport const config = {\n  api: {\n    bodyParser: { sizeLimit: '25mb' },\n    responseLimit: '25mb',\n    externalResolver: true, // Tell Next.js: \"This route resolves externally; don't monitor it\"\n  },\n};\n```\n\nWe also increased the body and response size limits since we’re dealing with audio data — the defaults would reject large recordings.\n\n`fetch`\n\nuses undici under the hood`fetch`\n\nin API routes without realizing it has default timeouts, you’ll end up with a hard-to-debug scenario: \"The service is running, but the proxy returns 502.\"`http`\n\n/`https`\n\nand managing the socket directly is more reliable and readable for this kind of requirement.`externalResolver: true`\n\n`fetch`\n\n(undici) `http`\n\n/`https`\n\n`setTimeout(0)`\n\non `connect`\n\nto fail fast only when unreachable`externalResolver: true`", "url": "https://wpnews.pro/news/next-js-api-proxy-times-out-after-long-ml-inference-502-navigating-undici-s", "canonical_source": "https://dev.to/orca_forge/nextjs-api-proxy-times-out-after-long-ml-inference-502-navigating-undicis-timeout-quagmire-14f0", "published_at": "2026-08-25 19:20:06+00:00", "updated_at": "2026-08-25 19:44:02.235043+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure"], "entities": ["Next.js", "FastAPI", "undici", "Node.js", "Workstyle Tech"], "alternates": {"html": "https://wpnews.pro/news/next-js-api-proxy-times-out-after-long-ml-inference-502-navigating-undici-s", "markdown": "https://wpnews.pro/news/next-js-api-proxy-times-out-after-long-ml-inference-502-navigating-undici-s.md", "text": "https://wpnews.pro/news/next-js-api-proxy-times-out-after-long-ml-inference-502-navigating-undici-s.txt", "jsonld": "https://wpnews.pro/news/next-js-api-proxy-times-out-after-long-ml-inference-502-navigating-undici-s.jsonld"}}