📝 Originally published (in Japanese) at
[forge.workstyle.tech].
When using Next.js to relay requests from the frontend to an inference service (FastAPI) running in a separate process via API routes, it's a common BFF (Backend for Frontend) architecture. However, when running audio generation with a 44.1kHz model on the CPU, processing can take several minutes — or even hours. And then, at some point, this happens:
The frontend receives a
502error even though the generation hasn't finished.
Checking the inference service logs shows that processing is still running smoothly. The issue isn't with the inference itself — the bottleneck is the proxy in between. This article documents how we identified that the culprit was Next.js's global fetch
(powered by undici) and its default timeout, and how we rewrote the proxy using Node's standard http
/https
modules to bypass it.
The key observations during debugging were:
The inference service continues running, but the proxy layer gives up first. The fact that it fails "after a fixed time" is a strong hint: there’s a hardcoded timeout somewhere.
fetch
(undici)
When you use fetch()
directly in a Next.js API route, under the hood it uses undici, Node.js’s built-in HTTP client. Undici has a default timeout for receiving headers (around 300 seconds), and if no response comes back within that window, it forcibly closes the connection. When inference takes more than 5 minutes, it gets cut off right there — resulting in a 502.
"Just disable undici’s timeout then!"
We tried adjusting headersTimeout
/bodyTimeout
via an Agent
, but ran into another wall: it’s hard to directly import and inject undici in this setup. Modifying the internal implementation of global fetch
isn’t clean, and the override may not work across environments.
Temporarily increasing the timeout just kicks the can down the road — eventually, another long-running generation will hit the same limit. The real fix wasn’t to tweak the timeout — it was to rewrite the proxy layer using a different timeout model entirely.
http
/https
Instead of fighting with undici, we rewrote the proxy using Node’s built-in http
/https
modules. With standard modules, we gain full control over timeout behavior.
The key design principle: distinguish between connection setup and response waiting.
This separation is critical.
// Disable response timeout entirely (allow long-running generation). Only enforce connection timeout.
preq.setTimeout(0);
preq.on('socket', (s) => {
s.setTimeout(30_000, () => {
if (!s.destroyed && (s.connecting || !s.writable)) s.destroy(new Error('connect timeout'));
});
s.once('connect', () => s.setTimeout(0));
});
Breaking it down:
preq.setTimeout(0)
— disables the overall request timeout, allowing the response to be awaited indefinitely30_000ms
timeout that triggers only if the socket is still connecting or not writable — i.e., connect
fires, we immediately disable the socket timeout (setTimeout(0)
), letting the response stream in however long it takesThis way:
We also added proper error handling:
preq.on('error', (e) => {
if (!res.headersSent) res.status(502).json({ error: `backend API unreachable: ${String(e)}` });
resolve();
});
Even after removing the timeout, Next.js API routes have a built-in mechanism that warns when a handler takes too long to respond. To suppress this warning for long-running proxies, we declare:
export const config = {
api: {
bodyParser: { sizeLimit: '25mb' },
responseLimit: '25mb',
externalResolver: true, // Tell Next.js: "This route resolves externally; don't monitor it"
},
};
We also increased the body and response size limits since we’re dealing with audio data — the defaults would reject large recordings.
fetch
uses undici under the hoodfetch
in API routes without realizing it has default timeouts, you’ll end up with a hard-to-debug scenario: "The service is running, but the proxy returns 502."http
/https
and managing the socket directly is more reliable and readable for this kind of requirement.externalResolver: true
fetch
(undici) http
/https
setTimeout(0)
on connect
to fail fast only when unreachableexternalResolver: true