# Next.js API Proxy Times Out After Long ML Inference (502) — Navigating Undici's Timeout Quagmire

> Source: <https://dev.to/orca_forge/nextjs-api-proxy-times-out-after-long-ml-inference-502-navigating-undicis-timeout-quagmire-14f0>
> Published: 2026-08-25 19:20:06+00:00

📝 Originally published (in Japanese) at

[forge.workstyle.tech].

When using Next.js to relay requests from the frontend to an inference service (FastAPI) running in a separate process via API routes, it's a common BFF (Backend for Frontend) architecture. However, when running audio generation with a 44.1kHz model on the CPU, processing can take several minutes — or even hours. And then, at some point, this happens:

The frontend receives a

502error even though the generation hasn't finished.

Checking the inference service logs shows that processing is still running smoothly. The issue isn't with the inference itself — **the bottleneck is the proxy in between**. This article documents how we identified that the culprit was Next.js's global `fetch`

(powered by undici) and its default timeout, and how we rewrote the proxy using Node's standard `http`

/`https`

modules to bypass it.

The key observations during debugging were:

The inference service continues running, but the proxy layer gives up first. The fact that it fails "after a fixed time" is a strong hint: there’s a hardcoded timeout somewhere.

`fetch`

(undici)
When you use `fetch()`

directly in a Next.js API route, under the hood it uses **undici**, Node.js’s built-in HTTP client. Undici has a default timeout for receiving headers (around 300 seconds), and if no response comes back within that window, it forcibly closes the connection. When inference takes more than 5 minutes, it gets cut off right there — resulting in a 502.

"Just disable undici’s timeout then!"

We tried adjusting `headersTimeout`

/`bodyTimeout`

via an `Agent`

, but ran into another wall: **it’s hard to directly import and inject undici in this setup**. Modifying the internal implementation of global `fetch`

isn’t clean, and the override may not work across environments.

Temporarily increasing the timeout just kicks the can down the road — eventually, another long-running generation will hit the same limit. The real fix wasn’t to tweak the timeout — it was to **rewrite the proxy layer using a different timeout model entirely**.

`http`

/`https`

Instead of fighting with undici, we rewrote the proxy using Node’s built-in `http`

/`https`

modules. With standard modules, we gain full control over timeout behavior.

The key design principle: **distinguish between connection setup and response waiting**.

This separation is critical.

```
// Disable response timeout entirely (allow long-running generation). Only enforce connection timeout.
preq.setTimeout(0);
preq.on('socket', (s) => {
  s.setTimeout(30_000, () => {
    if (!s.destroyed && (s.connecting || !s.writable)) s.destroy(new Error('connect timeout'));
  });
  s.once('connect', () => s.setTimeout(0));
});
```

Breaking it down:

`preq.setTimeout(0)`

— disables the overall request timeout, allowing the response to be awaited indefinitely`30_000ms`

timeout that triggers only if the socket is still connecting or not writable — i.e., `connect`

fires, we immediately disable the socket timeout (`setTimeout(0)`

), letting the response stream in however long it takesThis way:

We also added proper error handling:

``` js
preq.on('error', (e) => {
  if (!res.headersSent) res.status(502).json({ error: `backend API unreachable: ${String(e)}` });
  resolve();
});
```

Even after removing the timeout, Next.js API routes have a built-in mechanism that warns when a handler takes too long to respond. To suppress this warning for long-running proxies, we declare:

``` js
export const config = {
  api: {
    bodyParser: { sizeLimit: '25mb' },
    responseLimit: '25mb',
    externalResolver: true, // Tell Next.js: "This route resolves externally; don't monitor it"
  },
};
```

We also increased the body and response size limits since we’re dealing with audio data — the defaults would reject large recordings.

`fetch`

uses undici under the hood`fetch`

in API routes without realizing it has default timeouts, you’ll end up with a hard-to-debug scenario: "The service is running, but the proxy returns 502."`http`

/`https`

and managing the socket directly is more reliable and readable for this kind of requirement.`externalResolver: true`

`fetch`

(undici) `http`

/`https`

`setTimeout(0)`

on `connect`

to fail fast only when unreachable`externalResolver: true`