A Small Node.js Wrapper for LLM API Retries, Timeouts, and Logging

A developer created a small Node.js wrapper for LLM API calls that handles retries, timeouts, and logging. The wrapper uses fetch and provides exponential backoff with jitter, retry logic for specific HTTP status codes and abort errors, and structured logging. It is designed to be a lightweight alternative to full observability platforms for production LLM integrations.

Most LLM API integrations start with a direct SDK call. That is fine for a demo. But once the call is inside a real product, I usually want three things around it: Not a giant framework. Not a full observability platform. Just a small wrapper that makes the boring production stuff harder to forget. Here is the Node.js wrapper pattern I usually reach for. A normal LLM call often starts like this: js const response = await client.chat.completions.create { model: "gpt-4.1-mini", messages: { role: "user", content: "Summarize this support ticket." } } ; That is readable, but it hides a few production questions: The wrapper below is intentionally small. It does not try to solve every LLM reliability problem. It just gives each request a basic failure policy. This version uses fetch , so it works on modern Node.js without extra dependencies. js const DEFAULT TIMEOUT MS = 30 000; function sleep ms { return new Promise resolve = setTimeout resolve, ms ; } function createTimeoutSignal timeoutMs { const controller = new AbortController ; const timeout = setTimeout = controller.abort , timeoutMs ; return { signal: controller.signal, clear: = clearTimeout timeout }; } function shouldRetry { status, error, attempt, maxRetries } { if attempt = maxRetries return false; if error?.name === "AbortError" { return true; } if status { return true; } return status === 429 || status === 500 || status === 502 || status === 503 || status === 504; } function retryDelayMs attempt, retryAfterHeader { if retryAfterHeader { const seconds = Number retryAfterHeader ; if Number.isFinite seconds { return seconds 1000; } } const base = 500 2 attempt; const jitter = Math.floor Math.random 250 ; return base + jitter; } export async function callLlmWithPolicy { url, apiKey, body, timeoutMs = DEFAULT TIMEOUT MS, maxRetries = 2, requestId = crypto.randomUUID , logger = console } { let lastError; for let attempt = 0; attempt <= maxRetries; attempt++ { const startedAt = Date.now ; const timeout = createTimeoutSignal timeoutMs ; try { logger.info { event: "llm request started", requestId, attempt, model: body.model, timeoutMs } ; const response = await fetch url, { method: "POST", signal: timeout.signal, headers: { "Authorization": Bearer ${apiKey} , "Content-Type": "application/json" }, body: JSON.stringify body } ; const durationMs = Date.now - startedAt; const text = await response.text ; if response.ok { const retryAfter = response.headers.get "retry-after" ; logger.warn { event: "llm request failed", requestId, attempt, status: response.status, durationMs, retryAfter, bodyPreview: text.slice 0, 500 } ; if shouldRetry { status: response.status, attempt, maxRetries } { await sleep retryDelayMs attempt, retryAfter ; continue; } throw new Error LLM request failed with status ${response.status}: ${text} ; } logger.info { event: "llm request succeeded", requestId, attempt, status: response.status, durationMs } ; return JSON.parse text ; } catch error { const durationMs = Date.now - startedAt; lastError = error; logger.warn { event: "llm request error", requestId, attempt, errorName: error.name, errorMessage: error.message, durationMs } ; if shouldRetry { error, attempt, maxRetries } { await sleep retryDelayMs attempt ; continue; } throw error; } finally { timeout.clear ; } } throw lastError; } Here is a basic chat completion request: js import { callLlmWithPolicy } from "./llm-wrapper.js"; const result = await callLlmWithPolicy { url: "https://api.openai.com/v1/chat/completions", apiKey: process.env.OPENAI API KEY, body: { model: "gpt-4.1-mini", messages: { role: "user", content: "Write a one-paragraph summary of this incident report." } }, timeoutMs: 20 000, maxRetries: 2 } ; console.log result.choices 0 .message.content ; If you are using an OpenAI-compatible provider, the shape stays almost the same. The main change is usually the base URL and model name. That is one reason I like keeping this wrapper close to the HTTP boundary instead of burying all of the logic inside application code. The logs are deliberately simple. On start: { event: "llm request started", requestId: "01...", attempt: 0, model: "gpt-4.1-mini", timeoutMs: 20000 } On failure: { event: "llm request failed", requestId: "01...", attempt: 0, status: 429, durationMs: 813, retryAfter: "2" } On success: { event: "llm request succeeded", requestId: "01...", attempt: 1, status: 200, durationMs: 1420 } That is enough to answer a few important questions later: Retry-After ?For a lot of production debugging, that is already much better than “the AI call failed.” I would be careful about making retries automatic for every LLM call. Some calls are safe to repeat. For example: Other calls may not be safe to repeat. For example: The wrapper cannot know that from the HTTP status code. So in real apps, I usually add one more input: retryMode: "safe" | "unsafe" Then the retry rule becomes stricter: function shouldRetry { status, error, attempt, maxRetries, retryMode } { if attempt = maxRetries return false; if retryMode === "unsafe" { return false; } if error?.name === "AbortError" { return true; } if status { return true; } return status === 429 || status === 500 || status === 502 || status === 503 || status === 504; } That small distinction matters. A retry on a read-only summarization call is boring. A retry after an agent may have already triggered an external action is a completely different failure mode. I usually avoid one global timeout for every LLM call. Different workflows have different tolerance. For example: js const timeoutByWorkflow = { autocomplete: 5 000, support summary: 20 000, background analysis: 90 000 }; A user-facing autocomplete should probably fail fast. A background analysis job can wait longer. A support summary might sit somewhere in the middle. The point is not to find the perfect timeout. The point is to make timeout behavior intentional instead of accidental. It is tempting to keep adding features: Those can all be useful. But I like starting with the boring layer first: Retry-After That is the part you want in place before the first incident, not after. A small LLM wrapper will not make your app production-ready by itself. But it does create a place for production policy to live. Without that layer, retry behavior gets scattered across route handlers, background jobs, SDK calls, and agent tools. Eventually, nobody knows which LLM calls are safe to retry and which ones are quietly dangerous. My rule now is simple: If an LLM call matters to the product, it should not be a naked SDK call. Put a small policy around it. Your future debugging session will be shorter. I work on TokenBay https://www.tokenbay.com?utm source=devto&utm medium=community content&utm campaign=week1 free content , so I spend a lot of time thinking about OpenAI-compatible APIs, multi-model routing, retries, and failure handling. The wrapper above is intentionally provider-agnostic: the same basic shape works whether you call one provider directly or route through a compatible API gateway.