{"slug": "a-small-node-js-wrapper-for-llm-api-retries-timeouts-and-logging", "title": "A Small Node.js Wrapper for LLM API Retries, Timeouts, and Logging", "summary": "A developer created a small Node.js wrapper for LLM API calls that handles retries, timeouts, and logging. The wrapper uses fetch and provides exponential backoff with jitter, retry logic for specific HTTP status codes and abort errors, and structured logging. It is designed to be a lightweight alternative to full observability platforms for production LLM integrations.", "body_md": "Most LLM API integrations start with a direct SDK call.\n\nThat is fine for a demo.\n\nBut once the call is inside a real product, I usually want three things around it:\n\nNot a giant framework. Not a full observability platform. Just a small wrapper that makes the boring production stuff harder to forget.\n\nHere is the Node.js wrapper pattern I usually reach for.\n\nA normal LLM call often starts like this:\n\n``` js\nconst response = await client.chat.completions.create({\n  model: \"gpt-4.1-mini\",\n  messages: [\n    { role: \"user\", content: \"Summarize this support ticket.\" }\n  ]\n});\n```\n\nThat is readable, but it hides a few production questions:\n\nThe wrapper below is intentionally small. It does not try to solve every LLM reliability problem.\n\nIt just gives each request a basic failure policy.\n\nThis version uses `fetch`\n\n, so it works on modern Node.js without extra dependencies.\n\n``` js\nconst DEFAULT_TIMEOUT_MS = 30_000;\n\nfunction sleep(ms) {\n  return new Promise((resolve) => setTimeout(resolve, ms));\n}\n\nfunction createTimeoutSignal(timeoutMs) {\n  const controller = new AbortController();\n  const timeout = setTimeout(() => controller.abort(), timeoutMs);\n\n  return {\n    signal: controller.signal,\n    clear: () => clearTimeout(timeout)\n  };\n}\n\nfunction shouldRetry({ status, error, attempt, maxRetries }) {\n  if (attempt >= maxRetries) return false;\n\n  if (error?.name === \"AbortError\") {\n    return true;\n  }\n\n  if (!status) {\n    return true;\n  }\n\n  return status === 429 || status === 500 || status === 502 || status === 503 || status === 504;\n}\n\nfunction retryDelayMs(attempt, retryAfterHeader) {\n  if (retryAfterHeader) {\n    const seconds = Number(retryAfterHeader);\n    if (Number.isFinite(seconds)) {\n      return seconds * 1000;\n    }\n  }\n\n  const base = 500 * 2 ** attempt;\n  const jitter = Math.floor(Math.random() * 250);\n\n  return base + jitter;\n}\n\nexport async function callLlmWithPolicy({\n  url,\n  apiKey,\n  body,\n  timeoutMs = DEFAULT_TIMEOUT_MS,\n  maxRetries = 2,\n  requestId = crypto.randomUUID(),\n  logger = console\n}) {\n  let lastError;\n\n  for (let attempt = 0; attempt <= maxRetries; attempt++) {\n    const startedAt = Date.now();\n    const timeout = createTimeoutSignal(timeoutMs);\n\n    try {\n      logger.info({\n        event: \"llm_request_started\",\n        requestId,\n        attempt,\n        model: body.model,\n        timeoutMs\n      });\n\n      const response = await fetch(url, {\n        method: \"POST\",\n        signal: timeout.signal,\n        headers: {\n          \"Authorization\": `Bearer ${apiKey}`,\n          \"Content-Type\": \"application/json\"\n        },\n        body: JSON.stringify(body)\n      });\n\n      const durationMs = Date.now() - startedAt;\n      const text = await response.text();\n\n      if (!response.ok) {\n        const retryAfter = response.headers.get(\"retry-after\");\n\n        logger.warn({\n          event: \"llm_request_failed\",\n          requestId,\n          attempt,\n          status: response.status,\n          durationMs,\n          retryAfter,\n          bodyPreview: text.slice(0, 500)\n        });\n\n        if (shouldRetry({\n          status: response.status,\n          attempt,\n          maxRetries\n        })) {\n          await sleep(retryDelayMs(attempt, retryAfter));\n          continue;\n        }\n\n        throw new Error(`LLM request failed with status ${response.status}: ${text}`);\n      }\n\n      logger.info({\n        event: \"llm_request_succeeded\",\n        requestId,\n        attempt,\n        status: response.status,\n        durationMs\n      });\n\n      return JSON.parse(text);\n    } catch (error) {\n      const durationMs = Date.now() - startedAt;\n      lastError = error;\n\n      logger.warn({\n        event: \"llm_request_error\",\n        requestId,\n        attempt,\n        errorName: error.name,\n        errorMessage: error.message,\n        durationMs\n      });\n\n      if (shouldRetry({\n        error,\n        attempt,\n        maxRetries\n      })) {\n        await sleep(retryDelayMs(attempt));\n        continue;\n      }\n\n      throw error;\n    } finally {\n      timeout.clear();\n    }\n  }\n\n  throw lastError;\n}\n```\n\nHere is a basic chat completion request:\n\n``` js\nimport { callLlmWithPolicy } from \"./llm-wrapper.js\";\n\nconst result = await callLlmWithPolicy({\n  url: \"https://api.openai.com/v1/chat/completions\",\n  apiKey: process.env.OPENAI_API_KEY,\n  body: {\n    model: \"gpt-4.1-mini\",\n    messages: [\n      {\n        role: \"user\",\n        content: \"Write a one-paragraph summary of this incident report.\"\n      }\n    ]\n  },\n  timeoutMs: 20_000,\n  maxRetries: 2\n});\n\nconsole.log(result.choices[0].message.content);\n```\n\nIf you are using an OpenAI-compatible provider, the shape stays almost the same. The main change is usually the base URL and model name.\n\nThat is one reason I like keeping this wrapper close to the HTTP boundary instead of burying all of the logic inside application code.\n\nThe logs are deliberately simple.\n\nOn start:\n\n```\n{\n  event: \"llm_request_started\",\n  requestId: \"01...\",\n  attempt: 0,\n  model: \"gpt-4.1-mini\",\n  timeoutMs: 20000\n}\n```\n\nOn failure:\n\n```\n{\n  event: \"llm_request_failed\",\n  requestId: \"01...\",\n  attempt: 0,\n  status: 429,\n  durationMs: 813,\n  retryAfter: \"2\"\n}\n```\n\nOn success:\n\n```\n{\n  event: \"llm_request_succeeded\",\n  requestId: \"01...\",\n  attempt: 1,\n  status: 200,\n  durationMs: 1420\n}\n```\n\nThat is enough to answer a few important questions later:\n\n`Retry-After`\n\n?For a lot of production debugging, that is already much better than “the AI call failed.”\n\nI would be careful about making retries automatic for every LLM call.\n\nSome calls are safe to repeat.\n\nFor example:\n\nOther calls may not be safe to repeat.\n\nFor example:\n\nThe wrapper cannot know that from the HTTP status code.\n\nSo in real apps, I usually add one more input:\n\n```\nretryMode: \"safe\" | \"unsafe\"\n```\n\nThen the retry rule becomes stricter:\n\n```\nfunction shouldRetry({ status, error, attempt, maxRetries, retryMode }) {\n  if (attempt >= maxRetries) return false;\n\n  if (retryMode === \"unsafe\") {\n    return false;\n  }\n\n  if (error?.name === \"AbortError\") {\n    return true;\n  }\n\n  if (!status) {\n    return true;\n  }\n\n  return status === 429 || status === 500 || status === 502 || status === 503 || status === 504;\n}\n```\n\nThat small distinction matters.\n\nA retry on a read-only summarization call is boring.\n\nA retry after an agent may have already triggered an external action is a completely different failure mode.\n\nI usually avoid one global timeout for every LLM call.\n\nDifferent workflows have different tolerance.\n\nFor example:\n\n``` js\nconst timeoutByWorkflow = {\n  autocomplete: 5_000,\n  support_summary: 20_000,\n  background_analysis: 90_000\n};\n```\n\nA user-facing autocomplete should probably fail fast.\n\nA background analysis job can wait longer.\n\nA support summary might sit somewhere in the middle.\n\nThe point is not to find the perfect timeout. The point is to make timeout behavior intentional instead of accidental.\n\nIt is tempting to keep adding features:\n\nThose can all be useful.\n\nBut I like starting with the boring layer first:\n\n`Retry-After`\n\nThat is the part you want in place before the first incident, not after.\n\nA small LLM wrapper will not make your app production-ready by itself.\n\nBut it does create a place for production policy to live.\n\nWithout that layer, retry behavior gets scattered across route handlers, background jobs, SDK calls, and agent tools. Eventually, nobody knows which LLM calls are safe to retry and which ones are quietly dangerous.\n\nMy rule now is simple:\n\nIf an LLM call matters to the product, it should not be a naked SDK call.\n\nPut a small policy around it.\n\nYour future debugging session will be shorter.\n\nI work on [TokenBay](https://www.tokenbay.com?utm_source=devto&utm_medium=community_content&utm_campaign=week1_free_content), so I spend a lot of time thinking about OpenAI-compatible APIs, multi-model routing, retries, and failure handling. The wrapper above is intentionally provider-agnostic: the same basic shape works whether you call one provider directly or route through a compatible API gateway.", "url": "https://wpnews.pro/news/a-small-node-js-wrapper-for-llm-api-retries-timeouts-and-logging", "canonical_source": "https://dev.to/plasma_01/a-small-nodejs-wrapper-for-llm-api-retries-timeouts-and-logging-ff4", "published_at": "2026-07-07 08:25:13+00:00", "updated_at": "2026-07-07 08:28:18.354153+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models", "machine-learning"], "entities": ["Node.js", "OpenAI", "GPT-4.1"], "alternates": {"html": "https://wpnews.pro/news/a-small-node-js-wrapper-for-llm-api-retries-timeouts-and-logging", "markdown": "https://wpnews.pro/news/a-small-node-js-wrapper-for-llm-api-retries-timeouts-and-logging.md", "text": "https://wpnews.pro/news/a-small-node-js-wrapper-for-llm-api-retries-timeouts-and-logging.txt", "jsonld": "https://wpnews.pro/news/a-small-node-js-wrapper-for-llm-api-retries-timeouts-and-logging.jsonld"}}