{"slug": "a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures", "title": "A Tiny LLM Request Recorder I Use to Reproduce Production Failures", "summary": "A developer built a lightweight LLM request recorder around the fetch API to help reproduce production failures. The tool stores request details like model, parameters, and provider responses while redacting API keys and prompt content by default. It runs on Node.js 18+ with no external dependencies.", "body_md": "Most LLM failures are easy to describe and surprisingly hard to reproduce.\n\nA user reports that the model returned an empty answer. A tool call disappeared halfway through a stream. One provider rejected a request that worked everywhere else.\n\nThen I open the logs and find something like this:\n\n```\nLLM request failed: 400 Bad Request\n```\n\nTechnically true. Operationally useless.\n\nThe missing piece is usually the exact request shape: model, parameters, message roles, tool definitions, timeout behavior, and the raw provider response.\n\nI wanted something smaller than a full observability platform, so I built a request recorder around `fetch`\n\n. It stores enough information to inspect or replay a failed call without logging the API key.\n\nFor each request, I want:\n\nI deliberately do not record the `Authorization`\n\nheader.\n\nPrompt content is also redacted by default. Full payload capture must be enabled explicitly because storing production prompts can create a much worse problem than the bug being investigated.\n\nThis example runs on Node.js 18 or newer and has no external dependencies.\n\nCreate `recorded-fetch.mjs`\n\n:\n\n``` js\nimport { randomUUID } from \"node:crypto\";\nimport { mkdir, writeFile } from \"node:fs/promises\";\nimport path from \"node:path\";\n\nfunction sanitize(value, captureContent) {\n  if (Array.isArray(value)) {\n    return value.map((item) => sanitize(item, captureContent));\n  }\n\n  if (!value || typeof value !== \"object\") {\n    return value;\n  }\n\n  const result = {};\n\n  for (const [key, child] of Object.entries(value)) {\n    const normalizedKey = key.toLowerCase();\n\n    if (\n      normalizedKey.includes(\"api_key\") ||\n      normalizedKey.includes(\"apikey\") ||\n      normalizedKey.includes(\"authorization\")\n    ) {\n      result[key] = \"[REDACTED]\";\n      continue;\n    }\n\n    if (\n      !captureContent &&\n      (normalizedKey === \"content\" ||\n        normalizedKey === \"prompt\" ||\n        normalizedKey === \"input\")\n    ) {\n      result[key] = \"[REDACTED]\";\n      continue;\n    }\n\n    result[key] = sanitize(child, captureContent);\n  }\n\n  return result;\n}\n\nasync function saveRecording(directory, recording) {\n  await mkdir(directory, { recursive: true });\n\n  const filename = `${recording.startedAt.replaceAll(\":\", \"-\")}-${recording.id}.json`;\n  const destination = path.join(directory, filename);\n\n  await writeFile(\n    destination,\n    JSON.stringify(recording, null, 2),\n    \"utf8\"\n  );\n\n  return destination;\n}\n\nexport function createRecordedFetch({\n  directory = \".llm-recordings\",\n  captureContent = false,\n} = {}) {\n  return async function recordedFetch(url, options = {}) {\n    const id = randomUUID();\n    const startedAt = new Date().toISOString();\n    const start = performance.now();\n\n    let requestBody;\n\n    try {\n      requestBody =\n        typeof options.body === \"string\"\n          ? JSON.parse(options.body)\n          : options.body;\n    } catch {\n      requestBody = \"[UNPARSEABLE BODY]\";\n    }\n\n    const recording = {\n      id,\n      startedAt,\n      request: {\n        url: String(url),\n        method: options.method ?? \"GET\",\n        body: sanitize(requestBody, captureContent),\n      },\n    };\n\n    try {\n      const response = await fetch(url, options);\n      const rawResponse = await response.text();\n\n      recording.durationMs = Math.round(performance.now() - start);\n      recording.response = {\n        status: response.status,\n        statusText: response.statusText,\n        headers: Object.fromEntries(response.headers.entries()),\n        body: rawResponse,\n      };\n\n      const savedTo = await saveRecording(directory, recording);\n\n      if (!response.ok) {\n        throw new Error(\n          `LLM request ${id} failed with ${response.status}. Recording: ${savedTo}`\n        );\n      }\n\n      return {\n        requestId: id,\n        response,\n        rawResponse,\n        savedTo,\n      };\n    } catch (error) {\n      if (!recording.response) {\n        recording.durationMs = Math.round(performance.now() - start);\n        recording.error = {\n          name: error.name,\n          message: error.message,\n        };\n\n        await saveRecording(directory, recording);\n      }\n\n      throw error;\n    }\n  };\n}\n```\n\nThis wrapper reads the response body once and returns it as `rawResponse`\n\n. That is important because a fetch response body cannot normally be consumed twice.\n\nCreate `example.mjs`\n\n:\n\n``` js\nimport { createRecordedFetch } from \"./recorded-fetch.mjs\";\n\nconst recordedFetch = createRecordedFetch({\n  directory: \".llm-recordings\",\n  captureContent: false,\n});\n\nconst baseUrl =\n  process.env.LLM_BASE_URL ?? \"https://api.openai.com/v1\";\n\nconst apiKey = process.env.LLM_API_KEY;\n\nif (!apiKey) {\n  throw new Error(\"LLM_API_KEY is required\");\n}\n\nconst payload = {\n  model: process.env.LLM_MODEL ?? \"gpt-4.1-mini\",\n  messages: [\n    {\n      role: \"user\",\n      content: \"Explain idempotency in one paragraph.\",\n    },\n  ],\n  temperature: 0.2,\n};\n\nconst result = await recordedFetch(\n  `${baseUrl}/chat/completions`,\n  {\n    method: \"POST\",\n    headers: {\n      \"Authorization\": `Bearer ${apiKey}`,\n      \"Content-Type\": \"application/json\",\n    },\n    body: JSON.stringify(payload),\n    signal: AbortSignal.timeout(30_000),\n  }\n);\n\nconst data = JSON.parse(result.rawResponse);\n\nconsole.log({\n  requestId: result.requestId,\n  recording: result.savedTo,\n  answer: data.choices?.[0]?.message?.content,\n});\n```\n\nRun it with:\n\n```\nLLM_API_KEY=\"your-api-key\" node example.mjs\n```\n\nEvery call now produces a JSON file under `.llm-recordings`\n\n.\n\nA failed request might look like this:\n\n```\n{\n  \"id\": \"9e50d5a9-4a0d-42a2-94c9-711d36b2057d\",\n  \"startedAt\": \"2026-07-17T08:14:32.442Z\",\n  \"request\": {\n    \"url\": \"https://api.example.com/v1/chat/completions\",\n    \"method\": \"POST\",\n    \"body\": {\n      \"model\": \"example-model\",\n      \"messages\": [\n        {\n          \"role\": \"user\",\n          \"content\": \"[REDACTED]\"\n        }\n      ],\n      \"temperature\": 0.2\n    }\n  },\n  \"durationMs\": 418,\n  \"response\": {\n    \"status\": 400,\n    \"statusText\": \"Bad Request\",\n    \"headers\": {\n      \"content-type\": \"application/json\"\n    },\n    \"body\": \"{\\\"error\\\":{\\\"message\\\":\\\"Unsupported parameter: temperature\\\"}}\"\n  }\n}\n```\n\nThat is already more useful than a generic `400`\n\n.\n\nIt tells me the failure was probably caused by a provider compatibility difference, not the prompt, network, or model output.\n\nFor local or staging environments, I can temporarily set:\n\n``` js\nconst recordedFetch = createRecordedFetch({\n  captureContent: true,\n});\n```\n\nThe resulting request body can then be replayed with a small script.\n\nCreate `replay.mjs`\n\n:\n\n``` js\nimport { readFile } from \"node:fs/promises\";\n\nconst recordingPath = process.argv[2];\n\nif (!recordingPath) {\n  throw new Error(\"Usage: node replay.mjs <recording.json>\");\n}\n\nconst recording = JSON.parse(\n  await readFile(recordingPath, \"utf8\")\n);\n\nconst serializedBody = JSON.stringify(recording.request.body);\n\nif (serializedBody.includes(\"[REDACTED]\")) {\n  throw new Error(\n    \"This recording contains redacted values and cannot be replayed.\"\n  );\n}\n\nconst apiKey = process.env.LLM_API_KEY;\n\nif (!apiKey) {\n  throw new Error(\"LLM_API_KEY is required\");\n}\n\nconst response = await fetch(recording.request.url, {\n  method: recording.request.method,\n  headers: {\n    \"Authorization\": `Bearer ${apiKey}`,\n    \"Content-Type\": \"application/json\",\n  },\n  body: serializedBody,\n  signal: AbortSignal.timeout(30_000),\n});\n\nconsole.log(\"Status:\", response.status);\nconsole.log(await response.text());\n```\n\nThen run:\n\n```\nLLM_API_KEY=\"your-api-key\" \\\nnode replay.mjs .llm-recordings/example.json\n```\n\nI only enable full-content recording with synthetic or approved test data. I would not turn it on globally in production.\n\nThe recorder is most useful when several failures look identical from the outside.\n\nIt helps separate:\n\nUnsupported parameters, invalid tool schemas, wrong message roles, or model-specific limits.\n\nTimeouts, connection resets, interrupted streams, and responses that never reach the application.\n\nA structured error response, unexpected content type, or different interpretation of an OpenAI-compatible field.\n\nThe provider returned a valid response, but my parser, state machine, or tool executor mishandled it.\n\nThose categories lead to very different fixes. Retrying all four of them is not a debugging strategy.\n\nThis recorder is intentionally small, so it leaves several decisions to the application using it.\n\nBefore running something like this in production, I would add:\n\nI would also record failed calls more aggressively than successful ones. Keeping every response forever is an expensive way to create a privacy problem.\n\nThere are good observability products that can collect richer traces, token usage, latency distributions, and model-level metrics.\n\nI still like having a tiny recorder close to the HTTP boundary.\n\nIt gives me a provider-neutral artifact I can inspect before data passes through SDK abstractions, response parsers, retry policies, or agent frameworks.\n\nWhen I test multiple OpenAI-compatible endpoints—including my work on [TokenBay](https://www.tokenbay.com?utm_source=devto&utm_medium=community_content&utm_campaign=week1_free_content)—that raw boundary is often where compatibility problems become easiest to see.\n\nThe goal is not to log everything.\n\nIt is to make the next production failure reproducible enough that I do not have to debug it from a single error message.", "url": "https://wpnews.pro/news/a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures", "canonical_source": "https://dev.to/plasma_01/a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures-kna", "published_at": "2026-07-17 03:21:00+00:00", "updated_at": "2026-07-17 03:28:33.255664+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models", "ai-infrastructure"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures", "markdown": "https://wpnews.pro/news/a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures.md", "text": "https://wpnews.pro/news/a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures.txt", "jsonld": "https://wpnews.pro/news/a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures.jsonld"}}