A Tiny LLM Request Recorder I Use to Reproduce Production Failures

A developer built a lightweight LLM request recorder around the fetch API to help reproduce production failures. The tool stores request details like model, parameters, and provider responses while redacting API keys and prompt content by default. It runs on Node.js 18+ with no external dependencies.

Most LLM failures are easy to describe and surprisingly hard to reproduce. A user reports that the model returned an empty answer. A tool call disappeared halfway through a stream. One provider rejected a request that worked everywhere else. Then I open the logs and find something like this: LLM request failed: 400 Bad Request Technically true. Operationally useless. The missing piece is usually the exact request shape: model, parameters, message roles, tool definitions, timeout behavior, and the raw provider response. I wanted something smaller than a full observability platform, so I built a request recorder around fetch . It stores enough information to inspect or replay a failed call without logging the API key. For each request, I want: I deliberately do not record the Authorization header. Prompt content is also redacted by default. Full payload capture must be enabled explicitly because storing production prompts can create a much worse problem than the bug being investigated. This example runs on Node.js 18 or newer and has no external dependencies. Create recorded-fetch.mjs : js import { randomUUID } from "node:crypto"; import { mkdir, writeFile } from "node:fs/promises"; import path from "node:path"; function sanitize value, captureContent { if Array.isArray value { return value.map item = sanitize item, captureContent ; } if value || typeof value == "object" { return value; } const result = {}; for const key, child of Object.entries value { const normalizedKey = key.toLowerCase ; if normalizedKey.includes "api key" || normalizedKey.includes "apikey" || normalizedKey.includes "authorization" { result key = " REDACTED "; continue; } if captureContent && normalizedKey === "content" || normalizedKey === "prompt" || normalizedKey === "input" { result key = " REDACTED "; continue; } result key = sanitize child, captureContent ; } return result; } async function saveRecording directory, recording { await mkdir directory, { recursive: true } ; const filename = ${recording.startedAt.replaceAll ":", "-" }-${recording.id}.json ; const destination = path.join directory, filename ; await writeFile destination, JSON.stringify recording, null, 2 , "utf8" ; return destination; } export function createRecordedFetch { directory = ".llm-recordings", captureContent = false, } = {} { return async function recordedFetch url, options = {} { const id = randomUUID ; const startedAt = new Date .toISOString ; const start = performance.now ; let requestBody; try { requestBody = typeof options.body === "string" ? JSON.parse options.body : options.body; } catch { requestBody = " UNPARSEABLE BODY "; } const recording = { id, startedAt, request: { url: String url , method: options.method ?? "GET", body: sanitize requestBody, captureContent , }, }; try { const response = await fetch url, options ; const rawResponse = await response.text ; recording.durationMs = Math.round performance.now - start ; recording.response = { status: response.status, statusText: response.statusText, headers: Object.fromEntries response.headers.entries , body: rawResponse, }; const savedTo = await saveRecording directory, recording ; if response.ok { throw new Error LLM request ${id} failed with ${response.status}. Recording: ${savedTo} ; } return { requestId: id, response, rawResponse, savedTo, }; } catch error { if recording.response { recording.durationMs = Math.round performance.now - start ; recording.error = { name: error.name, message: error.message, }; await saveRecording directory, recording ; } throw error; } }; } This wrapper reads the response body once and returns it as rawResponse . That is important because a fetch response body cannot normally be consumed twice. Create example.mjs : js import { createRecordedFetch } from "./recorded-fetch.mjs"; const recordedFetch = createRecordedFetch { directory: ".llm-recordings", captureContent: false, } ; const baseUrl = process.env.LLM BASE URL ?? "https://api.openai.com/v1"; const apiKey = process.env.LLM API KEY; if apiKey { throw new Error "LLM API KEY is required" ; } const payload = { model: process.env.LLM MODEL ?? "gpt-4.1-mini", messages: { role: "user", content: "Explain idempotency in one paragraph.", }, , temperature: 0.2, }; const result = await recordedFetch ${baseUrl}/chat/completions , { method: "POST", headers: { "Authorization": Bearer ${apiKey} , "Content-Type": "application/json", }, body: JSON.stringify payload , signal: AbortSignal.timeout 30 000 , } ; const data = JSON.parse result.rawResponse ; console.log { requestId: result.requestId, recording: result.savedTo, answer: data.choices?. 0 ?.message?.content, } ; Run it with: LLM API KEY="your-api-key" node example.mjs Every call now produces a JSON file under .llm-recordings . A failed request might look like this: { "id": "9e50d5a9-4a0d-42a2-94c9-711d36b2057d", "startedAt": "2026-07-17T08:14:32.442Z", "request": { "url": "https://api.example.com/v1/chat/completions", "method": "POST", "body": { "model": "example-model", "messages": { "role": "user", "content": " REDACTED " } , "temperature": 0.2 } }, "durationMs": 418, "response": { "status": 400, "statusText": "Bad Request", "headers": { "content-type": "application/json" }, "body": "{\"error\":{\"message\":\"Unsupported parameter: temperature\"}}" } } That is already more useful than a generic 400 . It tells me the failure was probably caused by a provider compatibility difference, not the prompt, network, or model output. For local or staging environments, I can temporarily set: js const recordedFetch = createRecordedFetch { captureContent: true, } ; The resulting request body can then be replayed with a small script. Create replay.mjs : js import { readFile } from "node:fs/promises"; const recordingPath = process.argv 2 ; if recordingPath { throw new Error "Usage: node replay.mjs <recording.json " ; } const recording = JSON.parse await readFile recordingPath, "utf8" ; const serializedBody = JSON.stringify recording.request.body ; if serializedBody.includes " REDACTED " { throw new Error "This recording contains redacted values and cannot be replayed." ; } const apiKey = process.env.LLM API KEY; if apiKey { throw new Error "LLM API KEY is required" ; } const response = await fetch recording.request.url, { method: recording.request.method, headers: { "Authorization": Bearer ${apiKey} , "Content-Type": "application/json", }, body: serializedBody, signal: AbortSignal.timeout 30 000 , } ; console.log "Status:", response.status ; console.log await response.text ; Then run: LLM API KEY="your-api-key" \ node replay.mjs .llm-recordings/example.json I only enable full-content recording with synthetic or approved test data. I would not turn it on globally in production. The recorder is most useful when several failures look identical from the outside. It helps separate: Unsupported parameters, invalid tool schemas, wrong message roles, or model-specific limits. Timeouts, connection resets, interrupted streams, and responses that never reach the application. A structured error response, unexpected content type, or different interpretation of an OpenAI-compatible field. The provider returned a valid response, but my parser, state machine, or tool executor mishandled it. Those categories lead to very different fixes. Retrying all four of them is not a debugging strategy. This recorder is intentionally small, so it leaves several decisions to the application using it. Before running something like this in production, I would add: I would also record failed calls more aggressively than successful ones. Keeping every response forever is an expensive way to create a privacy problem. There are good observability products that can collect richer traces, token usage, latency distributions, and model-level metrics. I still like having a tiny recorder close to the HTTP boundary. It gives me a provider-neutral artifact I can inspect before data passes through SDK abstractions, response parsers, retry policies, or agent frameworks. When I test multiple OpenAI-compatible endpoints—including my work on TokenBay https://www.tokenbay.com?utm source=devto&utm medium=community content&utm campaign=week1 free content —that raw boundary is often where compatibility problems become easiest to see. The goal is not to log everything. It is to make the next production failure reproducible enough that I do not have to debug it from a single error message.