Most LLM failures are easy to describe and surprisingly hard to reproduce.
A user reports that the model returned an empty answer. A tool call disappeared halfway through a stream. One provider rejected a request that worked everywhere else.
Then I open the logs and find something like this:
LLM request failed: 400 Bad Request
Technically true. Operationally useless.
The missing piece is usually the exact request shape: model, parameters, message roles, tool definitions, timeout behavior, and the raw provider response.
I wanted something smaller than a full observability platform, so I built a request recorder around fetch
. It stores enough information to inspect or replay a failed call without logging the API key.
For each request, I want:
I deliberately do not record the Authorization
header.
Prompt content is also redacted by default. Full payload capture must be enabled explicitly because storing production prompts can create a much worse problem than the bug being investigated.
This example runs on Node.js 18 or newer and has no external dependencies.
Create recorded-fetch.mjs
:
import { randomUUID } from "node:crypto";
import { mkdir, writeFile } from "node:fs/promises";
import path from "node:path";
function sanitize(value, captureContent) {
if (Array.isArray(value)) {
return value.map((item) => sanitize(item, captureContent));
}
if (!value || typeof value !== "object") {
return value;
}
const result = {};
for (const [key, child] of Object.entries(value)) {
const normalizedKey = key.toLowerCase();
if (
normalizedKey.includes("api_key") ||
normalizedKey.includes("apikey") ||
normalizedKey.includes("authorization")
) {
result[key] = "[REDACTED]";
continue;
}
if (
!captureContent &&
(normalizedKey === "content" ||
normalizedKey === "prompt" ||
normalizedKey === "input")
) {
result[key] = "[REDACTED]";
continue;
}
result[key] = sanitize(child, captureContent);
}
return result;
}
async function saveRecording(directory, recording) {
await mkdir(directory, { recursive: true });
const filename = `${recording.startedAt.replaceAll(":", "-")}-${recording.id}.json`;
const destination = path.join(directory, filename);
await writeFile(
destination,
JSON.stringify(recording, null, 2),
"utf8"
);
return destination;
}
export function createRecordedFetch({
directory = ".llm-recordings",
captureContent = false,
} = {}) {
return async function recordedFetch(url, options = {}) {
const id = randomUUID();
const startedAt = new Date().toISOString();
const start = performance.now();
let requestBody;
try {
requestBody =
typeof options.body === "string"
? JSON.parse(options.body)
: options.body;
} catch {
requestBody = "[UNPARSEABLE BODY]";
}
const recording = {
id,
startedAt,
request: {
url: String(url),
method: options.method ?? "GET",
body: sanitize(requestBody, captureContent),
},
};
try {
const response = await fetch(url, options);
const rawResponse = await response.text();
recording.durationMs = Math.round(performance.now() - start);
recording.response = {
status: response.status,
statusText: response.statusText,
headers: Object.fromEntries(response.headers.entries()),
body: rawResponse,
};
const savedTo = await saveRecording(directory, recording);
if (!response.ok) {
throw new Error(
`LLM request ${id} failed with ${response.status}. Recording: ${savedTo}`
);
}
return {
requestId: id,
response,
rawResponse,
savedTo,
};
} catch (error) {
if (!recording.response) {
recording.durationMs = Math.round(performance.now() - start);
recording.error = {
name: error.name,
message: error.message,
};
await saveRecording(directory, recording);
}
throw error;
}
};
}
This wrapper reads the response body once and returns it as rawResponse
. That is important because a fetch response body cannot normally be consumed twice.
Create example.mjs
:
import { createRecordedFetch } from "./recorded-fetch.mjs";
const recordedFetch = createRecordedFetch({
directory: ".llm-recordings",
captureContent: false,
});
const baseUrl =
process.env.LLM_BASE_URL ?? "https://api.openai.com/v1";
const apiKey = process.env.LLM_API_KEY;
if (!apiKey) {
throw new Error("LLM_API_KEY is required");
}
const payload = {
model: process.env.LLM_MODEL ?? "gpt-4.1-mini",
messages: [
{
role: "user",
content: "Explain idempotency in one paragraph.",
},
],
temperature: 0.2,
};
const result = await recordedFetch(
`${baseUrl}/chat/completions`,
{
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(payload),
signal: AbortSignal.timeout(30_000),
}
);
const data = JSON.parse(result.rawResponse);
console.log({
requestId: result.requestId,
recording: result.savedTo,
answer: data.choices?.[0]?.message?.content,
});
Run it with:
LLM_API_KEY="your-api-key" node example.mjs
Every call now produces a JSON file under .llm-recordings
.
A failed request might look like this:
{
"id": "9e50d5a9-4a0d-42a2-94c9-711d36b2057d",
"startedAt": "2026-07-17T08:14:32.442Z",
"request": {
"url": "https://api.example.com/v1/chat/completions",
"method": "POST",
"body": {
"model": "example-model",
"messages": [
{
"role": "user",
"content": "[REDACTED]"
}
],
"temperature": 0.2
}
},
"durationMs": 418,
"response": {
"status": 400,
"statusText": "Bad Request",
"headers": {
"content-type": "application/json"
},
"body": "{\"error\":{\"message\":\"Unsupported parameter: temperature\"}}"
}
}
That is already more useful than a generic 400
.
It tells me the failure was probably caused by a provider compatibility difference, not the prompt, network, or model output.
For local or staging environments, I can temporarily set:
const recordedFetch = createRecordedFetch({
captureContent: true,
});
The resulting request body can then be replayed with a small script.
Create replay.mjs
:
import { readFile } from "node:fs/promises";
const recordingPath = process.argv[2];
if (!recordingPath) {
throw new Error("Usage: node replay.mjs <recording.json>");
}
const recording = JSON.parse(
await readFile(recordingPath, "utf8")
);
const serializedBody = JSON.stringify(recording.request.body);
if (serializedBody.includes("[REDACTED]")) {
throw new Error(
"This recording contains redacted values and cannot be replayed."
);
}
const apiKey = process.env.LLM_API_KEY;
if (!apiKey) {
throw new Error("LLM_API_KEY is required");
}
const response = await fetch(recording.request.url, {
method: recording.request.method,
headers: {
"Authorization": `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: serializedBody,
signal: AbortSignal.timeout(30_000),
});
console.log("Status:", response.status);
console.log(await response.text());
Then run:
LLM_API_KEY="your-api-key" \
node replay.mjs .llm-recordings/example.json
I only enable full-content recording with synthetic or approved test data. I would not turn it on globally in production.
The recorder is most useful when several failures look identical from the outside.
It helps separate:
Unsupported parameters, invalid tool schemas, wrong message roles, or model-specific limits.
Timeouts, connection resets, interrupted streams, and responses that never reach the application.
A structured error response, unexpected content type, or different interpretation of an OpenAI-compatible field.
The provider returned a valid response, but my parser, state machine, or tool executor mishandled it.
Those categories lead to very different fixes. Retrying all four of them is not a debugging strategy.
This recorder is intentionally small, so it leaves several decisions to the application using it.
Before running something like this in production, I would add:
I would also record failed calls more aggressively than successful ones. Keeping every response forever is an expensive way to create a privacy problem.
There are good observability products that can collect richer traces, token usage, latency distributions, and model-level metrics.
I still like having a tiny recorder close to the HTTP boundary.
It gives me a provider-neutral artifact I can inspect before data passes through SDK abstractions, response parsers, retry policies, or agent frameworks.
When I test multiple OpenAI-compatible endpoints—including my work on TokenBay—that raw boundary is often where compatibility problems become easiest to see.
The goal is not to log everything.
It is to make the next production failure reproducible enough that I do not have to debug it from a single error message.