# A Tiny LLM Request Recorder I Use to Reproduce Production Failures

> Source: <https://dev.to/plasma_01/a-tiny-llm-request-recorder-i-use-to-reproduce-production-failures-kna>
> Published: 2026-07-17 03:21:00+00:00

Most LLM failures are easy to describe and surprisingly hard to reproduce.

A user reports that the model returned an empty answer. A tool call disappeared halfway through a stream. One provider rejected a request that worked everywhere else.

Then I open the logs and find something like this:

```
LLM request failed: 400 Bad Request
```

Technically true. Operationally useless.

The missing piece is usually the exact request shape: model, parameters, message roles, tool definitions, timeout behavior, and the raw provider response.

I wanted something smaller than a full observability platform, so I built a request recorder around `fetch`

. It stores enough information to inspect or replay a failed call without logging the API key.

For each request, I want:

I deliberately do not record the `Authorization`

header.

Prompt content is also redacted by default. Full payload capture must be enabled explicitly because storing production prompts can create a much worse problem than the bug being investigated.

This example runs on Node.js 18 or newer and has no external dependencies.

Create `recorded-fetch.mjs`

:

``` js
import { randomUUID } from "node:crypto";
import { mkdir, writeFile } from "node:fs/promises";
import path from "node:path";

function sanitize(value, captureContent) {
  if (Array.isArray(value)) {
    return value.map((item) => sanitize(item, captureContent));
  }

  if (!value || typeof value !== "object") {
    return value;
  }

  const result = {};

  for (const [key, child] of Object.entries(value)) {
    const normalizedKey = key.toLowerCase();

    if (
      normalizedKey.includes("api_key") ||
      normalizedKey.includes("apikey") ||
      normalizedKey.includes("authorization")
    ) {
      result[key] = "[REDACTED]";
      continue;
    }

    if (
      !captureContent &&
      (normalizedKey === "content" ||
        normalizedKey === "prompt" ||
        normalizedKey === "input")
    ) {
      result[key] = "[REDACTED]";
      continue;
    }

    result[key] = sanitize(child, captureContent);
  }

  return result;
}

async function saveRecording(directory, recording) {
  await mkdir(directory, { recursive: true });

  const filename = `${recording.startedAt.replaceAll(":", "-")}-${recording.id}.json`;
  const destination = path.join(directory, filename);

  await writeFile(
    destination,
    JSON.stringify(recording, null, 2),
    "utf8"
  );

  return destination;
}

export function createRecordedFetch({
  directory = ".llm-recordings",
  captureContent = false,
} = {}) {
  return async function recordedFetch(url, options = {}) {
    const id = randomUUID();
    const startedAt = new Date().toISOString();
    const start = performance.now();

    let requestBody;

    try {
      requestBody =
        typeof options.body === "string"
          ? JSON.parse(options.body)
          : options.body;
    } catch {
      requestBody = "[UNPARSEABLE BODY]";
    }

    const recording = {
      id,
      startedAt,
      request: {
        url: String(url),
        method: options.method ?? "GET",
        body: sanitize(requestBody, captureContent),
      },
    };

    try {
      const response = await fetch(url, options);
      const rawResponse = await response.text();

      recording.durationMs = Math.round(performance.now() - start);
      recording.response = {
        status: response.status,
        statusText: response.statusText,
        headers: Object.fromEntries(response.headers.entries()),
        body: rawResponse,
      };

      const savedTo = await saveRecording(directory, recording);

      if (!response.ok) {
        throw new Error(
          `LLM request ${id} failed with ${response.status}. Recording: ${savedTo}`
        );
      }

      return {
        requestId: id,
        response,
        rawResponse,
        savedTo,
      };
    } catch (error) {
      if (!recording.response) {
        recording.durationMs = Math.round(performance.now() - start);
        recording.error = {
          name: error.name,
          message: error.message,
        };

        await saveRecording(directory, recording);
      }

      throw error;
    }
  };
}
```

This wrapper reads the response body once and returns it as `rawResponse`

. That is important because a fetch response body cannot normally be consumed twice.

Create `example.mjs`

:

``` js
import { createRecordedFetch } from "./recorded-fetch.mjs";

const recordedFetch = createRecordedFetch({
  directory: ".llm-recordings",
  captureContent: false,
});

const baseUrl =
  process.env.LLM_BASE_URL ?? "https://api.openai.com/v1";

const apiKey = process.env.LLM_API_KEY;

if (!apiKey) {
  throw new Error("LLM_API_KEY is required");
}

const payload = {
  model: process.env.LLM_MODEL ?? "gpt-4.1-mini",
  messages: [
    {
      role: "user",
      content: "Explain idempotency in one paragraph.",
    },
  ],
  temperature: 0.2,
};

const result = await recordedFetch(
  `${baseUrl}/chat/completions`,
  {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(payload),
    signal: AbortSignal.timeout(30_000),
  }
);

const data = JSON.parse(result.rawResponse);

console.log({
  requestId: result.requestId,
  recording: result.savedTo,
  answer: data.choices?.[0]?.message?.content,
});
```

Run it with:

```
LLM_API_KEY="your-api-key" node example.mjs
```

Every call now produces a JSON file under `.llm-recordings`

.

A failed request might look like this:

```
{
  "id": "9e50d5a9-4a0d-42a2-94c9-711d36b2057d",
  "startedAt": "2026-07-17T08:14:32.442Z",
  "request": {
    "url": "https://api.example.com/v1/chat/completions",
    "method": "POST",
    "body": {
      "model": "example-model",
      "messages": [
        {
          "role": "user",
          "content": "[REDACTED]"
        }
      ],
      "temperature": 0.2
    }
  },
  "durationMs": 418,
  "response": {
    "status": 400,
    "statusText": "Bad Request",
    "headers": {
      "content-type": "application/json"
    },
    "body": "{\"error\":{\"message\":\"Unsupported parameter: temperature\"}}"
  }
}
```

That is already more useful than a generic `400`

.

It tells me the failure was probably caused by a provider compatibility difference, not the prompt, network, or model output.

For local or staging environments, I can temporarily set:

``` js
const recordedFetch = createRecordedFetch({
  captureContent: true,
});
```

The resulting request body can then be replayed with a small script.

Create `replay.mjs`

:

``` js
import { readFile } from "node:fs/promises";

const recordingPath = process.argv[2];

if (!recordingPath) {
  throw new Error("Usage: node replay.mjs <recording.json>");
}

const recording = JSON.parse(
  await readFile(recordingPath, "utf8")
);

const serializedBody = JSON.stringify(recording.request.body);

if (serializedBody.includes("[REDACTED]")) {
  throw new Error(
    "This recording contains redacted values and cannot be replayed."
  );
}

const apiKey = process.env.LLM_API_KEY;

if (!apiKey) {
  throw new Error("LLM_API_KEY is required");
}

const response = await fetch(recording.request.url, {
  method: recording.request.method,
  headers: {
    "Authorization": `Bearer ${apiKey}`,
    "Content-Type": "application/json",
  },
  body: serializedBody,
  signal: AbortSignal.timeout(30_000),
});

console.log("Status:", response.status);
console.log(await response.text());
```

Then run:

```
LLM_API_KEY="your-api-key" \
node replay.mjs .llm-recordings/example.json
```

I only enable full-content recording with synthetic or approved test data. I would not turn it on globally in production.

The recorder is most useful when several failures look identical from the outside.

It helps separate:

Unsupported parameters, invalid tool schemas, wrong message roles, or model-specific limits.

Timeouts, connection resets, interrupted streams, and responses that never reach the application.

A structured error response, unexpected content type, or different interpretation of an OpenAI-compatible field.

The provider returned a valid response, but my parser, state machine, or tool executor mishandled it.

Those categories lead to very different fixes. Retrying all four of them is not a debugging strategy.

This recorder is intentionally small, so it leaves several decisions to the application using it.

Before running something like this in production, I would add:

I would also record failed calls more aggressively than successful ones. Keeping every response forever is an expensive way to create a privacy problem.

There are good observability products that can collect richer traces, token usage, latency distributions, and model-level metrics.

I still like having a tiny recorder close to the HTTP boundary.

It gives me a provider-neutral artifact I can inspect before data passes through SDK abstractions, response parsers, retry policies, or agent frameworks.

When I test multiple OpenAI-compatible endpoints—including my work on [TokenBay](https://www.tokenbay.com?utm_source=devto&utm_medium=community_content&utm_campaign=week1_free_content)—that raw boundary is often where compatibility problems become easiest to see.

The goal is not to log everything.

It is to make the next production failure reproducible enough that I do not have to debug it from a single error message.