Moderation Report Triage: Node.js LLM API JSON Contracts for Portable Summaries

Fintech moderation reports should not enter a human-review queue as a blob of model prose. The portable design is a narrow JSON contract at the Node.js API boundary: title, summary, bullets, and action items, with an explicit result for uncertainty. Keep the model provider behind one adapter, then test that contract against several providers before choosing a default.

Short answer: define the JSON shape first, validate it after every LLM call, and make provider replacement an adapter change rather than an application rewrite. This is slower to sketch than response.text

, but much faster to operate once reports feed queues, dashboards, and audit records.

That is the choice matrix. The recommendation is the middle row for this scenario: a small adapter plus a strict internal schema. It gives a fintech review service one place to swap credentials, model names, timeouts, and response parsing. The adapter is not a magic compatibility layer. It is a deliberately boring boundary.

No prose parser.

Start with the reviewer, not the model. A reviewer needs a short title, a plain summary, evidence bullets, a moderation category, and tasks that can be assigned. The contract should also preserve the distinction between “the report contains no such fact” and “the classifier is unsure.” Those are different states.

type ModerationSummary = {
  title: string;
  summary: string;
  bullets: string[];
  category: "fraud" | "abuse" | "privacy" | "other" | "unclear";
  confidence: "high" | "medium" | "low";
  action_items: Array<{
    text: string;
    owner: string | null;
    due: string | null;
  }>;
};

const summarySchema = {
  type: "object",
  additionalProperties: false,
  required: ["title", "summary", "bullets", "category", "confidence", "action_items"],
  properties: {
    title: { type: "string", minLength: 1 },
    summary: { type: "string", minLength: 1 },
    bullets: { type: "array", items: { type: "string" } },
    category: {
      type: "string",
      enum: ["fraud", "abuse", "privacy", "other", "unclear"],
    },
    confidence: { type: "string", enum: ["high", "medium", "low"] },
    action_items: {
      type: "array",
      items: {
        type: "object",
        additionalProperties: false,
        required: ["text", "owner", "due"],
        properties: {
          text: { type: "string" },
          owner: { type: ["string", "null"] },
          due: { type: ["string", "null"] },
        },
      },
    },
  },
} as const;

null

is intentional. If a report does not name an owner or due date, the service must not manufacture one. unclear

is intentional too. A model that cannot classify a report should create a review decision, not a confident-looking label.

Keep provider-specific request details in one function. The rest of the service should know about ModerationSummary

, not about a particular SDK response object. This example uses an injected HTTP endpoint, so the same application code can point at a hosted or self-hosted implementation without changing the domain contract.

type LlmClient = (input: {
  system: string;
  user: string;
  schema: typeof summarySchema;
}) => Promise<unknown>;

function isModerationSummary(value: unknown): value is ModerationSummary {
  if (!value || typeof value !== "object") return false;
  const item = value as Record<string, unknown>;
  if (typeof item.title !== "string" || typeof item.summary !== "string") return false;
  if (!Array.isArray(item.bullets) || !item.bullets.every((x) => typeof x === "string")) return false;
  if (!["fraud", "abuse", "privacy", "other", "unclear"].includes(String(item.category))) return false;
  if (!["high", "medium", "low"].includes(String(item.confidence))) return false;
  if (!Array.isArray(item.action_items)) return false;
  return item.action_items.every((entry) => {
    if (!entry || typeof entry !== "object") return false;
    const action = entry as Record<string, unknown>;
    return typeof action.text === "string" &&
      (typeof action.owner === "string" || action.owner === null) &&
      (typeof action.due === "string" || action.due === null);
  });
}

async function classifyReport(client: LlmClient, report: string): Promise<ModerationSummary> {
  const result = await client({
    system: [
      "Return JSON only.",
      "Use the supplied schema exactly.",
      "Use category unclear and confidence low when the evidence is insufficient.",
      "Never invent an owner, due date, account, transaction, or policy fact.",
    ].join(" "),
    user: `Classify this moderation report for human review:\n${report}`,
    schema: summarySchema,
  });

  if (!isModerationSummary(result)) {
    throw new Error("LLM output failed the moderation summary contract");
  }
  return result;
}

The adapter can ask a model for structured output when the selected API supports it. It still needs the local guard. Schema enforcement at the remote boundary is useful, but it does not replace checking array members, enum values, null handling, or maximum lengths before writing an audit record. I don't trust a remote promise to protect a local queue.

The report is not a prompt-shaped database

A moderation report can contain an account identifier, a payment reference, a quoted message, and a request to reverse a decision. Store the original report separately from the generated summary. Keep a hash or immutable identifier in the review record. That makes it possible to compare a later summary with the exact source without putting raw sensitive text into every downstream message. A practical fixture might contain a report saying that a card payment was unauthorized, a quoted support reply claiming the customer approved it, and an action request that names no owner. The correct JSON should preserve those competing claims in bullets, choose unclear

if the evidence does not settle the category, and leave owner

as null

. It should not turn the support reply into a fact, infer an account holder from an email signature, or convert an angry sentence into a fraud label. This is where a tiny schema earns its keep: reviewers can see what was extracted, while an audit job can reject a missing field without interpreting a paragraph.

Test the ugly cases. Empty reports, contradictory claims, duplicated reports, mixed languages, very long quoted threads, and a report that asks the model to ignore the classification task all deserve fixtures. The expected result is not always a category. For weak evidence, the expected result is unclear

, low confidence, and a human-review action.

I benchmark the boundary, not just the model. Measure JSON acceptance rate, p95 latency, retry count, token or input size, category disagreement, and reviewer overrides. A provider that wins a single sample on prose quality may lose in production if its output needs more repair or its adapter makes every retry expensive. Your mileage may vary; the evaluation set decides that question.

Retries need a policy. Retry malformed JSON only when the request is safe to repeat and the service can identify duplicate work. RFC 9110 defines the HTTP semantics worth using here: method safety and idempotency affect whether automatic retries are reasonable. For a classification endpoint, an internal request ID and an idempotent persistence step matter more than a clever backoff loop.

The trade-off is real: the common adapter is not suitable when the application depends on a provider-specific structured-output feature, private deployment controls, or a model behavior that cannot be expressed through the shared contract. Stick with a direct provider client when that capability is the product requirement; an adapter can add translation code without removing the underlying dependency. In that case, a direct integration can be the more honest design. Keep the domain schema anyway, and make the direct client implement it.

It is also a poor fit for automatic enforcement with no human path. A moderation summary is evidence compression, not a final fraud determination. For high-impact actions, route low-confidence and policy-sensitive reports to trained reviewers, retain the source evidence, and record who made the final decision.

Speech input is another separate boundary. The open-source Whisper project is a speech-recognition system, not a moderation-summary contract; adding transcription means adding a source-quality test before classification. Do not hide that extra stage behind a single summarize()

function.

The useful portability rule is small: standardize the object your application consumes, isolate the protocol each provider speaks, and let measurements decide which adapter is the default. Fewer assumptions survive a provider change.