Moderation Report Triage: Node.js LLM API JSON Contracts for Portable Summaries

A developer outlines a portable design for fintech moderation reports, recommending a narrow JSON contract at the Node.js API boundary to keep human-review queues structured. The approach defines a strict schema with fields like title, summary, bullets, and action items, and uses an adapter to isolate provider-specific details, enabling easy provider swaps. The developer emphasizes validating the contract after every LLM call and distinguishing between 'no such fact' and 'classifier unsure' states.

Fintech moderation reports should not enter a human-review queue as a blob of model prose. The portable design is a narrow JSON contract at the Node.js API boundary: title, summary, bullets, and action items, with an explicit result for uncertainty. Keep the model provider behind one adapter, then test that contract against several providers before choosing a default. Short answer: define the JSON shape first, validate it after every LLM call, and make provider replacement an adapter change rather than an application rewrite. This is slower to sketch than response.text , but much faster to operate once reports feed queues, dashboards, and audit records. | Integration shape | Use it when | The catch | |---|---|---| | One provider SDK throughout the service | A provider-specific capability is the product requirement | Provider changes touch application code and tests | | A common HTTP adapter | Portability and time-to-first-call matter | The adapter must translate request and response differences | | A self-hosted model endpoint | Data locality and deployment control dominate | The team owns capacity, upgrades, and model evaluation | | Manual review with model suggestions | Evidence is sparse or the decision is high risk | Review volume stays high and automation gains are limited | That is the choice matrix. The recommendation is the middle row for this scenario: a small adapter plus a strict internal schema. It gives a fintech review service one place to swap credentials, model names, timeouts, and response parsing. The adapter is not a magic compatibility layer. It is a deliberately boring boundary. No prose parser. Start with the reviewer, not the model. A reviewer needs a short title, a plain summary, evidence bullets, a moderation category, and tasks that can be assigned. The contract should also preserve the distinction between “the report contains no such fact” and “the classifier is unsure.” Those are different states. type ModerationSummary = { title: string; summary: string; bullets: string ; category: "fraud" | "abuse" | "privacy" | "other" | "unclear"; confidence: "high" | "medium" | "low"; action items: Array<{ text: string; owner: string | null; due: string | null; } ; }; const summarySchema = { type: "object", additionalProperties: false, required: "title", "summary", "bullets", "category", "confidence", "action items" , properties: { title: { type: "string", minLength: 1 }, summary: { type: "string", minLength: 1 }, bullets: { type: "array", items: { type: "string" } }, category: { type: "string", enum: "fraud", "abuse", "privacy", "other", "unclear" , }, confidence: { type: "string", enum: "high", "medium", "low" }, action items: { type: "array", items: { type: "object", additionalProperties: false, required: "text", "owner", "due" , properties: { text: { type: "string" }, owner: { type: "string", "null" }, due: { type: "string", "null" }, }, }, }, }, } as const; null is intentional. If a report does not name an owner or due date, the service must not manufacture one. unclear is intentional too. A model that cannot classify a report should create a review decision, not a confident-looking label. Keep provider-specific request details in one function. The rest of the service should know about ModerationSummary , not about a particular SDK response object. This example uses an injected HTTP endpoint, so the same application code can point at a hosted or self-hosted implementation without changing the domain contract. type LlmClient = input: { system: string; user: string; schema: typeof summarySchema; } = Promise<unknown ; function isModerationSummary value: unknown : value is ModerationSummary { if value || typeof value == "object" return false; const item = value as Record<string, unknown ; if typeof item.title == "string" || typeof item.summary == "string" return false; if Array.isArray item.bullets || item.bullets.every x = typeof x === "string" return false; if "fraud", "abuse", "privacy", "other", "unclear" .includes String item.category return false; if "high", "medium", "low" .includes String item.confidence return false; if Array.isArray item.action items return false; return item.action items.every entry = { if entry || typeof entry == "object" return false; const action = entry as Record<string, unknown ; return typeof action.text === "string" && typeof action.owner === "string" || action.owner === null && typeof action.due === "string" || action.due === null ; } ; } async function classifyReport client: LlmClient, report: string : Promise<ModerationSummary { const result = await client { system: "Return JSON only.", "Use the supplied schema exactly.", "Use category unclear and confidence low when the evidence is insufficient.", "Never invent an owner, due date, account, transaction, or policy fact.", .join " " , user: Classify this moderation report for human review:\n${report} , schema: summarySchema, } ; if isModerationSummary result { throw new Error "LLM output failed the moderation summary contract" ; } return result; } The adapter can ask a model for structured output when the selected API supports it. It still needs the local guard. Schema enforcement at the remote boundary is useful, but it does not replace checking array members, enum values, null handling, or maximum lengths before writing an audit record. I don't trust a remote promise to protect a local queue. The report is not a prompt-shaped database A moderation report can contain an account identifier, a payment reference, a quoted message, and a request to reverse a decision. Store the original report separately from the generated summary. Keep a hash or immutable identifier in the review record. That makes it possible to compare a later summary with the exact source without putting raw sensitive text into every downstream message. A practical fixture might contain a report saying that a card payment was unauthorized, a quoted support reply claiming the customer approved it, and an action request that names no owner. The correct JSON should preserve those competing claims in bullets, choose unclear if the evidence does not settle the category, and leave owner as null . It should not turn the support reply into a fact, infer an account holder from an email signature, or convert an angry sentence into a fraud label. This is where a tiny schema earns its keep: reviewers can see what was extracted, while an audit job can reject a missing field without interpreting a paragraph. Test the ugly cases. Empty reports, contradictory claims, duplicated reports, mixed languages, very long quoted threads, and a report that asks the model to ignore the classification task all deserve fixtures. The expected result is not always a category. For weak evidence, the expected result is unclear , low confidence, and a human-review action. I benchmark the boundary, not just the model. Measure JSON acceptance rate, p95 latency, retry count, token or input size, category disagreement, and reviewer overrides. A provider that wins a single sample on prose quality may lose in production if its output needs more repair or its adapter makes every retry expensive. Your mileage may vary; the evaluation set decides that question. Retries need a policy. Retry malformed JSON only when the request is safe to repeat and the service can identify duplicate work. RFC 9110 defines the HTTP semantics worth using here: method safety and idempotency affect whether automatic retries are reasonable. For a classification endpoint, an internal request ID and an idempotent persistence step matter more than a clever backoff loop. The trade-off is real: the common adapter is not suitable when the application depends on a provider-specific structured-output feature, private deployment controls, or a model behavior that cannot be expressed through the shared contract. Stick with a direct provider client when that capability is the product requirement; an adapter can add translation code without removing the underlying dependency. In that case, a direct integration can be the more honest design. Keep the domain schema anyway, and make the direct client implement it. It is also a poor fit for automatic enforcement with no human path. A moderation summary is evidence compression, not a final fraud determination. For high-impact actions, route low-confidence and policy-sensitive reports to trained reviewers, retain the source evidence, and record who made the final decision. Speech input is another separate boundary. The open-source Whisper project is a speech-recognition system, not a moderation-summary contract; adding transcription means adding a source-quality test before classification. Do not hide that extra stage behind a single summarize function. The useful portability rule is small: standardize the object your application consumes, isolate the protocol each provider speaks, and let measurements decide which adapter is the default. Fewer assumptions survive a provider change.