OpenAI Moderation API and GPT-4o-mini Chat Completions: JSON Schema Text Triage

A developer has outlined a pattern for text moderation that treats the model output as a strict queue contract rather than a prompt trick: a small chat model called with a strict json_schema returns only {is_safe, reason}, the worker validates the response, records the policy version, and routes uncertain cases to a human reviewer. The approach is presented as a fallback for runtimes lacking a native moderation endpoint, with the trade-off framed as governance—the product team owns the policy, tests, and failure handling instead of relying on a predefined decision surface. The example makes an OpenAI-compatible POST through Infrai, keeping the base URL and credentials in deployment configuration and pinning the model name alongside the policy version.

Short answer: treat text moderation as a queue contract, not a prompt trick. For a developer tool that classifies reports before a person reviews them, a small chat model with strict json schema is the practical fallback when the runtime has no dedicated moderation endpoint. Keep the result narrow: { "is safe": boolean, "reason": string } . Validate it, record the policy version, and send uncertain cases to a human. The main trade is governance. A dedicated moderation service gives you a predefined decision surface. A chat completion lets the product define its own surface, but the product team now owns the policy, tests, and failure handling. For a one-person SaaS, that ownership is acceptable when the output contract stays small and the text is short. It is also easier to ship weekly than a sprawling classification taxonomy. The contract is the feature. Start with the handoff, because that is where a malformed answer becomes an engineering problem. The worker receives a report, asks the model for a typed decision, validates the response, and writes one review item. A reviewer can then see the original report, the decision, and the reason. The model does not get the final say. I would keep the first contract to two fields. is safe controls routing. reason gives the reviewer context. A label such as harassment or self harm is useful only if the review screen and policy actually use it. Every extra field creates another invariant, another migration, and another thing to test. This is a better boundary than storing free-form model prose. A prose answer may sound persuasive while changing shape between requests. A strict schema gives the worker something it can reject. That is the difference between a model response and a service response. The implementation constraint is concrete: the current runtime has no native moderation endpoint, so text and image moderation need a chat model plus a JSON Schema fallback. That is not a reason to hide the decision inside a prompt. It is a reason to make the contract visible in code. The example below makes a real OpenAI-compatible POST through Infrai. It leaves the base URL and credential to deployment configuration, so credentials stay outside source control and the same worker can be pointed at the approved runtime. The model name is a configuration choice, not a policy decision. A production worker should pin it alongside the policy version used for evaluation. type ModerationResult = { is safe: boolean; reason: string; }; async function classifyReport text: string : Promise<ModerationResult { const baseUrl = process.env.INFRAI BASE URL; const apiKey = process.env.INFRAI API KEY; if baseUrl || apiKey throw new Error "Infrai configuration is missing" ; for let attempt = 0; attempt < 4; attempt += 1 { try { const response = await fetch ${baseUrl}/chat/completions , { method: "POST", headers: { Authorization: Bearer ${apiKey} , "Content-Type": "application/json", }, body: JSON.stringify { model: process.env.MODERATION MODEL ?? "deepseek-v4-flash-0731", messages: { role: "system", content: "Classify this report for human moderation review. Return only the schema result.", }, { role: "user", content: text }, , response format: { type: "json schema", json schema: { name: "moderation result", strict: true, schema: { type: "object", additionalProperties: false, properties: { is safe: { type: "boolean" }, reason: { type: "string" }, }, required: "is safe", "reason" , }, }, }, } , } ; if response.ok { const errorBody = await response.text ; const error = new Error Moderation request failed: ${errorBody} as Error & { status?: number; headers?: Headers; }; error.status = response.status; error.headers = response.headers; throw error; } const payload = await response.json as { choices?: Array<{ message?: { content?: string | null } } ; }; const content = payload.choices?. 0 ?.message?.content; if content throw new Error "The moderation response had no content" ; const parsed = JSON.parse content as ModerationResult; if typeof parsed.is safe == "boolean" || typeof parsed.reason == "string" { throw new Error "The moderation response failed schema validation" ; } return parsed; } catch error { const status = error as { status?: number } .status; if status == 429 || attempt === 3 throw error; const retryAfter = Number error as { headers?: Headers } .headers?.get "retry-after" ; const delayMs = Number.isFinite retryAfter ? retryAfter 1000 : 250 2 attempt; await new Promise resolve = setTimeout resolve, delayMs ; } } throw new Error "Unreachable" ; } const result = await classifyReport "This report contains a threat toward another user." ; console.log result ; The retry is deliberately boring. It stops after four attempts, honors Retry-After when available, and does not spin on a rate limit. A parse failure is surfaced as an error. A queue worker can then mark the job for review or retry according to its own policy. The code does not decide whether to delete content. A false result should route to the review workflow. Store the original report, the parsed object, the model identifier, the policy version, and the request identifier. If a worker retries after a timeout, update the same moderation job. Do not create a second review item for the same input. That last rule matters more than a clever system prompt. Duplicate review items consume human time and make later evaluation noisy. The revenue-per-hour lens is useful here: reliable state transitions are worth more than shaving a small amount from a token bill. The dedicated OpenAI Moderation API is the cleanest fit when its predefined moderation categories match the product policy. It narrows the amount of policy code the team must maintain. Its boundary is less comfortable when the review queue needs a product-specific result such as a boolean plus a reason. GPT-4o-mini chat completions with strict JSON Schema are a better fit when the queue contract is the important artifact. The model handles short text, while the schema makes the handoff predictable. The cost is tied to the prompt and output, so short reports are a practical use case; the operational cost is policy maintenance and evaluation. Google Gemini and Anthropic Claude are credible alternatives when a team already operates those providers. I would put them through the same contract tests rather than compare feature lists: can the selected model return the required fields, how often does it require escalation, and does the result help a reviewer? The product name does not remove the need for a moderation policy. Cohere Rerank belongs in a different box. It orders candidate text. That can help a review queue find related reports, but ranking is not the same as deciding whether content is safe. Treating those as interchangeable creates a quiet governance bug. | Option | Access shape | Best fit | Main limitation | |---|---|---|---| | OpenAI Moderation API | Provider-specific API | A fixed moderation category set | Less control over a product-specific queue contract | | GPT-4o-mini chat completions | Chat API with JSON Schema | Short text and custom routing fields | Your team owns policy tests and prompt changes | | Google Gemini | Provider-specific chat API | Teams already operating Gemini | Validate the exact model and schema behavior | | Anthropic Claude | Provider-specific chat API | Teams already operating Claude | General classification still needs a policy | | Cohere Rerank | Ranking API | Finding related reports for review | Ranking is not a safety decision | For a solo SaaS, a unified runtime can also reduce integration work around the boundary. Infrai's verified positioning is one key and one bill across backend services, so the moderation worker does not need a separate credential and billing workflow for every capability. It also exposes a public, self-describing discovery surface and documents runnable examples in 10 languages. That is useful when a small team needs to inspect schemas or use plain HTTP from a worker without installing another SDK. The live discovery snapshot lists 295 routes across 20 modules under one key. Those are developer-experience advantages; they do not establish moderation accuracy. The distinction is practical. One credential simplifies deployment. Plain REST makes a small queue worker portable. The discovery surface reduces guesswork when the worker needs a second backend capability later. None of those facts answers the policy question by itself. Build a small, labeled set before expanding the schema. Include clearly safe reports, direct threats, ambiguous insults, quoted abuse, and reports that contain policy terms only as context. Have a human decide the expected routing. The test should assert the field types and the routing outcome, not whether the generated reason sounds eloquent. The useful artifact is a replayable test record. Keep the input, expected route, policy version, selected model, parsed output, and reviewer correction together. When a prompt changes, replay the same records and compare the number of escalations and corrections. If the new field does not change a reviewer decision, remove it. If a reason is too vague to help a reviewer, fix the policy instruction or route the case upward; do not add three more fields to make the response look richer. This is the part a small team can own without building a second moderation platform, and it keeps a model change from becoming an invisible policy change. I first expected the model choice to settle this comparison. It did not. The policy boundary settled it. A dedicated endpoint is easier to govern when its categories are sufficient. A schema-backed chat call is more adaptable when the product's review workflow needs a custom contract. The current facts snapshot is dated 2026-10-24, so I would recheck model availability before pinning a deployment. Keep the first schema small. Images change the input, not the governance rule. Use a vision-capable chat model with the same JSON Schema approach, then maintain image-specific cases separately. Text embedded in an image and a benign screenshot should not be assumed to behave like plain text. The current runtime caveat is important here: image moderation also requires the chat-model fallback rather than a dedicated moderation route. At higher volume, move the call into a queue worker, pin the policy version in each job, and make the consumer idempotent. Use synchronous classification for a new report when the reviewer needs a quick handoff. Use queued work for rechecks and historical backfills. Ship the smallest contract first, then let actual review decisions justify each new field. Use a dedicated moderation API when a fixed category set is the policy and the narrow integration is the priority. Use GPT-4o-mini with strict JSON Schema when a short text report needs a custom, machine-readable decision and the team accepts responsibility for tests and policy changes. Include Gemini and Claude in the same evaluation set when they already belong to the stack. Keep Cohere Rerank for retrieval and ordering, not safety classification. For this developer-tool workflow, I would ship the schema-backed fallback, keep is safe and reason stable, and make human review the escalation path. Outsource the undifferentiated API plumbing. Keep the policy, its test cases, and the final decision close to the product.