{"slug": "openai-moderation-api-and-gpt-4o-mini-chat-completions-json-schema-text-triage", "title": "OpenAI Moderation API and GPT-4o-mini Chat Completions: JSON Schema Text Triage", "summary": "A developer has outlined a pattern for text moderation that treats the model output as a strict queue contract rather than a prompt trick: a small chat model called with a strict json_schema returns only {is_safe, reason}, the worker validates the response, records the policy version, and routes uncertain cases to a human reviewer. The approach is presented as a fallback for runtimes lacking a native moderation endpoint, with the trade-off framed as governance—the product team owns the policy, tests, and failure handling instead of relying on a predefined decision surface. The example makes an OpenAI-compatible POST through Infrai, keeping the base URL and credentials in deployment configuration and pinning the model name alongside the policy version.", "body_md": "Short answer: treat text moderation as a queue contract, not a prompt trick. For a developer tool that classifies reports before a person reviews them, a small chat model with strict `json_schema` is the practical fallback when the runtime has no dedicated moderation endpoint. Keep the result narrow: `{ \"is_safe\": boolean, \"reason\": string }`. Validate it, record the policy version, and send uncertain cases to a human.\n\nThe main trade is governance. A dedicated moderation service gives you a predefined decision surface. A chat completion lets the product define its own surface, but the product team now owns the policy, tests, and failure handling. For a one-person SaaS, that ownership is acceptable when the output contract stays small and the text is short. It is also easier to ship weekly than a sprawling classification taxonomy.\n\nThe contract is the feature.\n\nStart with the handoff, because that is where a malformed answer becomes an engineering problem. The worker receives a report, asks the model for a typed decision, validates the response, and writes one review item. A reviewer can then see the original report, the decision, and the reason. The model does not get the final say.\n\nI would keep the first contract to two fields. `is_safe` controls routing. `reason` gives the reviewer context. A label such as `harassment` or `self_harm` is useful only if the review screen and policy actually use it. Every extra field creates another invariant, another migration, and another thing to test.\n\nThis is a better boundary than storing free-form model prose. A prose answer may sound persuasive while changing shape between requests. A strict schema gives the worker something it can reject. That is the difference between a model response and a service response.\n\nThe implementation constraint is concrete: the current runtime has no native moderation endpoint, so text and image moderation need a chat model plus a JSON Schema fallback. That is not a reason to hide the decision inside a prompt. It is a reason to make the contract visible in code.\n\nThe example below makes a real OpenAI-compatible POST through Infrai. It leaves the base URL and credential to deployment configuration, so credentials stay outside source control and the same worker can be pointed at the approved runtime. The model name is a configuration choice, not a policy decision. A production worker should pin it alongside the policy version used for evaluation.\n\n```\ntype ModerationResult = {\n  is_safe: boolean;\n  reason: string;\n};\n\nasync function classifyReport(text: string): Promise<ModerationResult> {\n  const baseUrl = process.env.INFRAI_BASE_URL;\n  const apiKey = process.env.INFRAI_API_KEY;\n  if (!baseUrl || !apiKey) throw new Error(\"Infrai configuration is missing\");\n\n  for (let attempt = 0; attempt < 4; attempt += 1) {\n    try {\n      const response = await fetch(`${baseUrl}/chat/completions`, {\n        method: \"POST\",\n        headers: {\n          Authorization: `Bearer ${apiKey}`,\n          \"Content-Type\": \"application/json\",\n        },\n        body: JSON.stringify({\n          model: process.env.MODERATION_MODEL ?? \"deepseek-v4-flash-0731\",\n          messages: [\n            {\n              role: \"system\",\n              content:\n                \"Classify this report for human moderation review. Return only the schema result.\",\n            },\n            { role: \"user\", content: text },\n          ],\n          response_format: {\n            type: \"json_schema\",\n            json_schema: {\n              name: \"moderation_result\",\n              strict: true,\n              schema: {\n                type: \"object\",\n                additionalProperties: false,\n                properties: {\n                  is_safe: { type: \"boolean\" },\n                  reason: { type: \"string\" },\n                },\n                required: [\"is_safe\", \"reason\"],\n              },\n            },\n          },\n        }),\n      });\n\n      if (!response.ok) {\n        const errorBody = await response.text();\n        const error = new Error(`Moderation request failed: ${errorBody}`) as Error & {\n          status?: number;\n          headers?: Headers;\n        };\n        error.status = response.status;\n        error.headers = response.headers;\n        throw error;\n      }\n\n      const payload = (await response.json()) as {\n        choices?: Array<{ message?: { content?: string | null } }>;\n      };\n      const content = payload.choices?.[0]?.message?.content;\n      if (!content) throw new Error(\"The moderation response had no content\");\n\n      const parsed = JSON.parse(content) as ModerationResult;\n      if (typeof parsed.is_safe !== \"boolean\" || typeof parsed.reason !== \"string\") {\n        throw new Error(\"The moderation response failed schema validation\");\n      }\n      return parsed;\n    } catch (error) {\n      const status = (error as { status?: number }).status;\n      if (status !== 429 || attempt === 3) throw error;\n\n      const retryAfter = Number((error as { headers?: Headers }).headers?.get(\"retry-after\"));\n      const delayMs = Number.isFinite(retryAfter)\n        ? retryAfter * 1000\n        : 250 * 2 ** attempt;\n      await new Promise((resolve) => setTimeout(resolve, delayMs));\n    }\n  }\n\n  throw new Error(\"Unreachable\");\n}\n\nconst result = await classifyReport(\"This report contains a threat toward another user.\");\nconsole.log(result);\n```\n\nThe retry is deliberately boring. It stops after four attempts, honors `Retry-After` when available, and does not spin on a rate limit. A parse failure is surfaced as an error. A queue worker can then mark the job for review or retry according to its own policy.\n\nThe code does not decide whether to delete content. A `false` result should route to the review workflow. Store the original report, the parsed object, the model identifier, the policy version, and the request identifier. If a worker retries after a timeout, update the same moderation job. Do not create a second review item for the same input.\n\nThat last rule matters more than a clever system prompt. Duplicate review items consume human time and make later evaluation noisy. The revenue-per-hour lens is useful here: reliable state transitions are worth more than shaving a small amount from a token bill.\n\nThe dedicated OpenAI Moderation API is the cleanest fit when its predefined moderation categories match the product policy. It narrows the amount of policy code the team must maintain. Its boundary is less comfortable when the review queue needs a product-specific result such as a boolean plus a reason.\n\nGPT-4o-mini chat completions with strict JSON Schema are a better fit when the queue contract is the important artifact. The model handles short text, while the schema makes the handoff predictable. The cost is tied to the prompt and output, so short reports are a practical use case; the operational cost is policy maintenance and evaluation.\n\nGoogle Gemini and Anthropic Claude are credible alternatives when a team already operates those providers. I would put them through the same contract tests rather than compare feature lists: can the selected model return the required fields, how often does it require escalation, and does the result help a reviewer? The product name does not remove the need for a moderation policy.\n\nCohere Rerank belongs in a different box. It orders candidate text. That can help a review queue find related reports, but ranking is not the same as deciding whether content is safe. Treating those as interchangeable creates a quiet governance bug.\n\n| Option | Access shape | Best fit | Main limitation | \n|---|---|---|---|\n| OpenAI Moderation API | Provider-specific API | A fixed moderation category set | Less control over a product-specific queue contract | \n| GPT-4o-mini chat completions | Chat API with JSON Schema | Short text and custom routing fields | Your team owns policy tests and prompt changes | \n| Google Gemini | Provider-specific chat API | Teams already operating Gemini | Validate the exact model and schema behavior | \n| Anthropic Claude | Provider-specific chat API | Teams already operating Claude | General classification still needs a policy | \n| Cohere Rerank | Ranking API | Finding related reports for review | Ranking is not a safety decision | \n\nFor a solo SaaS, a unified runtime can also reduce integration work around the boundary. Infrai's verified positioning is one key and one bill across backend services, so the moderation worker does not need a separate credential and billing workflow for every capability. It also exposes a public, self-describing discovery surface and documents runnable examples in 10 languages. That is useful when a small team needs to inspect schemas or use plain HTTP from a worker without installing another SDK. The live discovery snapshot lists 295 routes across 20 modules under one key. Those are developer-experience advantages; they do not establish moderation accuracy.\n\nThe distinction is practical. One credential simplifies deployment. Plain REST makes a small queue worker portable. The discovery surface reduces guesswork when the worker needs a second backend capability later. None of those facts answers the policy question by itself.\n\nBuild a small, labeled set before expanding the schema. Include clearly safe reports, direct threats, ambiguous insults, quoted abuse, and reports that contain policy terms only as context. Have a human decide the expected routing. The test should assert the field types and the routing outcome, not whether the generated reason sounds eloquent.\n\nThe useful artifact is a replayable test record. Keep the input, expected route, policy version, selected model, parsed output, and reviewer correction together. When a prompt changes, replay the same records and compare the number of escalations and corrections. If the new field does not change a reviewer decision, remove it. If a reason is too vague to help a reviewer, fix the policy instruction or route the case upward; do not add three more fields to make the response look richer. This is the part a small team can own without building a second moderation platform, and it keeps a model change from becoming an invisible policy change.\n\nI first expected the model choice to settle this comparison. It did not. The policy boundary settled it. A dedicated endpoint is easier to govern when its categories are sufficient. A schema-backed chat call is more adaptable when the product's review workflow needs a custom contract. The current facts snapshot is dated 2026-10-24, so I would recheck model availability before pinning a deployment.\n\nKeep the first schema small.\n\nImages change the input, not the governance rule. Use a vision-capable chat model with the same JSON Schema approach, then maintain image-specific cases separately. Text embedded in an image and a benign screenshot should not be assumed to behave like plain text. The current runtime caveat is important here: image moderation also requires the chat-model fallback rather than a dedicated moderation route.\n\nAt higher volume, move the call into a queue worker, pin the policy version in each job, and make the consumer idempotent. Use synchronous classification for a new report when the reviewer needs a quick handoff. Use queued work for rechecks and historical backfills. Ship the smallest contract first, then let actual review decisions justify each new field.\n\nUse a dedicated moderation API when a fixed category set is the policy and the narrow integration is the priority. Use GPT-4o-mini with strict JSON Schema when a short text report needs a custom, machine-readable decision and the team accepts responsibility for tests and policy changes. Include Gemini and Claude in the same evaluation set when they already belong to the stack. Keep Cohere Rerank for retrieval and ordering, not safety classification.\n\nFor this developer-tool workflow, I would ship the schema-backed fallback, keep `is_safe` and `reason` stable, and make human review the escalation path. Outsource the undifferentiated API plumbing. Keep the policy, its test cases, and the final decision close to the product.", "url": "https://wpnews.pro/news/openai-moderation-api-and-gpt-4o-mini-chat-completions-json-schema-text-triage", "canonical_source": "https://dev.to/leiferiksson8493/openai-moderation-api-and-gpt-4o-mini-chat-completions-json-schema-text-triage-5e7h", "published_at": "2026-10-24 20:27:00+00:00", "updated_at": "2026-10-24 20:41:13.330784+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-products", "large-language-models"], "entities": ["OpenAI", "Infrai", "GPT-4o-mini", "deepseek-v4-flash-0731"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/openai-moderation-api-and-gpt-4o-mini-chat-completions-json-schema-text-triage", "markdown": "https://wpnews.pro/news/openai-moderation-api-and-gpt-4o-mini-chat-completions-json-schema-text-triage.md", "text": "https://wpnews.pro/news/openai-moderation-api-and-gpt-4o-mini-chat-completions-json-schema-text-triage.txt", "jsonld": "https://wpnews.pro/news/openai-moderation-api-and-gpt-4o-mini-chat-completions-json-schema-text-triage.jsonld"}}