Tenant-Aware: How to Moderate Text Prompts for Image Generation Without Chat JSON

Short answer: classify the complete image request before generation, return a small JSON decision, and attach the classifier and image calls to the same tenant ledger. The useful design choice is a cost boundary, not a particular moderation endpoint.

For a logistics product, that ledger matters. A carrier, warehouse operator, and internal support team may all send prompts through one image feature, while their review rates and prompt sizes differ. If classification is an invisible helper call, the team cannot explain why one tenant's monthly AI spend moved. I prefer a notebook-to-prod path that makes the policy contract, the tenant key, and the eval record explicit from the first example.

The gate runs before the image job enters the queue. It receives the exact text that will be sent to the image model: the user's description, a selected style, a negative prompt, and any template fields. It emits allow

, review

, or block

. Only allow

can create an image job.

That boundary is non-negotiable.

Treat the classifier as a policy component, not as a safety oracle. The application owns the policy categories and the action taken for each result. A JSON schema makes the boundary easy to validate, but valid JSON alone does not show that the policy is useful.

The following example uses plain HTTP-shaped configuration so the application can point at its chosen chat classifier and image service. It is Python because my production examples need to remain close to the eval harness; the same request bodies fit a Node.js HTTP client. The URLs are configuration, not recommendations.

import json
import os
import urllib.request
import uuid

DECISIONS = {"allow", "review", "block"}

def post_json(url, payload, headers):
    request = urllib.request.Request(
        url,
        data=json.dumps(payload).encode("utf-8"),
        headers={**headers, "Content-Type": "application/json"},
        method="POST",
    )
    with urllib.request.urlopen(request, timeout=60) as response:
        return json.loads(response.read().decode("utf-8"))

def moderate_and_queue(tenant_id, prompt, style, negative_prompt=""):
    candidate = {
        "prompt": prompt,
        "style": style,
        "negative_prompt": negative_prompt,
    }
    headers = {"Authorization": f"Bearer {os.environ['AI_API_KEY']}"}

    result = post_json(
        os.environ["CLASSIFIER_URL"],
        {
            "input": candidate,
            "response_schema": {
                "type": "object",
                "properties": {
                    "decision": {
                        "type": "string",
                        "enum": ["allow", "review", "block"],
                    },
                    "reason": {"type": "string"},
                },
                "required": ["decision", "reason"],
                "additionalProperties": False,
            },
        },
        headers,
    )
    decision = result.get("decision")
    if decision not in DECISIONS:
        return {"decision": "review", "reason": "Invalid policy result"}
    if decision != "allow":
        return result

    job_id = str(uuid.uuid4())
    job = post_json(
        os.environ["IMAGE_URL"],
        {"job_id": job_id, "prompt": candidate},
        {**headers, "Idempotency-Key": job_id},
    )
    return {"decision": "allow", "job": job}

print(moderate_and_queue(
    tenant_id="warehouse-west",
    prompt="A clean  dock diagram at sunrise",
    style="technical illustration",
))

One detail is intentionally boring: a malformed result becomes review

, never allow

. Network failures should take the same non-generation path in the surrounding worker, with a durable reason and a retry policy. The image request carries an idempotency key because a retry must not accidentally create two jobs; HTTP retry behavior and idempotency are application concerns that should be designed against the semantics in RFC 9110, not guessed from a client library.

The sample also passes tenant_id

into the function even though the remote payload does not need it. That is a reminder to record ownership locally. Before classification, create an accounting record containing tenant, request ID, policy version, and prompt-token estimate. After each call, append usage reported by the service when available. Do not combine classifier and image usage into one number.

The first failure is partial input. A classifier sees the main prompt but not a template's style field, so a harmless-looking request can acquire meaning later. Build one canonical candidate object, serialize that object for classification, and use the same object for image generation. This eliminates a surprisingly ordinary class of policy drift.

The second failure is an ambiguous result. “Probably okay” is not a production state. Keep the contract small, reject unknown enum values, and send review

to a human queue with enough context for adjudication. A block response should be generic to the requester; detailed policy reasoning belongs in restricted operational logs.

The third failure is cost attribution. A tenant with long prompts may consume more classifier tokens even when every image is blocked. That is a real cost and a useful signal. Track classifier input size, image input size, outcome, latency, and retry count separately. A per-tenant view should answer both “what did this request cost?” and “where did it stop?”

Here is the decision table I would put next to the queue design:

The table is a small thing, but it prevents a common design mistake: treating safety and accounting as separate middleware. They observe the same request, so they should share the same request ID.

Consider a tenant that submits a long prompt assembled from a route description, a warehouse preset, and a user-selected style. The classifier may return block

, so no image is produced, yet the classification attempt still consumed time and tokens. If the ledger records only successful images, the tenant sees an unexplained gap between its activity dashboard and its invoice. If the ledger records only raw text, the operations team has created a privacy problem while trying to solve a finance problem. The useful record is narrower: tenant ID, request ID, policy version, stage, measured usage, decision, and a short-lived fingerprint. That record supports reconciliation without turning every cost report into a copy of the prompt store.

Start with an eval set divided into ordinary creative requests, clear policy violations, ambiguous cases, and attempts to hide instructions in editable fields. Store expected action, not only a label. Replay the set when the policy prompt, classifier model, JSON schema, or image template changes. I’m not sure any single threshold will fit every logistics workflow; the review queue's adjudicated results are what should settle that question.

Measure false allows and false blocks by tenant segment. A warehouse diagram tool may need a different review threshold from a public creative feature, but that difference should live in a versioned policy configuration rather than an untracked prompt edit. Keep raw text retention as short as the product and compliance requirements allow, and use a fingerprint in routine cost reports.

Retries need boundaries. Honor a server-provided retry delay when one exists, cap attempts, and distinguish a retryable transport response from a policy decision. RFC 9110 is a useful baseline for thinking about method semantics and retry safety. For an image job, an idempotency key plus a durable request ID lets the worker recover without silently duplicating work; it does not make every failure retryable.

Use this architecture when you need custom review states, complete- input coverage, and a per-tenant explanation of classifier and generation spend. It works well with a standards-oriented HTTP boundary and a small application-owned schema.

The catch is policy ownership. This is not suitable when a managed moderation workflow, fixed safety taxonomy, or provider-specific audit surface is a hard requirement; choose that managed path when its operational guarantees matter more than a portable contract. It is also a poor fit if nobody can label review cases or maintain an eval set. A shared interface cannot compensate for absent policy stewardship.

Before launch, verify five things in prose: every user-editable field reaches the gate; only a validated allow

can enqueue an image; malformed or unavailable classifier responses fail closed into review; retries are bounded and idempotent; and the ledger records tenant, policy version, classifier usage, image usage, and outcome separately. Then run the eval set in CI and sample live review cases. Keep the implementation plain. Plain is easier to audit.