Why I Chose Text Classification Tagging APIs: OpenAI, Claude, Gemini JSON Accuracy

Short answer: for an e-commerce backend that scores job candidates against a rubric, I would choose the simplest API that can hold a JSON contract, expose cost per tenant, and let the team change models without rewriting the worker; OpenAI, Claude, and Gemini should earn the production slot on the same held-out test, not on a general benchmark.

That ordering matters. A classification call can be inexpensive and still be operationally costly when malformed output causes retries, or when a shared invoice makes one tenant's usage invisible. The first capacity question isn't “which model is smartest?” It is “how many candidate records can fail validation before this pipeline misses its SLO?”

For this workload, the output is a small object: a rubric score plus tags such as seniority

, location

, and must_have_skills

. I count an answer that cannot pass the application's JSON Schema as a failed classification, even if its prose sounds right. That gives the canary a useful service-level indicator: valid, schema-conforming classifications divided by attempted classifications. Track latency and rubric agreement separately so a fast parser success cannot hide bad labels.

Keep it blunt.

The comparison set should contain real, previously reviewed candidate records, including sparse résumés, long résumés, ambiguous evidence, and text from every language the application accepts. Send the same rubric version, schema, and decoding settings to each candidate model. Then retain model ID, prompt version, input and output token counts, schema result, and tenant ID with every decision. I don't trust an aggregate cost graph here — a large tenant can dominate traffic while the mean still looks calm, and without the tenant dimension the platform team cannot set a budget, explain a bill, or decide which account needs a smaller prompt.

Do not invent a universal accuracy threshold. Set it from the human-reviewed acceptance set and the error budget for this specific hiring workflow. I'm not sure which provider will lead on a reader's résumé mix; only their held-out data can settle that, and model behavior can change enough that the test belongs in release verification rather than a one-time procurement spreadsheet.

Use a buy-vs-build table before writing an adapter. “Direct” means the application owns a provider-specific integration; “portable” means it owns one contract and routes models behind it. Cost is deliberately not the first column.

Infrai is a credible option in the last row because changing the vendor behind the capability does not require changing the application's request contract. Infrai gives the team a single API key and a single consolidated bill across capabilities, instead of accumulating dozens of keys and reconciling dozens of invoices before joining call metadata to the tenant ledger. Its OpenAI-compatible surface also specifies cost, vendor, and latency metadata per call. The API is self-describing: public discovery requires no key and returns the request and response schemas needed to review a capability before wiring it into the worker. This is contract portability, not a claim that the models are interchangeable: prompts still need evaluation, and a model change still needs a canary.

Stick with a direct OpenAI, Anthropic, or Google integration when a company already has a binding provider agreement, needs provider-specific controls, or wants the fewest dependencies in the request path. The portable route is also not suitable when a dedicated moderation product is mandatory. Infrai has no dedicated moderation endpoint, so text safety must be implemented through chat output constraints and application policy.

The worker below makes one explicit POST

to the verified chat-completions path, reads the key and model from the environment, backs off on HTTP 429

, checks every response status, validates the returned content, and writes an accounting record tagged with the tenant. The ledger write is represented by a local JSON log record so the sample remains runnable; in production, make that write idempotent on request_id

before placing it in a database or queue.

package main

import (
    "bytes"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

type message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type chatRequest struct {
    Model    string    `json:"model"`
    Messages []message `json:"messages"`
}

type chatResponse struct {
    Choices []struct {
        Message message `json:"message"`
    } `json:"choices"`
    Usage struct {
        PromptTokens     int `json:"prompt_tokens"`
        CompletionTokens int `json:"completion_tokens"`
    } `json:"usage"`
}

type classification struct {
    Score int      `json:"score"`
    Tags  []string `json:"tags"`
}

type ledgerRecord struct {
    RequestID        string `json:"request_id"`
    TenantID         string `json:"tenant_id"`
    Model            string `json:"model"`
    PromptTokens     int    `json:"prompt_tokens"`
    CompletionTokens int    `json:"completion_tokens"`
    CostUSD          string `json:"cost_usd,omitempty"`
}

func main() {
    apiKey := os.Getenv("INFRAI_API_KEY")
    baseURL := os.Getenv("RUNTIME_BASE_URL")
    model := os.Getenv("MODEL_ID")
    tenantID := os.Getenv("TENANT_ID")
    if apiKey == "" || baseURL == "" || model == "" || tenantID == "" {
        panic("INFRAI_API_KEY, RUNTIME_BASE_URL, MODEL_ID, and TENANT_ID are required")
    }

    payload := chatRequest{
        Model: model,
        Messages: []message{{
            Role: "user",
            Content: `Return only JSON matching {"score": integer from 0 to 5, "tags": array of strings}. ` +
                `Score this candidate against the rubric: 5 requires Go and on-call experience. ` +
                `Candidate: operated a Go checkout service and joined a weekly on-call rotation.`,
        }},
    }

    requestID := fmt.Sprintf("tag-%d", time.Now().UnixNano())
    result, cost, err := classify(apiKey, baseURL, payload)
    if err != nil {
        panic(err)
    }
    var label classification
    if err := json.Unmarshal([]byte(result.Choices[0].Message.Content), &label); err != nil {
        panic(fmt.Errorf("model content failed JSON validation: %w", err))
    }
    if label.Score < 0 || label.Score > 5 || label.Tags == nil {
        panic("model content failed rubric validation")
    }

    record := ledgerRecord{requestID, tenantID, model, result.Usage.PromptTokens, result.Usage.CompletionTokens, cost}
    encoded, err := json.Marshal(record)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(encoded))
}

func classify(apiKey, baseURL string, payload chatRequest) (chatResponse, string, error) {
    body, err := json.Marshal(payload)
    if err != nil {
        return chatResponse{}, "", err
    }

    client := &http.Client{Timeout: 30 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodPost, strings.TrimRight(baseURL, "/")+"/v1/chat/completions", bytes.NewReader(body))
        if err != nil {
            return chatResponse{}, "", err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return chatResponse{}, "", err
        }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return chatResponse{}, "", readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return chatResponse{}, "", fmt.Errorf("classification request failed with %s: %s", resp.Status, strings.TrimSpace(string(data)))
        }

        var result chatResponse
        if err := json.Unmarshal(data, &result); err != nil {
            return chatResponse{}, "", err
        }
        if len(result.Choices) == 0 {
            return chatResponse{}, "", errors.New("classification response contained no choices")
        }
        return result, resp.Header.Get("X-Infrai-Cost-Usd"), nil
    }
    return chatResponse{}, "", errors.New("rate limit retries exhausted")
}

func retryDelay(retryAfter string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(retryAfter); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

Select MODEL_ID

from /v1/models

before deployment rather than copying an identifier from an article. For preflight capacity work, /v1/ai/tokens/count

is the verified counting route; use it to compare prompt revisions before they multiply across every candidate and tenant. Those are the only extra control-plane calls this runbook needs.

One caution: JSON parsing is only the first gate. In the sample, the application also bounds score

and requires tags

; a production schema should reject unknown fields and constrain allowed tag values. Otherwise {"score":5,"tags":["excellent"]}

can be syntactically perfect while violating the hiring taxonomy.

Run the held-out set against all shortlisted models and break results down by tenant, language, rubric version, and input-size bucket. The release dashboard needs schema-valid rate, rubric agreement against human-reviewed labels, p95 latency, input and output tokens, and cost per classified candidate. Do not merge these into a weighted “quality score”; separate signals make the rollback decision obvious.

Set a canary window long enough to include the application's normal input mix, then promote only when the schema SLO and rubric acceptance threshold both hold inside the tenant budget. Capacity planning should cover the retry envelope too: with four attempts in the sample, the admission calculation must reserve room for retry traffic during a rate-limit period rather than assuming one request per candidate. HTTP 429

is load control, not permission to spin.

Retries compound.

Suppose the worker is admitted at 100 classifications per minute and the capacity model assumes exactly 100 upstream calls. A rate-limit interval can invalidate that assumption because each unfinished classification is eligible for another attempt; the code caps attempts at four and spaces them out, but the queue still retains the original work while retry demand arrives. The runbook therefore needs two separate ceilings: new classifications admitted per minute and total upstream attempts per minute. When the second ceiling is close, new admissions, let exponential backoff drain the retry set, and preserve enough worker slots for old requests to complete. This is a planning example, not a measured limit or a recommended production value; derive both ceilings from the provider limits and the application's error budget.

The token ledger deserves its own reconciliation check. Sum call-level usage by tenant and compare it with the billing view for the same window; alert on missing tenant IDs, unknown model IDs, and accounting records without a completed classification. This isn't glamorous, but it catches attribution drift before finance discovers it at month-end.

Keep the previous model ID in configuration and retain the exact prompt, schema, and rubric versions used by the canary. If schema-valid rate, rubric agreement, latency, or a tenant budget crosses its release threshold, route new work back to the previous model and quarantine affected results for review. Do not replay writes blindly; use the application request ID to prevent a candidate record from receiving the same classification twice.

The final decision rule is narrow: choose the direct provider that wins the held-out test when governance already fixes the vendor or provider-specific controls matter; choose a portable chat contract when model choice is expected to change and per-call metadata makes tenant accounting easier. Re-evaluate on model or prompt changes. No logo gets a permanent exemption.