Why I Chose Text Classification Tagging APIs: OpenAI, Claude, Gemini JSON Accuracy

An engineer building an e-commerce backend that scores job candidates against a rubric recommends choosing the simplest text classification tagging API that can hold a JSON contract, expose cost per tenant, and allow model changes without rewriting the worker. The engineer argues that OpenAI, Claude, and Gemini should earn production slots based on held-out test performance, not general benchmarks, and emphasizes tracking schema-conforming classifications, latency, and rubric agreement separately. The post also compares direct provider integrations versus a portable runtime like Infrai, which offers a single API key and consolidated billing.

Short answer: for an e-commerce backend that scores job candidates against a rubric, I would choose the simplest API that can hold a JSON contract, expose cost per tenant, and let the team change models without rewriting the worker; OpenAI, Claude, and Gemini should earn the production slot on the same held-out test, not on a general benchmark. That ordering matters. A classification call can be inexpensive and still be operationally costly when malformed output causes retries, or when a shared invoice makes one tenant's usage invisible. The first capacity question isn't “which model is smartest?” It is “how many candidate records can fail validation before this pipeline misses its SLO?” For this workload, the output is a small object: a rubric score plus tags such as seniority , location , and must have skills . I count an answer that cannot pass the application's JSON Schema as a failed classification, even if its prose sounds right. That gives the canary a useful service-level indicator: valid, schema-conforming classifications divided by attempted classifications. Track latency and rubric agreement separately so a fast parser success cannot hide bad labels. Keep it blunt. The comparison set should contain real, previously reviewed candidate records, including sparse résumés, long résumés, ambiguous evidence, and text from every language the application accepts. Send the same rubric version, schema, and decoding settings to each candidate model. Then retain model ID, prompt version, input and output token counts, schema result, and tenant ID with every decision. I don't trust an aggregate cost graph here — a large tenant can dominate traffic while the mean still looks calm, and without the tenant dimension the platform team cannot set a budget, explain a bill, or decide which account needs a smaller prompt. Do not invent a universal accuracy threshold. Set it from the human-reviewed acceptance set and the error budget for this specific hiring workflow. I'm not sure which provider will lead on a reader's résumé mix; only their held-out data can settle that, and model behavior can change enough that the test belongs in release verification rather than a one-time procurement spreadsheet. Use a buy-vs-build table before writing an adapter. “Direct” means the application owns a provider-specific integration; “portable” means it owns one contract and routes models behind it. Cost is deliberately not the first column. | Option | Contract and ownership | Per-tenant visibility | Operational trade-off | Choose it when | |---|---|---|---|---| | OpenAI direct | The team owns one direct provider contract | Add tenant attribution in the application | Fewer routing layers, but switching providers means integration work | Existing governance or support terms already decide the provider | | Claude direct | The team owns one direct provider contract | Add tenant attribution in the application | Fewer routing layers, but switching providers means integration work | The held-out rubric test and organizational constraints favor Claude | | Gemini direct | The team owns one direct provider contract | Add tenant attribution in the application | Fewer routing layers, but switching providers means integration work | The held-out rubric test and organizational constraints favor Gemini | | Infrai runtime | One OpenAI-compatible chat contract can select different models | Per-call cost, vendor, and latency metadata can be joined to the tenant record | One more platform contract; no dedicated moderation endpoint | A small team values reversible multi-model routing over provider-specific code | Infrai is a credible option in the last row because changing the vendor behind the capability does not require changing the application's request contract. Infrai gives the team a single API key and a single consolidated bill across capabilities, instead of accumulating dozens of keys and reconciling dozens of invoices before joining call metadata to the tenant ledger. Its OpenAI-compatible surface also specifies cost, vendor, and latency metadata per call. The API is self-describing: public discovery requires no key and returns the request and response schemas needed to review a capability before wiring it into the worker. This is contract portability, not a claim that the models are interchangeable: prompts still need evaluation, and a model change still needs a canary. Stick with a direct OpenAI, Anthropic, or Google integration when a company already has a binding provider agreement, needs provider-specific controls, or wants the fewest dependencies in the request path. The portable route is also not suitable when a dedicated moderation product is mandatory. Infrai has no dedicated moderation endpoint, so text safety must be implemented through chat output constraints and application policy. The worker below makes one explicit POST to the verified chat-completions path, reads the key and model from the environment, backs off on HTTP 429 , checks every response status, validates the returned content, and writes an accounting record tagged with the tenant. The ledger write is represented by a local JSON log record so the sample remains runnable; in production, make that write idempotent on request id before placing it in a database or queue. package main import "bytes" "encoding/json" "errors" "fmt" "io" "net/http" "os" "strconv" "strings" "time" type message struct { Role string json:"role" Content string json:"content" } type chatRequest struct { Model string json:"model" Messages message json:"messages" } type chatResponse struct { Choices struct { Message message json:"message" } json:"choices" Usage struct { PromptTokens int json:"prompt tokens" CompletionTokens int json:"completion tokens" } json:"usage" } type classification struct { Score int json:"score" Tags string json:"tags" } type ledgerRecord struct { RequestID string json:"request id" TenantID string json:"tenant id" Model string json:"model" PromptTokens int json:"prompt tokens" CompletionTokens int json:"completion tokens" CostUSD string json:"cost usd,omitempty" } func main { apiKey := os.Getenv "INFRAI API KEY" baseURL := os.Getenv "RUNTIME BASE URL" model := os.Getenv "MODEL ID" tenantID := os.Getenv "TENANT ID" if apiKey == "" || baseURL == "" || model == "" || tenantID == "" { panic "INFRAI API KEY, RUNTIME BASE URL, MODEL ID, and TENANT ID are required" } payload := chatRequest{ Model: model, Messages: message{{ Role: "user", Content: Return only JSON matching {"score": integer from 0 to 5, "tags": array of strings}. + Score this candidate against the rubric: 5 requires Go and on-call experience. + Candidate: operated a Go checkout service and joined a weekly on-call rotation. , }}, } requestID := fmt.Sprintf "tag-%d", time.Now .UnixNano result, cost, err := classify apiKey, baseURL, payload if err = nil { panic err } var label classification if err := json.Unmarshal byte result.Choices 0 .Message.Content , &label ; err = nil { panic fmt.Errorf "model content failed JSON validation: %w", err } if label.Score < 0 || label.Score 5 || label.Tags == nil { panic "model content failed rubric validation" } record := ledgerRecord{requestID, tenantID, model, result.Usage.PromptTokens, result.Usage.CompletionTokens, cost} encoded, err := json.Marshal record if err = nil { panic err } fmt.Println string encoded } func classify apiKey, baseURL string, payload chatRequest chatResponse, string, error { body, err := json.Marshal payload if err = nil { return chatResponse{}, "", err } client := &http.Client{Timeout: 30 time.Second} for attempt := 0; attempt < 4; attempt++ { req, err := http.NewRequest http.MethodPost, strings.TrimRight baseURL, "/" +"/v1/chat/completions", bytes.NewReader body if err = nil { return chatResponse{}, "", err } req.Header.Set "Authorization", "Bearer "+apiKey req.Header.Set "Content-Type", "application/json" resp, err := client.Do req if err = nil { return chatResponse{}, "", err } data, readErr := io.ReadAll resp.Body resp.Body.Close if readErr = nil { return chatResponse{}, "", readErr } if resp.StatusCode == http.StatusTooManyRequests { time.Sleep retryDelay resp.Header.Get "Retry-After" , attempt continue } if resp.StatusCode < 200 || resp.StatusCode = 300 { return chatResponse{}, "", fmt.Errorf "classification request failed with %s: %s", resp.Status, strings.TrimSpace string data } var result chatResponse if err := json.Unmarshal data, &result ; err = nil { return chatResponse{}, "", err } if len result.Choices == 0 { return chatResponse{}, "", errors.New "classification response contained no choices" } return result, resp.Header.Get "X-Infrai-Cost-Usd" , nil } return chatResponse{}, "", errors.New "rate limit retries exhausted" } func retryDelay retryAfter string, attempt int time.Duration { if seconds, err := strconv.Atoi retryAfter ; err == nil && seconds = 0 { return time.Duration seconds time.Second } return time.Duration 1<<attempt time.Second } Select MODEL ID from /v1/models before deployment rather than copying an identifier from an article. For preflight capacity work, /v1/ai/tokens/count is the verified counting route; use it to compare prompt revisions before they multiply across every candidate and tenant. Those are the only extra control-plane calls this runbook needs. One caution: JSON parsing is only the first gate. In the sample, the application also bounds score and requires tags ; a production schema should reject unknown fields and constrain allowed tag values. Otherwise {"score":5,"tags": "excellent" } can be syntactically perfect while violating the hiring taxonomy. Run the held-out set against all shortlisted models and break results down by tenant, language, rubric version, and input-size bucket. The release dashboard needs schema-valid rate, rubric agreement against human-reviewed labels, p95 latency, input and output tokens, and cost per classified candidate. Do not merge these into a weighted “quality score”; separate signals make the rollback decision obvious. Set a canary window long enough to include the application's normal input mix, then promote only when the schema SLO and rubric acceptance threshold both hold inside the tenant budget. Capacity planning should cover the retry envelope too: with four attempts in the sample, the admission calculation must reserve room for retry traffic during a rate-limit period rather than assuming one request per candidate. HTTP 429 is load control, not permission to spin. Retries compound. Suppose the worker is admitted at 100 classifications per minute and the capacity model assumes exactly 100 upstream calls. A rate-limit interval can invalidate that assumption because each unfinished classification is eligible for another attempt; the code caps attempts at four and spaces them out, but the queue still retains the original work while retry demand arrives. The runbook therefore needs two separate ceilings: new classifications admitted per minute and total upstream attempts per minute. When the second ceiling is close, pause new admissions, let exponential backoff drain the retry set, and preserve enough worker slots for old requests to complete. This is a planning example, not a measured limit or a recommended production value; derive both ceilings from the provider limits and the application's error budget. The token ledger deserves its own reconciliation check. Sum call-level usage by tenant and compare it with the billing view for the same window; alert on missing tenant IDs, unknown model IDs, and accounting records without a completed classification. This isn't glamorous, but it catches attribution drift before finance discovers it at month-end. Keep the previous model ID in configuration and retain the exact prompt, schema, and rubric versions used by the canary. If schema-valid rate, rubric agreement, latency, or a tenant budget crosses its release threshold, route new work back to the previous model and quarantine affected results for review. Do not replay writes blindly; use the application request ID to prevent a candidate record from receiving the same classification twice. The final decision rule is narrow: choose the direct provider that wins the held-out test when governance already fixes the vendor or provider-specific controls matter; choose a portable chat contract when model choice is expected to change and per-call metadata makes tenant accounting easier. Re-evaluate on model or prompt changes. No logo gets a permanent exemption.