Chatbot AI Service Comparison: Reviewable Features for Candidate Scoring Pipelines

A developer outlines a design for candidate-scoring pipelines that treats chatbot services like OpenAI ChatGPT, Claude, and Gemini as interchangeable components behind a single adapter rather than as the hiring decision itself. The approach returns four bounded criterion states (meets, partial, does_not_meet, insufficient_evidence) with evidence, rationale, and confidence, validates and rejects malformed output, and keeps provider-specific request and response handling isolated so a request can move between providers without changing its meaning. The developer recommends measuring human agreement, invalid-output rate, latency, and token use before letting a service comparison drive a product decision.

Use a chatbot service comparison as a way to choose a replaceable scoring component, not as the hiring decision. For a media company scoring candidates against a job rubric, compare OpenAI ChatGPT, Claude, and Gemini by the contract around the model: preserved evidence, explicit uncertainty, rejected malformed output, and a request that can move between providers without changing its meaning. The tempting implementation sends a resume and rubric to one service, asks for a score from 1 to 10, and stores the number. That ships quickly. It is difficult to audit and painful to replace. A better first version produces four bounded states per criterion, validates the result, and keeps each provider request behind one adapter. Measure human agreement, invalid-output rate, latency, and token use before turning a service comparison into a product decision. This is the useful answer to the OpenAI ChatGPT versus Claude versus Gemini question. Their names belong in the test matrix, not in the hiring rule. The score is not the evidence. A score without evidence is a misleading number. Ask for a criterion-level result with one of meets , partial , does not meet , or insufficient evidence , plus evidence, a rationale, and confidence with a defined meaning. Confidence describes evidence strength; it does not promise job performance. For a senior audio producer, a rubric might require a relevant project and role, an editorial decision, and a cross-functional example. The final score can be a deterministic function of those states. The model extracts and classifies evidence; application code applies weights and decides when a human must review. If a prompt revision changes a weight, the stored score should not change unless the scoring version changes too. type CriterionStatus = | "meets" | "partial" | "does not meet" | "insufficient evidence"; type CriterionResult = { criterionId: string; status: CriterionStatus; evidence: string ; rationale: string; confidence: number; }; type CandidateAssessment = { rubricVersion: string; results: CriterionResult ; needsHumanReview: boolean; }; Reject empty evidence for meets and partial , confidence outside 0 through 1 , and unknown criterion IDs. Reject extra fields if storage cannot preserve them. A fallback that stores half-valid JSON creates a quiet audit problem. Portability is usually described as moving an API call. The harder move is preserving semantics. OpenAI, Claude, and Gemini can sit behind a common interface, but request shape, response handling, model lifecycle, safety controls, and structured-output behavior still need separate treatment. A shared TypeScript function does not erase those boundaries. The adapter accepts domain data and returns the domain contract. It should not leak a vendor response object into scoring or persistence. This boundary is where a small implementation earns its keep. The scoring worker can retry a transport failure without knowing which service produced it, while the reviewer can see that an assessment was retried and still inspect the same evidence fields. Persistence does not need a new migration for every provider-specific response property. The trade is a little adapter code up front, paid back whenever a model identifier, response format, or operational policy changes. Keep the interface boring. type ScoreRequest = { rubricVersion: string; rubric: string; candidateText: string; }; type ScoreResponse = { assessment: CandidateAssessment; metadata: { provider: string; model: string; requestId?: string }; }; interface CandidateScorer { score request: ScoreRequest : Promise<ScoreResponse ; } Each implementation translates the request and normalizes the response. The interface stays boring. That makes a provider swap a controlled experiment instead of a rewrite of persistence, review screens, and background jobs. The trade-off is real: a lowest common denominator can hide useful capability exposed by one service. Use an explicit capability record rather than provider-specific branches throughout the application. An adapter can report whether it supports the response mode required by the current contract, and the job can stop before a candidate receives an assessment. Compare the same five things for OpenAI, Claude, and Gemini: contract support, refusal behavior on sensitive input, usable context for the actual resume set, latency distribution, and evidence extraction. Do not compare a feature list to a hiring outcome. Start with a fixed set containing easy matches, clear non-matches, sparse resumes, contradictory dates, and adjacent experience. A set of 30 reviewed examples can expose contract errors; it cannot establish broad model quality. Keep it versioned and remove identifying candidate data from fixtures. Run the same rubric and prompt versions through each adapter. Distinguish three failures: no response, invalid contract, and a valid assessment that disagrees with the human label. Transport errors belong in a queue policy. Invalid structure belongs in adapter telemetry and a bounded retry or review path. Substantive disagreement belongs in rubric review. Retrying the third case only makes the system slower. Record a correlation ID, rubric version, prompt version, provider, model, validation outcome, and latency. Avoid logging full resumes by default. Candidate material is sensitive, and a debug log can become a second, poorly governed copy of the hiring dataset. | Observation | Interpretation | Action | |---|---|---| | High contract failure rate | Adapter or response constraint is unreliable | Fix normalization before rollout | | Low contract failure, low reviewer agreement | Rubric or evidence instruction is weak | Review criteria and examples | | Similar agreement, different latency | Operational trade-off | Test queue capacity and reviewer needs | | Similar quality, different portability effort | Adapter boundary is incomplete | Move provider details downward | A hiring rubric has its own vocabulary and distribution. The right comparison is the one that survives the application’s real inputs. Use an asynchronous path. A reviewer submits a candidate, a job records rubric and prompt versions, the scorer returns a validated assessment, and the review interface shows evidence before any aggregate score. A timeout produces a pending state, not a fabricated zero. An invalid response produces a reviewable error state, not an empty assessment. Keep the decision policy outside the model. needsHumanReview can be true when a criterion is insufficient evidence , when candidate text contains conflicting dates, or when validation required a retry. The exact policy belongs to the hiring workflow and should be tested like any other business rule. Prompt injection is part of this design problem. A resume can contain instructions aimed at the model rather than facts about the candidate. Treat candidate text as untrusted data, delimit it clearly, and instruct the scorer to extract evidence rather than follow commands found in the text. OWASP’s LLM guidance is a useful threat-modeling reference; the relevant control is the boundary between instructions and imported content. The practical decision is to choose the service that passes the same contract and review set for the workload you actually have, then keep the adapter replaceable. Do not let a chatbot name become a hiring rule. A portable pipeline leaves room to change providers, prompts, or review thresholds without losing the evidence trail that makes a score defensible.