OpenAI's Decisions API can help applications organize feature requests, assess incident handoffs, and flag documentation that needs review. You supply evidence and define the question's possible answers. The model returns a choice, score, or probability that your code can use to suggest a next step.
The useful starting point is a judgment someone already makes repeatedly. Write down the evidence they need and what a correct answer would change in the workflow, then test whether the model makes that judgment on representative examples.
Copy link to headingWhich question type fits each use case? #
OpenAI calls its yes/no question type a predicate. Through AI SDK decisions, the corresponding type is boolean; both return the probability that the statement is true. Choice questions select an allowed option, and Score questions assess evidence against ordered levels.
These are workflow designs to evaluate against your own examples. Their usefulness depends on the question definitions and the evidence your application supplies.
Copy link to heading1. Match feature requests to roadmap themes #
Feature requests often describe a proposed solution without naming the underlying need. For example, "Let me send my saved view to a teammate" describes sharing an existing view, which may fit a collaboration theme.
Supply the request alongside your current theme definitions. Ask, "Which theme best matches the outcome this person wants?" Define collaboration around sharing work with others and reporting around producing or inspecting summaries. Include an outside_themes option for a clear request that doesn't fit, and needs_context when the goal is unclear.
The selected theme can become a suggestion beside the request. Product staff can correct it before using grouped requests to assess demand.
Copy link to heading2. Check incident handoffs for missing context #
"Rolled back, looks better" leaves the next responder guessing about the original impact and what still needs attention. An effective handoff states what customers experienced, describes the attempted mitigation and its outcome, and names the next action's owner.
Ask separate questions about those requirements. For example, a Boolean question can check whether customer impact is described, while a Score question assesses coverage of the whole handoff. The application can show these assessments beside the note, prompting its author to check the requirements and add missing context before transferring responsibility.
Copy link to headingTry a handoff check through AI Gateway
With Node.js 22.18 or later, install the packages for the experimental decision API. Use ai
npm install ai @ai-sdk/gateway
Follow the decision quickstart to configure server authentication with AI_GATEWAY_API_KEYVercel OIDC.
This example uses GPT-6 Luna Decisions on AI Gateway. It prints three assessments of a fictional handoff without changing the incident or notifying another responder.
import { experimental_decide as decide } from 'ai';import { gateway } from '@ai-sdk/gateway';
const handoff = `Checkout requests failed for about 10% of customers.Rolling back the payment adapter restored checkout success rates.Maya will watch the checkout error dashboard until 15:00 UTC andreopen the investigation if the error rate exceeds 2%.`;
try { const result = await decide({ model: gateway.decisionModel('openai/gpt-6-luna-decisions'), state: handoff, questions: { impactDescribed: { type: 'boolean', instructions: 'Does the note describe an effect on customers or their work? ' + 'An internal alert without a described customer effect does not count.', }, nextStep: { type: 'choice', instructions: 'Which next step does the note explicitly describe?', criteria: { investigate: 'A concrete diagnostic or repair action is the next step.', monitor: 'Observe a named signal after a mitigation or recovery.', clarify: 'The next step is absent, vague, or contradictory.', }, }, coverage: { type: 'score', instructions: 'Assess how completely the note documents customer impact, ' + 'attempted mitigation and its outcome, and a next action with an owner. ' + 'Judge only what is written; do not assume omitted information.', criteria: [ 'None of those handoff requirements is meaningfully documented.', 'Some requirements are documented, but at least one is missing.', 'All requirements are documented, including an owner for the next action.', ], }, }, maxRetries: 0, abortSignal: AbortSignal.timeout(30_000), });
console.log(JSON.stringify(result.answers, null, 2));} catch { console.error('Handoff assessment failed. Keep the note available for review.'); process.exitCode = 1;}
Save the example as review-handoff.mjs and run node review-handoff.mjs with your API key exported in the shell. If the key is in .env.local, add Node's --env-file=.env.local option before the filename.
For local OIDC, run vercel link to connect your working directory to a Vercel project, then vercel env pull .env.local to download its environment variables, including the OIDC token. Use the same Node option to load that file, and rerun the pull command when the token expires.
Read impactDescribed.probability as the probability that the note describes customer impact. The nextStep.choice value classifies the action written in the note. Neither answer establishes that the service has recovered.
The coverage rubric has indices 0, 1, and 2. Its score is a probability-weighted position on those levels and can fall between them. Display the score with the rubric and original text so the reviewer can inspect the judgment. The example has no automatic acceptance threshold; choose one only after testing reviewed handoffs, including notes with a convincing recovery claim but no owner.
Copy link to heading3. Identify contradictions between documentation pages #
Suppose an installation guide says a service reloads configuration automatically, but a troubleshooting page instructs users to restart it after the same change. An editor needs to know whether the pages disagree or describe different conditions.
Provide both passages with their product versions and deployment context. Ask which relationship applies: consistent, conflicting, or insufficient_context. Define conflict as incompatible instructions for the same supported situation. Differences between a hosted deployment and a local process should not count as a contradiction when each page states that scope.
Send suspected conflicts to the owner of the affected documentation. Include both source passages in the review item. The answer identifies a possible inconsistency; deciding which instruction is correct still requires implementation evidence or a verified procedure.
Copy link to heading4. Assess whether a migration guide covers required changes #
Migration instructions can mention a new configuration name while omitting what readers must do with an existing value. To check coverage, supply the guide with the actual migration requirements, including any conditions under which a step applies.
For a fictional SDK update, one requirement might be, "Move the timeout from the client constructor to each request, preserving its value." Ask how fully the guide explains that requirement, using levels such as absent, mentioned without actionable instructions, and explained with an applicable before-and-after example.
Define one question per requirement so a strong configuration section cannot hide missing instructions elsewhere. Each question can share the same guide text. Use the returned scores to prioritize editorial work, then execute the documented migration against a sample project to check that the instructions work.
Copy link to heading5. Decide whether to ask a follow-up question #
Before preparing a data export, an assistant may need a workspace and a date range. If the conversation says "Export last month's data" and the account has several workspaces, choosing one silently could produce the wrong file.
Resolve exact values in code first, including relative dates in the user's timezone and any structured workspace selection. Supply the remaining conversation context and ask, "Has the user unambiguously identified which workspace to export?" Define references such as "the same workspace as before" using the actual conversation history available to the application.
If the evidence is insufficient, your application can ask a prepared clarification question. Decisions API returns the assessment; a template or separate language-model call writes the follow-up. Check the user's access to the chosen workspace before starting the export, regardless of the model's answer.
Copy link to heading6. Check screenshots against submission requirements #
When reporting a failed deployment, users sometimes attach a project overview instead of the failure details. You could ask a Predicate question such as, "Does the screenshot visibly include both the deployment status and an error message?" When the returned probability is low, prompt the submitter to inspect the attachment before sending the report.
OpenAI's direct Decisions endpoint accepts image input alongside text. Send the screenshot as an inline base64 data URL with an input_image part and supply the visible-content requirements in the question. Gateway's OpenAI-compatible Decisions endpoint currently accepts text only, so this image workflow requires direct OpenAI access.
Test cropped images, small text, and screenshots of successful deployments as well as clear failures. Keep an option to submit for human review when the image is unreadable or the reported problem doesn't produce an error message.
Copy link to heading7. Triage GitHub pull requests by review needs #
Some changes need a compatibility review even when the diff is small. Renaming an exported configuration field, for example, can require callers to update their code.
Supply the changed code and relevant interface documentation, then ask which review track fits the visible change. An independent Predicate question can assess whether existing callers need migration guidance. Include a review outcome for diffs that don't provide enough context to judge the effect.
The pull request triage tutorial shows how to collect GitHub evidence and interpret the returned answers. Use those answers to organize review while keeping repository checks and reviewer approval responsible for merging the change.
Copy link to headingWhat should stay outside the model's decision? #
Fixed requirements belong in application code. Field presence, file-size limits, and permissions can be checked directly. Use a decision model where satisfying the requirement depends on interpreting the supplied content, such as whether an incident note explains the customer impact.
Probability describes the model's prediction about the stated question. Set acceptance rules using labeled examples and the consequences of an incorrect answer. Keep a completed assessment that requests clarification distinguishable from a failed request. Refused questions have no usable answer, and AI SDK rejects the call if any question is refused.
The API also cannot inspect material your application hasn't supplied. Documentation comparisons need the relevant passages and scope, while a migration review needs the requirements against which the guide should be assessed. Preserve that evidence with the result so a reviewer can correct the judgment.
Copy link to headingWhen should you compare OpenAI's Decisions API with Jev? #
For the text-based workflows above, you can also evaluate Jev through AI Gateway. The AI SDK example can use typesafe-ai/jev with the same question definitions, letting you compare both models against the same handoff notes and rubric.
If your application already uses Jev, include its current results as the baseline. Compare the judgments with reviewed examples and check how each model's probabilities affect your acceptance rules. Matching answer formats don't establish that an existing threshold will work equally well with both models.
The OpenAI Decisions API vs. Jev comparison covers input support, response formats, and confidence handling when choosing an integration.
Copy link to headingFrequently asked questions #
Copy link to headingWhich OpenAI Decisions API use case should I try first?
Start with a repeated review task for which you already have examples and agreed criteria, such as checking incident handoffs. Use the first integration to suggest assessments beside the original evidence, then compare its judgments with those of your reviewers before automating a follow-up action.
Copy link to headingCan one request assess several aspects of the same input?
Yes. OpenAI's Decisions API supports multiple independent questions about shared input. Give each question the evidence it needs; if a later question depends on an earlier answer, have your application make separate calls.
Copy link to headingCan the API explain what is missing or rewrite the input?
No. Its typed answers provide selections, scores, and probabilities rather than a written explanation or revision. Use individual questions to identify which requirement needs attention, then let a person or a separate generation step revise the content.
Copy link to headingDo these use cases require training a custom model?
You can start by defining questions and allowed answers in the request, without training a custom model. You still need representative examples with reviewed answers to assess whether the model and question definitions meet the workflow's requirements.
Copy link to headingDoes a low Boolean probability mean the answer is uncertain?
No. Low probability indicates that the stated condition is unlikely to be true. In the handoff example, that means the note probably does not describe customer impact; it does not indicate whether the real incident affected customers.