Stop Spending a Full LLM Call on a Yes or No Question: OpenAI's Decisions API, Explained

OpenAI has made its Decisions API generally available, a GPT-6 Luna-powered endpoint that returns bounded answers instead of generated text. The API supports three result types — predicate (a 0-1 probability), choice (a selection with probabilities and confidence), and score (a probability-weighted rubric average) — and is priced at $0.10 per million input tokens, $0.01 cached, with no output token charges. OpenAI says it is up to 10x faster than querying Luna through the Responses API, and Box reported routing certificate-of-insurance documents about 5x faster with the same accuracy while sending low-confidence cases to human reviewers.

Count the decisions inside any agent you have shipped. Is this ticket billing or technical? Should the agent retry, continue or ask for clarification? Is this image damaged? Is the caller angry enough to escalate? None of these need a paragraph of reasoning. They need one answer from a short list. Yet most of us send them through the same chat completion endpoint we use for writing emails, pay for the output tokens, parse the JSON, and add a retry for when the model gets creative. On October 22, OpenAI made the Decisions API generally available, and it is built for exactly this OpenAI https://openai.com/index/decisions-api/ . Here is what it does, the three answer types, and the patterns worth stealing even if you never call it. Decisions runs on GPT-6 Luna, but instead of generating text it answers questions with bounded answers. OpenAI says it is up to 10x faster than asking Luna the same thing through the Responses API. Since the beta it has gained prompt caching, native JSON inputs, images inside instructions, up to 400 questions per request, and US and EU inference residency. The pricing is the interesting part: $0.10 per million input tokens, $0.01 cached, and no output token charges at all. For a router that reads a 600 token ticket, that is about six thousandths of a cent per decision before caching. You define a question and its allowed answers, pass context as text, images or JSON, and get back one of three result shapes: | Type | Use it for | You get | |---|---|---| | predicate | Is a condition true? "Does this photo show visible damage?" | A probability from 0 to 1 | | choice | Pick one of your options "continue", "retry", "ask" | The choice, plus probabilities and confidence | | score | Rate against an ordered rubric "damage severity 0 to 4" | A probability-weighted average of the level index | The shape matters more than it looks. A choice with probabilities is something you can threshold . A score that is a weighted average is something you can sort . A free-text answer from a chat model is neither until you parse it. The most useful thing you get back is not the answer. It is how sure the model is. Box shared that in its evaluation the Decisions API picked routes about 5x faster on certificate of insurance documents with the same accuracy as Responses, and sent lower-confidence cases to human reviewers. That is the pattern: cheap decision first, expensive path only when needed. php def route ticket: dict - str: d = decide thin wrapper around the Decisions endpoint question={ "type": "choice", "name": "queue", "instructions": "Which team should handle this support ticket?", "choices": {"value": v} for v in "billing", "technical", "account", "sales" , }, context=ticket, native JSON, no prompt templating if d.confidence = 0.85: return d.choice fast path, most traffic if d.confidence = 0.55: return escalate to large model ticket, hint=d.choice slow path return "human review" the model knows it does not know Tune the two thresholds on a labelled sample, not by feel. The first one controls cost, the second controls how much lands on people. Up to 400 questions per request means you can stop making serial calls for related checks. Moderating a listing? Ask "contains contact details?", "price looks like a typo?", "photo matches category?" and "language is English?" in one request against one cached context. One round trip instead of four, and the shared context is billed once. checks = {"type": "predicate", "name": "has contact info", "instructions": "Does the listing text include a phone number or email?"}, {"type": "predicate", "name": "price suspicious", "instructions": "Is the price implausible for this item category?"}, {"type": "score", "name": "photo quality", "instructions": "Rate photo quality.", "levels": "unusable", "poor", "fine", "good", "excellent" }, result = decide many checks, context=listing This is the part I find most exciting. Decisions now plugs into GPT-Live sessions as a trigger on transcript events. While the caller is still speaking, the app can decide whether to ask a clarifying question, route, or escalate, without pausing the conversation to run a full model turn. OpenAI's demo uses it to pick an avatar's facial expression from the live transcript: { "type": "decisions", "event type": "session.input transcript.delta", "decisions": { "questions": { "type": "choice", "name": "reaction", "instructions": "Choose an appropriate reaction for the avatar based on the user's latest message.", "choices": { "value": "neutral" }, { "value": "happy" }, { "value": "sad" }, { "value": "concerned" } } } } Swap "reaction" for escalate , needs human or intent , and you have the side channel every voice agent needs: a continuous, cheap read on the conversation that never interrupts it. Most agent cost is not in the clever steps. It is in the dozens of small, boring classifications around them. Pulling those out into a dedicated, structured, thresholded layer is good architecture whichever vendor you use: it makes agents faster, cheaper and easier to debug, because every routing choice now has a name, a confidence and a log line. We design agent architectures like this at Axionry https://axionry.com , an AI product development studio. If you are planning an agent and want to know what it will cost to run, try our AI product cost estimator https://axionry.com/tools/ai-product-cost-estimator , or see how we build on our AI product development https://axionry.com/services/ai-product-development page.