{"slug": "stop-spending-a-full-llm-call-on-a-yes-or-no-question-openai-s-decisions-api", "title": "Stop Spending a Full LLM Call on a Yes or No Question: OpenAI's Decisions API, Explained", "summary": "OpenAI has made its Decisions API generally available, a GPT-6 Luna-powered endpoint that returns bounded answers instead of generated text. The API supports three result types — predicate (a 0-1 probability), choice (a selection with probabilities and confidence), and score (a probability-weighted rubric average) — and is priced at $0.10 per million input tokens, $0.01 cached, with no output token charges. OpenAI says it is up to 10x faster than querying Luna through the Responses API, and Box reported routing certificate-of-insurance documents about 5x faster with the same accuracy while sending low-confidence cases to human reviewers.", "body_md": "Count the decisions inside any agent you have shipped. Is this ticket billing or technical? Should the agent retry, continue or ask for clarification? Is this image damaged? Is the caller angry enough to escalate?\n\nNone of these need a paragraph of reasoning. They need one answer from a short list. Yet most of us send them through the same chat completion endpoint we use for writing emails, pay for the output tokens, parse the JSON, and add a retry for when the model gets creative.\n\nOn October 22, OpenAI made the Decisions API generally available, and it is built for exactly this ([OpenAI](https://openai.com/index/decisions-api/)). Here is what it does, the three answer types, and the patterns worth stealing even if you never call it.\n\nDecisions runs on GPT-6 Luna, but instead of generating text it answers questions with bounded answers. OpenAI says it is up to 10x faster than asking Luna the same thing through the Responses API. Since the beta it has gained prompt caching, native JSON inputs, images inside instructions, up to 400 questions per request, and US and EU inference residency.\n\nThe pricing is the interesting part: **$0.10 per million input tokens, $0.01 cached, and no output token charges at all.** For a router that reads a 600 token ticket, that is about six thousandths of a cent per decision before caching.\n\nYou define a question and its allowed answers, pass context as text, images or JSON, and get back one of three result shapes:\n\n| Type | Use it for | You get | \n|---|---|---|\n| `predicate` | Is a condition true? (\"Does this photo show visible damage?\") | A probability from 0 to 1 | \n| `choice` | Pick one of your options (\"continue\", \"retry\", \"ask\") | The choice, plus probabilities and confidence | \n| `score` | Rate against an ordered rubric (\"damage severity 0 to 4\") | A probability-weighted average of the level index | \n\nThe shape matters more than it looks. A choice with probabilities is something you can **threshold**. A score that is a weighted average is something you can **sort**. A free-text answer from a chat model is neither until you parse it.\n\nThe most useful thing you get back is not the answer. It is how sure the model is. Box shared that in its evaluation the Decisions API picked routes about 5x faster on certificate of insurance documents with the same accuracy as Responses, and sent lower-confidence cases to human reviewers.\n\nThat is the pattern: cheap decision first, expensive path only when needed.\n\n``` php\ndef route(ticket: dict) -> str:\n    d = decide(  # thin wrapper around the Decisions endpoint\n        question={\n            \"type\": \"choice\",\n            \"name\": \"queue\",\n            \"instructions\": \"Which team should handle this support ticket?\",\n            \"choices\": [{\"value\": v} for v in [\"billing\", \"technical\", \"account\", \"sales\"]],\n        },\n        context=ticket,  # native JSON, no prompt templating\n    )\n    if d.confidence >= 0.85:\n        return d.choice                  # fast path, most traffic\n    if d.confidence >= 0.55:\n        return escalate_to_large_model(ticket, hint=d.choice)  # slow path\n    return \"human_review\"                # the model knows it does not know\n```\n\nTune the two thresholds on a labelled sample, not by feel. The first one controls cost, the second controls how much lands on people.\n\nUp to 400 questions per request means you can stop making serial calls for related checks. Moderating a listing? Ask \"contains contact details?\", \"price looks like a typo?\", \"photo matches category?\" and \"language is English?\" in one request against one cached context. One round trip instead of four, and the shared context is billed once.\n\n```\nchecks = [\n    {\"type\": \"predicate\", \"name\": \"has_contact_info\", \"instructions\": \"Does the listing text include a phone number or email?\"},\n    {\"type\": \"predicate\", \"name\": \"price_suspicious\", \"instructions\": \"Is the price implausible for this item category?\"},\n    {\"type\": \"score\", \"name\": \"photo_quality\", \"instructions\": \"Rate photo quality.\",\n     \"levels\": [\"unusable\", \"poor\", \"fine\", \"good\", \"excellent\"]},\n]\nresult = decide_many(checks, context=listing)\n```\n\nThis is the part I find most exciting. Decisions now plugs into GPT-Live sessions as a trigger on transcript events. While the caller is still speaking, the app can decide whether to ask a clarifying question, route, or escalate, without pausing the conversation to run a full model turn. OpenAI's demo uses it to pick an avatar's facial expression from the live transcript:\n\n```\n{\n  \"type\": \"decisions\",\n  \"event_type\": \"session.input_transcript.delta\",\n  \"decisions\": {\n    \"questions\": [{\n      \"type\": \"choice\",\n      \"name\": \"reaction\",\n      \"instructions\": \"Choose an appropriate reaction for the avatar based on the user's latest message.\",\n      \"choices\": [{ \"value\": \"neutral\" }, { \"value\": \"happy\" }, { \"value\": \"sad\" }, { \"value\": \"concerned\" }]\n    }]\n  }\n}\n```\n\nSwap \"reaction\" for `escalate`, `needs_human` or `intent`, and you have the side channel every voice agent needs: a continuous, cheap read on the conversation that never interrupts it.\n\nMost agent cost is not in the clever steps. It is in the dozens of small, boring classifications around them. Pulling those out into a dedicated, structured, thresholded layer is good architecture whichever vendor you use: it makes agents faster, cheaper and easier to debug, because every routing choice now has a name, a confidence and a log line.\n\nWe design agent architectures like this at [Axionry](https://axionry.com), an AI product development studio. If you are planning an agent and want to know what it will cost to run, try our [AI product cost estimator](https://axionry.com/tools/ai-product-cost-estimator), or see how we build on our [AI product development](https://axionry.com/services/ai-product-development) page.", "url": "https://wpnews.pro/news/stop-spending-a-full-llm-call-on-a-yes-or-no-question-openai-s-decisions-api", "canonical_source": "https://dev.to/neeraj_sharma_757fbb49aa7/stop-spending-a-full-llm-call-on-a-yes-or-no-question-openais-decisions-api-explained-31c1", "published_at": "2026-10-25 21:05:52+00:00", "updated_at": "2026-10-25 21:12:54.775643+00:00", "lang": "en", "topics": ["ai-products", "ai-tools", "large-language-models", "ai-agents", "developer-tools"], "entities": ["OpenAI", "GPT-6 Luna", "Decisions API", "Responses API", "Box"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/stop-spending-a-full-llm-call-on-a-yes-or-no-question-openai-s-decisions-api", "markdown": "https://wpnews.pro/news/stop-spending-a-full-llm-call-on-a-yes-or-no-question-openai-s-decisions-api.md", "text": "https://wpnews.pro/news/stop-spending-a-full-llm-call-on-a-yes-or-no-question-openai-s-decisions-api.txt", "jsonld": "https://wpnews.pro/news/stop-spending-a-full-llm-call-on-a-yes-or-no-question-openai-s-decisions-api.jsonld"}}