A support bot decides whether an incoming ticket is a refund request. That’s it. Ten words of classification work, and most stacks still route it through a full reasoning model: a few seconds of latency, a few cents of token spend, and a JSON parse at the end to pull out one word. Scale that to a few million tickets a month and the decision step alone is paying for a model that never needed to write a sentence.
That gap is why a new category of model showed up this fall, and why it’s worth asking which parts of your agent loop actually need a reasoning model and which ones just need a fast, typed answer.
In this article:
Strip the prompt engineering away and most agent loops make the same three kinds of calls over and over. Routing decides which tool, which subagent, or which branch of a workflow should handle the current input, the exact failure mode I wrote about in MCP tool design: why your agent keeps calling the wrong tool. Gating decides whether a guardrail, a policy check, or an input filter passes or blocks. Escalation decides whether the current case is simple enough to resolve automatically or needs to go to a bigger model, a human, or both.
None of those three decisions require prose. They require a label, a score, or a yes/no, picked from a known set of options. That’s a narrower problem than “have a conversation,” and for a long time the only tool available to solve it was a general-purpose LLM asked to behave narrowly through prompting. It worked, but it was never the efficient shape for the job, which is also the core argument in Why Your Agent Loop Costs More Than Its Model Price Tag: the sticker price on a model card is not what your loop actually spends per task.
In September, a startup called TypeSafe AI released Jev, a model that does not generate text at all. You send it a state and a set of typed questions, Jev evaluates them in one pass, and returns a choice, a score, or a probability with a calibrated confidence attached [1]. TypeSafe calls it a “System One” model, after Daniel Kahneman’s term for fast judgment, and it was built specifically to answer the kind of narrow questions a routing or gating step asks.
Jev went viral fast. Vercel reported that within 24 hours of listing it on their AI Gateway, nearly 13% of their paid teams had tried it, more than twice the adoption rate of any previous model launch on the platform [2]. That kind of uptake doesn’t happen because developers love a new model; it happens because a real cost was suddenly visible and addressable.
Three more vendors followed within weeks. Cloudflare open-sourced Clef and Clef-flash on October 1, Apache 2.0 licensed and hosted on Workers AI, with Clef adding image and video input on top of text [3]. AWS released Strands Decider 2B the same day, a roughly 2-billion-parameter open model built on a Qwen3.5 base with the usual text-generation head swapped for a compact “pointer head,” designed to run locally under 150 milliseconds [4]. And on October 6, OpenAI shipped the Decisions API, built on GPT-6 Luna, which it says returns decisions up to 10 times faster than calling the same model through its general Responses API [2]. OpenAI’s own Nikunj Handa credited Jev with “inspiring this whole thing,” and said the API wasn’t even on the roadmap four weeks earlier [2].
If you’re trying to pick between those four, that’s a separate question with its own tradeoffs on licensing, modality, and hosting, and I covered that comparison directly in Four Decision Models Shipped in Three Weeks. Choose Wisely. This piece is about a different question: given that these models exist, which steps in your loop should actually use one, and what do you do if you’re building on a model family that doesn’t have one yet. As of this writing, I could not find any official Anthropic announcement of a decision-only model along the lines of Jev, Clef, or Strands Decider. That’s worth flagging as an open gap rather than guessing at a roadmap. What Anthropic did ship, on October 7, is Claude Haiku 5.5, and it closes a meaningful part of that gap even without being purpose-built for it.
Haiku 5.5 dropped its price for prompts under 100,000 tokens to $0.10 per million input tokens and $0.50 per million output, down from Haiku 4.5’s $1 and $5, which Anthropic says covers roughly 90% of the requests the previous Haiku model handled [5]. It also introduced an effort control that lets you dial reasoning up or down per call instead of treating latency and cost as fixed. Neither of those is a decision-model feature on its own, but paired with Claude’s structured outputs, they get you most of the way there.
Structured outputs on the Claude Developer Platform compile your JSON schema into a grammar and constrain decoding to it, so a well-formed schema produces a guaranteed schema-valid response rather than a hopeful one [6]. That’s the exact property a routing or gating step needs: send Claude a predicate like “is this a refund request” with a boolean schema, and the response is a parsed boolean, not a paragraph you regex out of. The feature is supported on Haiku 5.5 as well as the larger Claude models, which is what makes this a real alternative and not just a workaround, and it’s the same mechanism I walked through in more depth in Claude API structured outputs: schemas, not hope.
The honest comparison: a dedicated decision model like Jev or Clef-flash is still faster and cheaper per call than Haiku 5.5 with structured outputs, because it skips token generation and the grammar-compilation step entirely. But if your stack is already built on Claude, routing that one narrow decision to Haiku 5.5 with a tight schema is a smaller integration lift than adding a second model provider, and it removes most of the token waste a full reasoning call would otherwise burn on a one-word answer.
Not every routing, gating, or escalation decision belongs on a decision-only model, and not every one needs a full reasoning model either. The split comes down to three questions asked at each decision point in the loop.
First, is the input a closed set of known options, or does it require synthesizing new information the model hasn’t seen phrased this way before? Jev, Clef, and Strands Decider all assume you can specify the exact choices or score range up front. If the decision requires actual reasoning about a novel case, a decision-only model will confidently return a wrong answer with a believable confidence score attached, which is a worse failure mode than a slow correct one.
Second, does the decision run often enough that the per-call savings compound into something that matters? AWS pitches Strands Decider for exactly this: routing, tool selection, context management, and guardrail enforcement are all steps that fire on every single turn of every agent run, not once per session [4]. A decision that runs once per conversation rarely justifies adding a new model to your stack.
Third, what happens when the decision is wrong? Gating decisions that block a harmful action can tolerate a cheap model with a conservative threshold, because the cost of a false block is a retry. Escalation decisions that route a case away from human review cannot, because the cost of a false negative there is a customer who never gets help. Put the cheap, fast model on the steps where being wrong is recoverable, and keep the expensive, careful model on the steps where it isn’t.
Every vendor in this category leads with a speed and cost multiple, and those numbers deserve more skepticism than they usually get. TypeSafe’s headline claim, 444.6 times cheaper and 193.6 times faster than an LLM-based workflow, comes from a single workflow the company designed and ran itself, with no independent lab reproducing it [7]. An independent analysis of launch-week user reports found a median reported speedup closer to 7 times and a median cost saving around 30 times, which is still significant but a fraction of the headline figure [1].
More importantly, “zero hallucinations” in TypeSafe’s own framing means the output matched the schema, not that the decision was correct. A model can return a perfectly valid, perfectly wrong label. Schema compliance is a floor, not a quality guarantee, and that caveat applies just as much to Haiku 5.5’s structured outputs as it does to Jev or Clef. Whatever you put on the decision layer, you still need an accuracy check against your own data before you trust it on traffic that matters, the same “run it on your own task first” advice that holds for picking between the four decision models themselves.
The pattern underneath all of this isn’t that reasoning models are obsolete for agent loops. It’s that most agent loops have been asking a reasoning model to do work that was never reasoning in the first place, and now there’s a cheaper, faster place to put it.
If you enjoyed the article and wish to show your support, make sure to: Your Agent Loop Doesn’t Need an LLM for Every Decision was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.