← Blog Decision models have acquired competitors, integrations, and production customers in a matter of weeks. Their appeal is straightforward: many applications need a judgment, and generating a paragraph is an expensive way to deliver one. The harder question is what happens when that judgment takes over part of a running application.
In its first two weeks on Vercel AI Gateway, Jev reached 35.3% of paid teams, the fastest adoption of any model in the gateway's history. It also received more requests than any other model during September, despite arriving halfway through the month. Yet its share of September spending was just 0.15%. These are Vercel's platform-specific figures, with spending estimated at list prices; the adoption window and monthly spending window differ. Vercel's October Production Index
Those numbers establish considerable interest and usage. They cannot tell us how many teams replaced an existing production workflow, or whether the replacement improved its outcome.
That distinction frames this survey. As of November 10, 2026, the public record contains products developers can use, organizations reporting integrations, and measurements of particular workloads. Keeping those kinds of evidence separate makes the emerging market easier to understand.
What is a decision model? #
TypeSafe AI introduced Jev on September 15 with an interface organized around application state and typed questions. Its three primitives are Choice, for selecting from supplied alternatives; Noul, for estimating whether a proposition holds; and Score, for evaluating an ordered rubric. TypeSafe describes a parallel output mechanism that returns decisions and probabilities rather than generating a free-form answer token by token. Jev's launch
For a support application, that could mean supplying a customer message and the available queues, then asking which queue should receive it. The output is already shaped for a software branch. Classification is not new. The attraction of these general decision interfaces is that developers can describe different tasks at request time without training a separate classifier for each one. The recent commercial wave makes that capability available through specialized models and APIs.
“System One” is a useful market label, but it is not a single architecture or a guarantee that a task is easy. A yes-or-no question can require substantial reasoning. Constraining an answer to permitted values prevents an invalid label; it does not prevent choosing the wrong label. Nor does returning a probability establish that the probability is reliable on a particular application's traffic.
The landscape is already broader than Jev #
The following is a representative map, not a performance ranking. It describes provider-published capabilities; comparable application-level evidence is much thinner.
| Provider and offering | Distinction |
|---|---|
| **TypeSafe AI**[Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) | Hosted Choice, Noul, and Score decisions. |
| OpenAIDecisions API | Predicate, choice, and score; currently supports GPT-6 Luna. |
| **Microsoft**[Microsoft-Decision-1](https://commandline.microsoft.com/microsoft-decision-1-model-foundry/) | Routing, classification, verification, and workflow control. |
| **Cloudflare**[Clef family](https://blog.cloudflare.com/clef-faster-cheaper-multimodal/) | Open weights and Workers AI; Clef-omni adds multimodal inputs. |
| **Perplexity**[pplx-decider](https://www.perplexity.ai/changelog/ultrafast-computer-models-decisions-api-and-connector-controls) | Open 4B and 27B models; multiple questions per pass. |
| **Liquid AI**[d1 and d1-ultra](https://www.liquid.ai/blog/d1-ultra) | Text and image decisions; a stated emphasis on calibration. |
| Nace AIDrex family | Open models including Drex Max; a separate finance offering . | | FastinoGLiNER2.5-Decide andGLiDE | Compact classification with GLiNER; 27B adaptive reasoning with GLiDE. | | SubquadraticSubQ Snap 1.1 | Sparse attention; advertised context up to 384K tokens. | | SalesforceHyperClassifier / Koa Decisions | Agentforce decision inference; broader API expansion announced. |
Three differences matter more than a shared label.
The input can change the application. Cloudflare's multimodal offering addresses decisions over media, while SubQ Snap targets much longer contexts. Neither advertised modality support nor context length establishes accuracy on a customer's evidence. They widen the set of workloads worth testing.
Deployment changes the economics. Open weights make local or private serving possible, but hardware, utilization, and operations still cost money. A hosted price per request and a self-hosted model size are not directly comparable cost measurements.
The execution mechanism differs. Some offerings emphasize direct readout; others incorporate adaptive reasoning. Compatible endpoints can reduce integration work, but compatibility is specific to those endpoints. It does not make the whole table interchangeable or establish equivalent behavior.
Adjacent specialist models widen the picture further. ScaleDown's LangChain integration covers context compression, extraction, and summarization through middleware. These operations can reduce work inside an agent, but summarization still produces text. It belongs in the broader specialization story without being collapsed into finite-choice inference.
Reading the cases: six engineering problems #
The cases below are grouped by the work the integration must get right. Each separates what was attempted, what happened, and what the result suggests for an engineer. Production accounts, shadow comparisons, demos, and research experiments have different evidential weight; none of these measurements was independently reproduced for this article.
- Filtering: deciding what expensive work can be skipped.
- Control: placing a fast decision inside an explicit workflow.
- Contracts: preserving everything the old call actually did.
- Evaluation: testing realistic inputs against a strong baseline.
- Uncertainty and trust: deciding when a judgment may cause an action.
- System economics: measuring recovery, total cost, and the path that actually runs.
This is a selected case collection, not a census of deployments. It cannot estimate the industry's migration success rate. The engineering lessons are our interpretation of the cited evidence.
1. Filtering: cheap decisions, expensive misses #
Jack & Jill — preserving candidates while reducing screening cost
Evidence: supplier-published customer case study; reported production deployment.
Attempt. Replace Gemini 3.1 Flash Lite in an initial recruiting screen that narrows roughly 6,000 candidates to 2,000 before more detailed review. The team deployed Jev in ten days and subsequently expanded it to more than 15 workflows.
Result. Across 150 roles, recall of candidates later selected by hiring managers was 93.9% for Gemini and 94.6% for Jev. The difference's 90% confidence interval was −0.4 to +1.8 percentage points. Reported cost fell 88%, and median screening time per role fell from 20.3 to 10.3 seconds. Customer study
Friction and lesson. The important failure is discarding a candidate the later process would want. The evaluation therefore follows candidates through the filter. Similar measured recall supports this specific substitution, but historical hiring-manager selections are a proxy, not a measure of hiring quality or proof of equivalent performance everywhere.
Modem — deciding whether an agent needs to run
Evidence: product announcement; reported shipped feature, limited public evaluation.
Attempt. Smart Triggers applies exact filters such as source, author, and state, then uses Jev to evaluate a user-written semantic condition before starting an agent.
Result. Modem reports evaluations under one second at approximately $0.0001 per event. The announcement does not supply an aggregate savings study or a false-negative rate. Implementation account
Friction and lesson. The economic mechanism is clear: avoid a whole agent run. The missing measurement is equally clear: how often does the gate suppress a run that should have happened? A fast trigger needs an evaluation set of important missed events, not just examples it correctly ignores.
SecureWorks-GROUP repository — a reread gate that stayed too risky
Evidence: public repository migration notes; shadow comparison, then disabled. The repository name does not establish affiliation with the similarly named security company.
Attempt. Use Jev to judge whether context had changed enough to need an expensive full reread.
Result. The disabling commit records 68% agreement across 1,626 comparisons. More revealingly, Jev labeled 382 of 1,307 readings that the full reader considered changed as unchanged: approximately 29%, against a stated 2% tolerance. The gate never actually skipped the reread, so this deployment produced no skip savings. Disabling migration
Friction and lesson. Overall agreement concealed the costly direction of disagreement. These are developer-reported counts against the incumbent, not independently adjudicated truth. Even so, they explain why the team withheld authority to skip work.
Pattern: evaluate the work you suppress. A filter's value depends on both the cost avoided and the consequences of false negatives. Shadow mode can measure disagreement before the system gives the filter authority.
2. Control: the surrounding software still does the work #
Browserbase Stagehand — selecting an action is only one step
Evidence: engineering account and early component benchmark.
Attempt. Move parts of browser interaction into Jev. Stagehand constructs candidates from interactable accessibility nodes, classifies the requested action, prepares arguments and context, and asks Jev to select a target. A separate match check and a confidence threshold determine whether to execute or fall back to an LLM.
Result. Early testing reported median act latency falling from 1.97 to 0.46 seconds. This measures one browser operation, not a complete browsing task. The authors also describe trying standalone Jev agents without endorsing their production readiness. Browserbase's implementation
Friction and lesson. A model cannot select the correct target if the candidate builder omitted it. Nor does selecting “fill” supply the correct value. Candidate construction, argument preparation, a “no suitable target” outcome, and fallback are part of the implementation being evaluated.
Box Extract — routing a document still leads to human review
Evidence: demonstration, not production workflow software.
Attempt. Extract insurance requirements and submitted document data, then ask the Decisions API to select one of three routes. Code validates the route; low confidence or an invalid value sends the case to specialist review.
Result. Every route creates a human review task, including the route for apparently complete material. Box explicitly describes production work still needed, including durable state, safe retries, and monitoring. Demo architecture
OpenAI separately reports roughly fivefold faster routing with equivalent measured accuracy on 30 cases. That is a small routing evaluation, not a fivefold improvement in document processing or evidence of automated approval. Launch evaluation
Friction and lesson. Extraction quality and document association constrain the decision before inference begins. The useful architectural boundary is visible: the model recommends a route, code enforces the allowed transition, and a person reviews the case.
Salesforce Agentforce — a router needs an operational lifecycle
Evidence: vendor-reported production integration and engineering account.
Attempt. Use HyperClassifier for Agentforce subagent routing, with Koa Decisions extending the approach. Salesforce reports illustrative routing measurements around 26 milliseconds versus 1,446 milliseconds for a general-purpose model. These are vendor measurements, not a standardized cross-provider benchmark. Deployment and measurements
Friction and lesson. Salesforce's engineering account includes prompt drift, organization-level rollout controls, regional fallback, dashboards, and rollback. Its validation also uses model consensus, which is not independent human ground truth. The integration is a maintained routing service, not simply a smaller model behind an endpoint. Engineering account
Other announcements extend the adoption picture without supplying equivalent detail. OpenAI reports Sierra using Decisions in live conversation orchestration with roughly threefold lower latency and 90% lower per-call cost; Datadog reports faster command-risk scoring; Ramp reports faster, cheaper attribution. Speak's reported next-action improvement is a test, not an established production outcome. Those accounts omit enough evaluation detail that they should remain testimonials here. Customer announcements
Workday's Sana announcement likewise credits Jev in adaptive interfaces while retaining business context, permissions, and action confirmation. It does not substantiate a specific Jev latency improvement.
Pattern: assign authority explicitly. The candidate list, arguments, permissions, state transitions, and recovery policy belong to the application contract. A finite set of outputs helps express that contract; it does not implement all of it.
3. Contracts: the old call may do more than choose #
Audiala — a good classification probe missed the router's job
Evidence: developer's production rollback and follow-up experiments.
Attempt. Replace the main intent router after tuning Jev to 27 correct classifications out of 28 examples at roughly a quarter of the call cost.
Result. The production change was rolled back after 23 minutes. The incumbent also resolved cities and conversational references and supplied tool permissions. Initial wiring even targeted an outdated, rarely used route. Later testing on 65 scenarios repeated three times yielded 121/195 passing runs for the incumbent and 117/195 for Jev. Router latency improved, but complete-turn latency did not improve consistently across runs. Migration account
Friction and lesson. The small probe was tuned on the same examples used to judge it, and “correct intent” covered only part of the behavior. Audiala retained narrower reranking with fallback; its small quality study had input differences between systems and Jev was slower than the incumbent reranker. That supports a selective integration, not a general quality or speed ranking. First identify every responsibility of the actual serving path, then evaluate the replacement against that whole contract.
Jev-LiteMAS — routing labels lost handoff and repair information
Evidence: controlled research evaluation, not a production migration.
Attempt. Replace multi-agent routing and artifact review with bounded decisions. The researchers identified missing generative outputs: planners also wrote handoff instructions, while reviewers provided repair guidance.
Result. Their hybrid forwards accumulated expert artifacts and retains an LLM reviewer for escalations. Across four benchmarks and two backbones, it reduced generative-LLM tokens by 12.7–54.6% versus the LAS multi-agent baseline, with accuracy differences within 1.34 percentage points; latency improved in seven of eight settings. Those token counts exclude decision-model tokens. Removing substantive intermediate content from the router substantially worsened results in an ablation. Paper and ablations
Friction and lesson. The consumer may need an explanation, transformed data, or corrective instructions even when the apparent output is a label. Preserving that information can require changing the workflow. The experimental success belongs to the resulting hybrid, not to a label-for-text swap.
Pattern: inventory what downstream code consumes. Read the call sites, intermediate artifacts, and recovery paths. A replacement is incomplete if it returns the right label but removes evidence or instructions the next step needs.
4. Evaluation: realistic traffic and ordinary code are hard baselines #
Ahrefs Reviewotron — keep the gate, reject the ranking idea
Evidence: public experiment reports using captured review data; replay estimates, not a live randomized trial.
Attempt. Use Jev both to rank candidate findings for validation and to skip unnecessary review routes.
Result. Ranking achieved AUC 1.0 on 16 authored cases, then only about 0.71–0.74 on 26 captured candidates, with five repetitions in each evaluation. The team rejected that optimization. Labels came from validator outcomes; the underlying private review corpus is not included. Ranking experiment
The separate routing replay covered 14 reviews and 160 files, retaining all eight confirmed finding classes and all 20 captured findings while consistently skipping eight routes. Estimated net analysis savings were $1.84–$2.35 per replay of the 14 reviews, based on captured costs. Routing experiment
Friction and lesson. Different uses of the same model deserve different acceptance decisions. Authored examples overstated ranking quality; the narrower gate looked useful on replay. Neither repeated runs nor perfect retention on a small captured set establishes future recall.
Coroot and Taskelith — the incumbent can be a rule
Evidence: synthetic anomaly benchmark and small code-review shadow evaluation.
Attempt. Coroot tested whether Jev should page an engineer for latency anomalies. Taskelith tested replacing deterministic review-complexity tiers.
Result. On Coroot's 31 synthetic scenarios, a burn-rate rule scored 30/31, tuned Jev 29/31, and default Jev 27/31. The tuned system still misclassified spike patterns; thresholds and hold behavior were tuned on the same dataset. Coroot experiment
Taskelith found agreement on 48 of 50 changes, but the model's tiers did not better separate changes that needed a third review pass. It kept the rule. That measure was a limited proxy for quality, not a deployment outcome. Taskelith decision
Friction and lesson. A model that reproduces an existing rule adds a dependency unless it improves a relevant outcome. Include ordinary code in the comparison, and separate calibration data from the examples used to claim improvement.
Bocconi's Jev study — equivalent presentations can change the answer
Evidence: controlled prompt experiments on Jev 1.13.0.
Attempt. Test reasoning and judgment using familiar problems, adapted scenarios, and reordered answer options.
Result. A familiar conjunction problem was answered correctly on 500/500 calls; an adapted version was answered incorrectly on 500/500. Across 30 further scenarios, average accuracy differed by answer order: 39.79% when the conjunction appeared first versus 52.29% second. Repeated calls are not independent real-world tasks, and the paper also reports stronger results on other tests. Study
Friction and lesson. The presentation is part of the effective behavior. Test option permutations, equivalent wording, and evidence ordering. Record probability changes near an execution threshold as well as whether the winning label changes.
Pattern: earn the claim on a separate evaluation. Use captured difficult cases, important error categories, and a credible incumbent. Repeating a small prompt set measures repeatability on that set; it does not create broader task coverage.
5. Uncertainty and trust: a confident judgment is not permission #
Memory retirement — “suggested” is not “superseded”
Evidence: a research evaluation of memory-retirement decisions; no deployed memory system.
Attempt. Decide whether new information should retire an older memory. The hard distinction was between an established change, a suggested change, and compatible information.
Result. Of 97 test pairs needing confirmation, binary Jev prematurely retired 38. A three-way choice reduced that to 15, but detection of established changes also fell from 67/72 to 56/72. In an exploratory 176-pair holdout comparison, a reasoning-model cascade matched a stricter standalone threshold. Annotation provenance and limited human adjudication constrain the evidence. Suggested Is Not Superseded
Friction and lesson. More conservative behavior trades one error for another. A pending-evidence state can preserve uncertainty without deleting information. A larger fallback model still needs comparison against simpler alternatives on independently reviewed examples.
JevAgentHarness — some mistakes cannot be repaired later
Evidence: controlled agent-task evaluations with fitted delegation policies.
Attempt. Delegate tool and argument choices, then examine when calibrated fallback preserves outcomes.
Result. One failure selected a refund with 0.98 confidence before the customer supplied the refund method; the retained LLM then invented an argument. In one 200-task setting, all 34 Jev failures began with irreversible wrong tool choices. A separately evaluated fitted cascade delegated 27.2% of tool choices and 20.8% of argument choices, saving 14.2% cost with unchanged outcomes across 500 tasks. With a different, cheaper incumbent, the fitted policy delegated no tool choices. JevAgentHarness
Friction and lesson. Eligibility depends on the workflow and incumbent. “Ask the user first” must be an available action, and irreversible writes need their own acceptance criteria. A confidence score cannot supply a missing customer instruction.
JevBreak and Orchestkit — bounded answers still consume untrusted evidence
Evidence: adversarial research benchmark and a separate repository security fix.
Attempt. JevBreak appended adversarial text to an untrusted evidence field while preserving the original record and allowed answers. Its adaptive attacks used returned probabilities and up to 256 queries.
Result. On a benchmark of 240 records across 16 tasks and seven models, targeted attack success ranged from 31.6% to 99.4%, conditional on initially correct decisions. This is an adaptive attack result, not a deployment error rate. Confidence did not reliably catch the manipulated decisions. JevBreak
Separately, Orchestkit changed its browser-action policy so a confident Jev choice could not simply override the intended action. Executing the Jev-proposed action requires agreement or explicit confirmation, including agreement on action arguments such as a field's value; fallback to the agent's intended action remains available. Policy fix
Friction and lesson. Restricting output syntax does not protect the evidence channel. Test adversarial inputs at the actual decision boundary, and keep authorization and argument validation outside the model's confidence judgment.
Pattern: define uncertainty as behavior. Specify when to ask, abstain, preserve state, or escalate. Those are application transitions with consequences, not merely probability thresholds.
6. System economics: measure the trajectory and the serving path #
τ-bench — the median improved while success and the tail worsened
Evidence: controlled research comparison; each of 165 tasks ran once per system.
Attempt. Let Jev select tools and actions while DeepSeek-V4-Flash generates arguments and user responses.
Result. Overall success fell from 77.6% to 66.1%. Median episode time improved from 267.1 to 160.6 seconds, while P95 rose from 695.5 to 816.0 seconds. The paper traces long failures to both inappropriate action choices and incorrect arguments from the retained LLM, followed by extra lookups, retries, and recovery. General Decision Models, Table 3 and failure analysis
Friction and lesson. A local speedup can change which states an agent visits and how much repair it needs. These results describe this configuration; they do not assign every failure to Jev or establish a universal substitution penalty.
jev-routing — token savings depended on the coding model
Evidence: small paired coding-task experiment; repository subsequently archived.
Attempt. Insert Jev into coding-agent decisions, including forced tool selection.
Result. On six paired runs per model of a chess bug-fix task, median paired changes in upstream tokens for forced tool steering were −19.3% for Sol GPT-5.6, +37.1% for Terra GPT-5.6, and +50.8% for Luna GPT-6. These exclude Jev tokens and are not dollar savings. The workload is too narrow to represent general coding sessions. Experiment record
Friction and lesson. A steering policy can help one incumbent and burden another. Account for both systems' usage, and evaluate complete tasks before converting a component's token reduction into a cost claim.
Fastbrowse and Fast Jev Compaction — the glue can determine the outcome
Evidence: issue reports; distinct integration failures, not model benchmarks.
Attempt. Fastbrowse used support judgments to accept an answer or trigger repair. Its reported branch logic handled values above and below 0.5 differently, but exactly 0.5 could suppress appropriate repair. The issue was closed as completed without a post-fix benchmark. Fastbrowse report
Fast Jev Compaction users reported a host update silently bypassing their compaction hook. That is an integration-path report, not evidence that the replacement model became worse. Compaction issue
Friction and lesson. Specify equality, missing-result, timeout, and retry behavior. Instrument which implementation actually ran. Otherwise, a threshold edge case or upstream update can invalidate a careful model evaluation without changing the model at all.
Pattern: make the complete execution observable. Measure successful tasks, critical errors, recovery calls, P95 latency, and total cost. Verify that production uses the path whose behavior you measured.
What these cases imply for a refactor #
Across these reports, the recurring problem is a mismatch between the judgment being benchmarked and the behavior being replaced. Sometimes the model is inaccurate. Sometimes the candidates are incomplete, the reference labels are weak, an argument is missing, or an apparently harmless branch changes the entire subsequent task.
The cases suggest six concrete artifacts for a replacement review:
- A responsibility inventory: inputs, outputs, intermediate evidence, side effects, and downstream consumers of the original call.
- An explicit decision contract: candidate construction, allowed actions, argument requirements, uncertainty states, and authorization rules.
- An evaluation split: separate calibration and held-out examples, with real difficult cases and a strong rule-based or model-based incumbent.
- An error budget by consequence: missed work, unnecessary work, incorrect writes, and cases requiring human confirmation evaluated separately.
- A complete-task comparison: success, critical errors, recovery traffic, latency distribution, and cost including fallbacks and the decision model itself.
- An operating plan: shadowing where appropriate, serving-path telemetry, rollback, and revalidation when prompts, business rules, providers, or host software change.
This is why a typed endpoint is only the beginning of a successful replacement. The migration has to establish that the cheaper implementation preserves the behavior people depend on—and make departures visible when it does not.
The next adoption metric is the successful replacement #
The available evidence supports a meaningful shift: developers have more ways to implement semantic judgments, and some organizations report substantial gains. It does not yet establish a general success rate for production migrations or show that refactoring is the dominant barrier across the industry.
Our interpretation is that the growing supply of models makes the integration problem more valuable to solve. Teams need to identify what an operation is responsible for, test alternative implementations, understand the changed error distribution, and decide whether the savings repay the migration and upkeep.
Useful future reports would therefore measure engineering time, complete task success, critical errors, tail latency, and total cost per successful task. They would include fallback and maintenance costs, and record cases where keeping the incumbent was the right result.
This motivates our work on Seldon. Today's Functions workflow starts with an explicit request or short workflow, searches supported implementations, and presents measured alternatives for a developer to activate behind a stable endpoint. Our broader Semantic Compute thesis asks how much of the work of constructing and maintaining those alternatives can become automatic. That remains a research and engineering question.
In Part 2, Your LLM call is doing more than you think, we'll look inside the refactoring problem: recovering hidden responsibilities, constructing and evaluating replacements, and accounting for the cost of keeping them reliable. We'll also explain how this shapes Seldon's approach to Semantic Compute.
Route your traffic through Seldon and investigate recurring work that could benefit from a specialized program.