{"slug": "decider-one-forward-pass-typed-decisions-calibrated-probabilities", "title": "decider: one forward pass, typed decisions, calibrated probabilities", "summary": "A developer released decider, an open-source language model family that returns calibrated probability distributions over typed, fixed-option questions from a single forward pass instead of generating text, with no decoding or parsing. The models, ranging from 0.8B to 35B parameters and released under Apache 2.0, run a decision in as little as 18 ms on a B300 GPU, and the 35B's text weights answered held-out image questions zero-shot at 91.5% accuracy, above the purpose-trained vision model's 87.3%.", "body_md": "decider is a language model that does not generate text. It reads a state and a set of typed questions and returns, from one forward pass, a probability distribution for every question. There is no decoding, no parsing, and no output outside the options you defined.\n\nThis post is what it does, how it is built, what the two public leaderboards say about it, and where it fails.\n\nA typed decision is a question with a fixed answer set. Three kinds:\n\nYou pass a state (any JSON) and a dictionary of questions. You get back, per question, the chosen option, its confidence, and the full distribution.\n\n```\npip install decider-ai\npython\nfrom decider.infer import Decider\nd = Decider(\"Mapika/decider-2b\")     # one CUDA GPU, bf16, about 4 GB\n\nd.system_one(\n    {\"ticket\": {\"messages\": [{\"from\": \"customer\", \"text\": \"I was charged twice for order A-104. Please refund the duplicate.\"}]},\n     \"refund_policy\": \"Duplicate charges are eligible for a refund.\"},\n    {\"department\": {\"type\": \"choice\", \"instructions\": \"Which team should handle this?\",\n                    \"criteria\": {\"returns\": \"Exchanges, refunds, wrong or damaged items\",\n                                 \"billing\": {\"what\": \"Charges, invoices\", \"not_for\": \"delivery\"},\n                                 \"other\": None}},\n     \"refund_requested\": {\"type\": \"noul\", \"instructions\": \"Does `ticket.messages[0].text` request a refund?\"},\n     \"frustration\": {\"type\": \"score\", \"instructions\": \"How frustrated is the customer?\",\n                     \"criteria\": [\"calm\", \"frustrated\", \"very frustrated\"]}})\n# department: billing 0.56 (returns 0.44, other 0.00)\n# refund_requested: 0.99\n# frustration: level 1 of 3, 0.55\n```\n\nThe plain form is `d.decide(state, [{\"question\": ..., \"options\": [...]}])`. There is an HTTP server with the same wire format as TypeSafe's Jev API, so their SDKs work against a local `decider` unchanged.\n\nEvery question is rendered into the prompt with lettered options and an answer slot. The model is run once. At each answer slot the logits of the option letters are read, divided by a fitted temperature, and passed through a softmax. That is the whole readout: the distribution is the model's own next-token belief over the letters, nothing is sampled and nothing is parsed.\n\nBecause the readout is a single position in one forward pass, the cost of a decision is the cost of a prefill. On one B300 in bf16, batch of one, no CUDA graphs and no compilation:\n\n| model | median per decision | \n|---|---|\n| decider-2b | 18 ms | \n| decider-35b-a3b (3B active) | 41 ms | \n\nThe CUDA-graph engine the package uses by default is faster than these plain numbers.\n\nAll weights are on the Hub under Apache 2.0.\n\n| model | base | trained how | \n|---|---|---|\n| decider-0.8b | Qwen3.5-0.8B-Base | supervised | \n| decider-2b (v10) | Qwen3.5-2B-Base | supervised, then calibration-aware RL on live browser tasks and exact games | \n| decider-2b-vision | Qwen3.5-2B vision-language | supervised on image questions | \n| decider-35b-a3b | Qwen3.5-35B-A3B-Base | supervised, routed experts frozen, Muon on the block matrices | \n\nThe training mixture is public datasets plus questions labelled by a local 27B teacher. Nothing was distilled from Jev. The mixture, the trainer and the evaluation code are in the repository; the recipe runs on one GPU for the small models.\n\nA result we did not plan for: the 35B's text weights loaded onto the vision-language version of its base answer image questions zero-shot. On 71 held-out image questions from The Cauldron it picks the gold answer 91.5% of the time, against 87.3% for the purpose-trained decider-2b-vision on the same rows.\n\nTwo third-party leaderboards rank this model class. We did not run either; the numbers are theirs.\n\n**Decision Index** (edition 0.1, 22 September 2026, by multimodalart) runs every open reproduction of Jev over the same 132,422 requests on one RTX PRO 6000, with unanswered requests counted as wrong. Its score is the mean of five capability areas. decider-35b-a3b is fourth at 54.3, behind Jev at 59.5 and two zero-training wrappers on stock models at 55.7 and 55.6. The gap is knowledge and reasoning: GPQA, GSM8K, CRUXEval and MMLU.\n\nThe index also reports calibration: expected calibration error over ten confidence bins on 33 benchmarks, where confidence is the probability placed on the chosen option.\n\n| entry | calibration error (points) | wrong at 95%+ confidence | \n|---|---|---|\n| decider-35b-a3b | 3.1 | 0.4% | \n| jevfire (stock Qwen3.8-27B) | 6.3 | 1.5% | \n| Jev 1.13.0 | 6.5 | 2.0% | \n| decider-2b | 8.8 | 0.9% | \n\ndecider-35b-a3b is the best-calibrated entry of the 32, Jev included. Its mean confidence is 0.673 and its accuracy on the same answers 0.675. Calibration is a readout on the site, not part of the score; a model can be well calibrated and wrong.\n\n**JevBench** (read 21 September 2026) scores four axes. decider-35b-a3b is tenth of 36 at 68.9, pulled down by the cost axis; decider-2b is twenty-first at 64.6, pulled down by calibration on the hard tier.\n\nThe full measurement tables, including the regressions between versions, are in `docs/RESULTS.md` and `docs/CHANGELOG.md` in the repository.\n\nThe base model sets the knowledge score. Under our readout, stock Qwen3.6-27B and Gemma-4-26B-A4B-it both read above our trained 35B on a sample of the index, so the next series will start from post-trained bases, keep the readout and the calibration, and train only where training beats the stock model, which on our measurements is the 2B and 4B sizes.\n\n`pip install decider-ai`\nThis is an independent project, not affiliated with or endorsed by TypeSafe AI.", "url": "https://wpnews.pro/news/decider-one-forward-pass-typed-decisions-calibrated-probabilities", "canonical_source": "https://dev.to/mapika/decider-one-forward-pass-typed-decisions-calibrated-probabilities-32c9", "published_at": "2026-09-22 12:11:22+00:00", "updated_at": "2026-09-22 12:23:15.898500+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools", "machine-learning", "ai-products"], "entities": ["decider", "Qwen3.5", "Mapika", "TypeSafe", "Jev", "The Cauldron", "Decision Index", "multimodalart"], "alternates": {"html": "https://wpnews.pro/news/decider-one-forward-pass-typed-decisions-calibrated-probabilities", "markdown": "https://wpnews.pro/news/decider-one-forward-pass-typed-decisions-calibrated-probabilities.md", "text": "https://wpnews.pro/news/decider-one-forward-pass-typed-decisions-calibrated-probabilities.txt", "jsonld": "https://wpnews.pro/news/decider-one-forward-pass-typed-decisions-calibrated-probabilities.jsonld"}}