{"slug": "decision-ai-explained-in-5-minutes", "title": "Decision AI Explained in 5 Minutes", "summary": "TypeSafe AI released Jev, a proprietary decision model that returns probabilities over predefined answers instead of generating text, and the company has not published its full architecture, model weights, or training recipe. The model handles three question types — choice, score, and yes/no — by extracting logits for candidate answers and applying softmax, which skips token-by-token generation and makes repeated decisions faster. The article also walks through running open-source decision models locally with llama.cpp.", "body_md": "# Decision AI Explained in 5 Minutes\n\nLearn how decision models work, how they turn LLMs into faster decision-making systems, and how to run open-source alternatives locally.\n\nNot every AI task needs a large language model (LLM) that generates text. Sometimes, we just need a model to make a quick decision, such as choosing the right tool, classifying a customer complaint, or deciding whether a transaction looks suspicious.\n\nThis is where Decision AI comes in. With the release of **[Jev by TypeSafe AI](https://typesafe.ai/blog/introducing-system-one-models-and-jev)** and several open-source alternatives, we now have models specifically designed to make these decisions without generating unnecessary text.\n\nIn this article, we will learn how Decision AI works, explore open-source models, and run our first decision model locally using **[llama.cpp](https://github.com/ggml-org/llama.cpp)**.\n\n## What Is Decision AI?\n\nDecision AI is the use of AI to help make decisions. It is a broad field, but a newer generation of decision models focuses on something quite simple: **instead of generating text, the model selects from predefined answers and returns probabilities.**\n\nThese models generally support three types of questions:\n\n- **Choice:** Select an option, such as billing, sales, or technical support.\n- **Score:** Rate something on a scale, such as the urgency of a customer complaint.\n- **Yes or no:** Estimate whether something is true, such as whether an email is spam.\n\nFor example, imagine a customer sends an email saying, \"I was charged twice for my order.\"\n\nWe don't need an LLM to write a lengthy explanation. A decision model can simply identify billing as the most appropriate department and return a probability for that answer.\n\nThis makes decision models useful for applications that process thousands of similar requests.\n\n## How Does Decision AI Work?\n\nThe process is straightforward. We provide the model with some information, define a question, and specify the possible answers. The model evaluates the information and returns its decision with probabilities.\n\nHere is a simple workflow:\n\nGoing back to our customer support example, an illustrative result might look like this:\n\n| Question | Answer | Probability | \n|---|---|---|\n| Which department? | Billing | 92% | \n| Is it urgent? | Yes | 81% | \n| Does it need human review? | No | 76% | \n\nThese are hypothetical results. In practice, we would test the model's accuracy and probabilities on real customer messages before automating anything.\n\nThe main advantage is that the answers are structured, so our application can use them directly without extracting information from generated paragraphs.\n\n## Turning Large Language Models Into Decision Models\n\nWe can turn an existing LLM into a decision model by using its internal predictions instead of generating text.\n\nWhen an LLM processes an input, it produces **logits**, which are raw scores for possible next tokens. We can extract the scores for our predefined answers and apply a mathematical function called **softmax** to convert them into probabilities.\n\nFor example, when classifying a product review, the model might produce:\n\n- Positive: 2.0 (logit)\n- Negative: 1.0 (logit)\n\nAfter applying softmax, we get approximately 73% positive and 27% negative. These probabilities are relative to the options we selected.\n\nWe can take this further by replacing the usual text-generation layer with a small prediction head and fine-tuning the model on decision-making tasks. This lets it score multiple options, and potentially answer several questions, in a single forward pass.\n\n**The main advantage is that we skip token-by-token text generation**, making repeated decisions faster and more efficient.\n\n## Open-Source Decision AI Models and Frameworks\n\nJev is a proprietary model available through TypeSafe's API. Although the company has shared some details about its approach, it hasn't published the full architecture, model weights, or training recipe. So we still don't know exactly how everything works under the hood.\n\nMeanwhile, the open-source community has been moving incredibly fast, releasing alternatives that we can inspect, fine-tune, and run locally. Some are already reporting better results on specific benchmarks, faster inference, and potentially lower running costs.\n\nHere are five projects worth exploring:\n\n- **[Laya](https://github.com/NandhaKishorM/laya):** A lightweight decision model supporting over 100 languages. It offers fast inference, automatically selects the appropriate model, and runs on modest hardware.\n- **[Kev](https://github.com/jaredpalmer/kev):** A family of Qwen-based models ranging from 0.8B to 27B parameters. It supports long documents, custom fine-tuning, and deployment through a Jev-compatible API.\n- **[Simple Jev](https://github.com/featherless-ai/simple-jev):** Lets us use existing Hugging Face models for classification and scoring without additional training. It also supports vision models for making decisions about images.\n- **[Decider](https://github.com/Mapika/decider):** Designed to handle multiple decisions in a single forward pass. It supports up to 255 choices per question and includes experimental demonstrations involving games such as Tetris and Pong.\n- **[SemIf](https://github.com/TheoLeeCJ/SemIf):** Focuses on local decision-making, with support for NVIDIA GPUs, Apple Silicon, and CPUs. It includes an interactive browser demo and reproducible benchmarks for comparing accuracy and inference speed.\n\nThe interesting part is that we now have several ways to build decision-making applications without depending on a closed-source API. We can choose smaller models for speed, fine-tune them for our specific tasks, or run everything locally to reduce API costs.\n\n## Running Decision AI Locally With llama.cpp\n\nThings are moving quickly. Both **[Ollama](https://ollama.com/blog/ollama-now-supports-jev-style-decision-models)** and [llama.cpp](https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp) now support decision models through the TypeSafe-compatible `/v1/systemone` endpoint.\n\nOllama supports models such as Nimble and Tev1, while llama.cpp supports Laya, Kev, Julia-1, and OpenJev.\n\nLet's try Laya using the latest llama.cpp. Start the local server:\n\n```\nllama update\nllama serve -hf ggml-org/Laya-GGUF\n```\n\nOnce the model is running, open another terminal and send a customer support request:\n\n```\ncurl http://localhost:8080/v1/systemone \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"state\": \"I was charged twice for my order.\",\n    \"questions\": {\n      \"department\": {\n        \"type\": \"choice\",\n        \"instructions\": \"Which team should handle this?\",\n        \"criteria\": {\n          \"billing\": \"Payments and refunds\",\n          \"shipping\": \"Delivery problems\",\n          \"technical\": \"Bugs and account issues\"\n        }\n      }\n    }\n  }'\n```\n\nThe model will evaluate the message and return the selected department along with a probability for each option.\n\nThat's it. We can now experiment with a decision model locally without using a paid API.\n\n## The Future of Decision AI\n\nI think the most interesting part of Decision AI is how it can work alongside existing LLMs to make AI applications faster and cheaper.\n\nImagine an AI coding agent using a small decision model to select the right tool, decide whether to search the web, or determine if a task needs a more powerful LLM. Instead of using an expensive reasoning model for every little decision, we can reserve it for tasks that actually need it.\n\nAnother exciting area is real-time applications. Decision models are already being tested in games, where they can choose the next move in milliseconds. We could see similar applications in robotics, fraud detection, and autonomous AI agents.\n\nWith Ollama and llama.cpp now supporting these models, we can also run them locally, reducing API costs and keeping sensitive data on our own hardware.\n\nThe biggest challenge is reliability. If a model says it is 95% confident, we need to know whether we can trust that number. Better training, calibration, and human review will be important as these models move into real-world applications.\n\n \n\n \n\n[**\\[Abid Ali Awan\\](https://abid.work)**](https://abid.work) ([@1abidaliawan](https://www.linkedin.com/in/1abidaliawan)) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in technology management and a bachelor's degree in telecommunication engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.", "url": "https://wpnews.pro/news/decision-ai-explained-in-5-minutes", "canonical_source": "https://www.kdnuggets.com/decision-ai-explained-in-5-minutes", "published_at": "2026-11-20 13:00:00+00:00", "updated_at": "2026-11-20 13:16:26.556951+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-tools", "machine-learning"], "entities": ["TypeSafe AI", "Jev", "llama.cpp"], "also_reported_by": [], "takeaway": "Extract the logits for predefined answer tokens and apply softmax to get decision probabilities — the article's example turns logits of 2.0 (positive) and 1.0 (negative) into roughly 73% positive and 27% negative — skipping token-by-token generation.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/decision-ai-explained-in-5-minutes", "markdown": "https://wpnews.pro/news/decision-ai-explained-in-5-minutes.md", "text": "https://wpnews.pro/news/decision-ai-explained-in-5-minutes.txt", "jsonld": "https://wpnews.pro/news/decision-ai-explained-in-5-minutes.jsonld"}}