{"slug": "fast-cheap-agent-decisions", "title": "Fast, Cheap Agent Decisions", "summary": "TypeSafe launched Jev, a specialized \"System One\" model, on September 15, 2026 alongside disclosure of a $40 million seed round, claiming end-to-end response times of 70 ms to 500 ms versus 3 s to 329 s for frontier models. TypeSafe's own evaluations put Jev at 40x to 200x faster on System One queries, 0.114 s per workflow task versus 8.566 s for LLMs (193.6x slower), and $0.000081 per task versus $0.013880 (444.6x more expensive), with input tokens priced at $0.042 per million and free output tokens. TypeSafe reports 67.8% average accuracy across four workflows at $0.0004 per workflow, against GPT-5.6 Terra at 67.9% for $0.0304 and Claude Sonnet 5 at 67.8% for $0.1174, and says Jev returns a calibrated probability with every answer and zero hallucinations guaranteed by the output schema, a figure it did not measure.", "body_md": "# Fast, Cheap Agent Decisions\n\n*This post was originally published on the [Akka blog](https://akka.io/blog/fast-cheap-agent-decisions).*\n\nJev has only been available to the public for about a week (unveiled September 15th, 2026), but it’s already created a ton of excitement and discussion within the AI community. [TypeSafe](https://typesafe.ai) made Jev available through limited, early access at the same time they disclosed their $40 million seed round.\n\nLet’s strip away the hype and talk about what Jev is, and what it isn’t. When our agentic code submits a prompt to an LLM, it can include a description of available tools, context, and conversation history. The typical pattern is to stream text into a model and we get the answer streamed back out.\n\nIf we want more structured replies, we can give these LLMs a schema and tell them that their output must conform to that schema. This structured output can be anything from data queried from a customer’s account to a set of product recommendations or an itinerary for a vacation based on weather forecasts and traveler preferences.\n\nSometimes this output schema is extremely focused, like asking specific questions. Is there PII exposed anywhere in this text? Is the sentiment in the text negative or positive? Does the supplied context refer to a tech support issue or is it an account query? Narrowing down the potential output of an LLM to these extremely focused questions dramatically increases accuracy and reduces hallucinations.\n\nHowever, using a general-purpose LLM to answer these questions is extremely inefficient. While you could technically use a jumbo jet at runway taxi speed to commute from home to the office, it’s slow, inefficient, and costly. Overkill has a cost and many of us are using all-purpose LLMs when we could be using something more focused and efficient.\n\nThis is where Jev comes in. It’s a special kind of model with its own interface style. It doesn’t stream text bi-directionally like LLMs. Instead, you ask it questions based on some state and it gives you answers and their probability scores.\n\n## Performance and cost figures TypeSafe published for Jev\n\nTypeSafe published these figures with the Jev launch on September 15, 2026. All of them come from TypeSafe’s own evaluations and demos.\n\n| Claim | Jev | Comparison, as TypeSafe states it | \n|---|---|---|\n| End-to-end response time | 70 ms to 500 ms | 3 s to 329 s for frontier models | \n| Speed on System One queries | 40x to 200x faster | The same level of frontier intelligence | \n| Workflow evaluation, time per task | 0.114 s | 8.566 s for LLMs, 193.6x slower | \n| Workflow evaluation, cost per task | $0.000081 | $0.013880 for LLMs, 444.6x more expensive | \n| Accuracy, average of four workflows | 67.8% at $0.0004 per workflow | GPT-5.6 Terra 67.9% at $0.0304; Claude Sonnet 5 67.8% at $0.1174 | \n| Input token price | $0.042 per million ($42 per billion) | $0.20 to $10 per million | \n| Input price against Claude Fable 5.1 | 238x lower | $10 per million | \n| Output token price | Free | About 5x the input price | \n| Hallucinations and type errors | Zero, guaranteed by the output schema | LLMs hallucinate and make type errors | \n| Confidence | A calibrated probability with every answer | Overconfident and inconsistent, even when prompted for confidence | \n| Doom demo | 10 queries per second for about $7 per hour | None given | \n\nTypeSafe states that the 193.6x and 444.6x figures are on the higher end of real-world gains. The zero-hallucination figure follows from the output schema and was not measured. The LLM error rates TypeSafe compares against come from OpenRouter traffic. Sources: [TypeSafe launch post](https://typesafe.ai/blog/introducing-system-one-models-and-jev), [typesafe.ai](https://typesafe.ai), [evals.typesafe.ai](https://evals.typesafe.ai/).\n\nLet’s take a look at a classic use case for the kind of interrogations Jev makes fast and cheap. The user’s original prompt looks like this:\n\n“Hi, I ordered the blue jacket on the 9th and it still hasn’t shipped. I’ve emailed twice already. If it’s not out the door by Friday I want a refund. Order #48213.”\n\nUsing a traditional LLM, we might be able to get some actionable routing or planning decisions out of this. What we really want are some answers to discrete questions, and then our agent can take the right actions based on those answers with high confidence.\n\n| Question | Allowed Answers | Jev’s Answer | Probability | \n|---|---|---|---|\n| What is this about? | **Shipping** , returns, billing, product question, other | **Shipping** | 0.94 | \n| How is the customer feeling? | Calm, **frustrated** , angry | **Frustrated** | 0.81 | \n| Is this a repeat contact? | **Yes** , no | **Yes** | 0.97 | \n| Is the customer threatening to leave or dispute? | **Yes** , no | **Yes** | 0.88 | \n| Does this need a human, or can automation handle it? | **Human** , automation | **Human** | 0.76 | \n| Which queue? | Tier 1, tier 2, **retention** | **Retention** | 0.71 | \n| Is there an order number in the message? | **Yes** , no | **Yes** | 0.99 | \n| Priority | Low, medium, **high** | **High** | 0.83 | \n\nJev does one thing: answer questions. It does this quickly and cheaply. If we want to interrogate a state based on a user’s input, then this kind of Q&A is the right optimization. Take a look at the question, “Is there an order number in the message?”. Jev-style interactions don’t let us extract the order number from the message. Instead, we can only ask questions with a fixed number of potential responses. That limited inference target area is what makes this kind of fast and cheap interrogation possible.\n\nA pretty popular pattern is to interrogate the state and message and, based on the answers we get, we decide whether we need to make a full (relatively slow and costly) call to an LLM. We might use this to extract an order ID only if our interrogation is confident there is one in the message. If we are going somewhere on a nice day, we can choose to ride a bike, otherwise, we can use the massive transport truck.\n\n## Asking questions in Akka\n\nIn the [Akka SDK](https://akka.io/platform/sdk), interaction with models is managed via an effects API. Sending a system prompt, user prompt, and other context to a model and getting a stream of text back is I/O and not a “pure” function. The same applies to asking Jev (or technically anything that talks SystemOne, which isn’t yet a standardized protocol) questions, though talking to Jev is always synchronous.\n\nHere’s how you can easily use the effects API (available to the public soon in an upcoming SDK release) to have your agent ask Jev questions:\n\n```\npublic Effect<Judgment> triage(String ticket) {\n  return effects()\n    .judgment()\n    .state(ticket)\n    .question(\"route\", Question.choice(\"Which team should handle this?\")\n        .option(\"billing\", \"Payments, invoicing, refunds\")\n        .option(\"technical\", \"Bugs, outages, integrations\"))\n    .question(\"severity\", Question.score(\"How severe is this?\",\n        \"Low\", \"Medium\", \"High\", \"Critical\"))\n    .question(\"urgent\", Question.yesNo(\"Does this need a reply today?\",\n        \"Time-sensitive\", \"Can wait\"))\n    .thenReply();\n}\n```\n\nHere the `Judgment` class from the Akka SDK contains the answers and their probabilities.\n\nThis is incredibly powerful on its own, but it’s really a game-changer when combined with the rest of Akka’s agentic arsenal, including durable workflows, autonomous agents, and composed agentic hierarchies. Using fast and cheap triage on user input can save the LLM use only for when it’s really needed, increasing performance and decreasing cost.\n\nJust as with LLM interactions, the model provider settings are provided through regular configuration properties at runtime.\n\n## Using Jev to make routing decisions in the Akka AI Gateway\n\nIf you’re running your Akka projects on Akka’s automated operations infrastructure with [Akka Optimize](https://akka.io/platform/optimize), then you have access to Jev through our AI gateway. Akka’s AI gateway can route requests either to the API-based Jev hosted at [Typesafe.ai’s portal](https://typesafe.ai), or, you can self-host an OpenJev open-weight model within Akka’s inference engine for local inference.\n\nOne of the many things this gateway does is provide traffic routing for LLM interactions.\n\nWhat is routing if not asking the question, “Where should this request go?” It’s possible to use a full LLM for routing decisions, but that’s often overkill. A popular alternative is to use a locally hosted SLM to make those decisions. When you leverage an SLM to aide in routing decisions, you can start to make routing decisions based upon the substance and the content of the AI message itself, in other words routing based upon the semantic context of the prompt.\n\nAkka’s gateway can also use semantic routing to quickly route based on use case. This uses embeddings and doesn’t involve an LLM at all. If you want to route based on interrogation of the message and its content, then you can use Jev to triage your AI message traffic.\n\nSimply supply questions and answer possibilities to the router and choose the destination backend based on those answers and their probabilities. The answers, chosen backend, and probabilities will all be logged for compliance and governance just like everything else that goes through our AI gateway.\n\nAdding in new support for Jev, here are some of the different ways you can route AI traffic in an Akka environment (these are fictional examples):\n\n- **Static routing.** A prompt under 300 tokens and no tool definitions attached goes to Claude Haiku 4.5; anything with attached documents over 20k tokens goes to Claude Sonnet 5; everything else defaults to Sonnet 5 with Opus 5 only when the request carries a`reasoning: high` hint.\n- **Tool guardrail.** An agent loop asks Jev “which tool should run next?” as a choice over the registered tool names, and only invokes the full LLM to author the tool arguments once Jev has picked the tool with confidence above 0.85. Below that threshold, the LLM makes the full decision itself. This happens before traffic gets to the gateway, which can make further decisions based on cost, quota, etc.\n- **Fast Q&A routing.** Jev scores an incoming support ticket on urgency (1 to 5) and the router sends 4 and 5 to a Sonnet-backed agent with escalation tools, and 1 to 3 to a Haiku-backed responder with a canned-answer retrieval index.\n- **Sovereign routing.** A tenant with an EU data residency clause routes to Vertex AI in europe-west4 and is never allowed to touch the US-hosted direct API, enforced by policy rather than by the caller.\n\n## Summary\n\nJev is an incredibly new product, yet there are already a number of open source alternatives that all communicate using TypeSafe’s SystemOne concept. This specific interaction type is new, but wanting to interrogate AI input and context isn’t.\n\nIncorporating structured questions and answers into your agentic applications has the potential to make things more predictable, more reliable, faster, and even cheaper. We’d love to hear what you’re building and how this kind of optimized Q&A fits into your strategy.", "url": "https://wpnews.pro/news/fast-cheap-agent-decisions", "canonical_source": "https://kevinhoffman.blog/posts/fast-cheap-agent-decisions/", "published_at": "2026-09-23 00:00:00+00:00", "updated_at": "2026-09-23 16:33:14.277365+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-startups", "ai-products", "ai-agents"], "entities": ["TypeSafe", "Jev", "GPT-5.6 Terra", "Claude Sonnet 5", "Claude Fable 5.1", "OpenRouter", "Akka"], "alternates": {"html": "https://wpnews.pro/news/fast-cheap-agent-decisions", "markdown": "https://wpnews.pro/news/fast-cheap-agent-decisions.md", "text": "https://wpnews.pro/news/fast-cheap-agent-decisions.txt", "jsonld": "https://wpnews.pro/news/fast-cheap-agent-decisions.jsonld"}}