# I Put a Classifier in Front of My Agent's Judgment Calls > Source: > Published: 2026-09-17 00:00:00+00:00 I typed `/d read the article and give me your perspective on it as an article`. About a second later, Jev had given the technical-journalist agent a 0.67 fit score and the assessment skill a 0.53 fit score. The router also decided that the request did not need a larger workflow. There was no paragraph explaining its reasoning for me to parse. The program had numbers and a small set of typed decisions, so it could move. That is what interested me about Jev. Jev is Typesafe.ai’s decision model. It gives up string generation and answers typed questions instead. A Noul returns the probability of yes or no. A Choice distributes probability across a fixed set of options. A Score rates an input against ordered, described levels. I had spent months asking generative models to make decisions in prose. The model would return `yes`, `Yes.`, or `yes, I think so`, and I would write another layer of code to turn the sentence back into the boolean I wanted in the first place. I started replacing those calls. ## The Probability Is Available[#](#the-probability-is-available) Jev returns a distribution over outcomes I define before the call. That interface is useful to my program, but the interface alone says nothing about whether the probabilities are calibrated. If a Noul returns 0.12, Jev has assigned a 12 percent probability to `yes`. That is not a separate confidence score. The program can accept it, reject it, send the case to a person, or choose another path. It does not have to infer uncertainty from a sentence. Typesafe trains Jev for calibration, but I have not established that calibration on my own workloads. Calibration is a property of predictions across many labeled cases. One result at 0.12 proves nothing by itself. What I can establish is smaller and still useful: Jev exposes the probability, I can store it, and I can test later whether decisions made at that probability deserve my trust. As of September 2026, Typesafe lists Jev input at $0.042 per million tokens and output tokens as free. At that price, I stopped treating each judgment as a scarce call. The scarce thing became a good question. ## What the Toolkit Does[#](#what-the-toolkit-does) The plugin snapshot I checked while revising this article contains 38 `jev-*.py` scripts and 24,805 lines of Python. Twenty-seven of those scripts are candidates the compositor can select. It asks one Noul about each candidate and one Choice about pipeline shape, which makes 28 questions in the request. The remaining scripts support transport, evaluation, reporting, and other parts of the system. Those counts describe that snapshot, not an architectural constant. The toolkit has kept changing while I write about it. The title used to claim a different number and was stale within days. The tools sit at points where an agent system needs a bounded judgment: - Does this request need a security review? - Did an agent touch files outside the request? - Does this output contain a secret? - Is this diff ready to commit? - Which known recovery category fits this error? Some of those questions belong in ordinary code. A regex can find a literal token, a parser can validate JSON, and `git diff` can say which files changed. I use Jev when the evidence is available but the boundary still requires a reading. The `/d` router is one example. Its first Jev request ranks a wide manifest and decides whether the request is trivial. If more routing is needed, code builds a shortlist. A second request reranks that smaller set with fuller criteria and checks whether each selected agent, skill, or pipeline actually fits. After the route is chosen, another bounded check compares the proposed restatement with the user’s request before any work begins. The request at the start of this article took 1,004 milliseconds for the two routing stages. Stage one consumed 4,631 input tokens. Stage two consumed 3,389. The result selected the technical-journalist agent and assessment skill that produced the first editorial reading of this draft. Jev did not write the review. It made the bounded decisions that determined who should write it and what method that writer should use. ## The Questions Got Smaller[#](#the-questions-got-smaller) My first questions were broad. I asked things like, “Did the agent expand scope?” and received probabilities that I could not use with much confidence. The question hid several different behaviors. An agent might reformat a file it only read, add an unrequested feature, change error handling in an unrelated package, or create a test next to code it legitimately edited. Calling all of that “scope” made the boundary fuzzy before Jev saw it. I split the judgment into separate questions: - Did the agent modify a file outside the relevant import chain? - Did it reformat code it only needed to read? - Did it add behavior the request did not require? - Did it change an unrelated error path? Each question gets its own true and false criteria, including the cases that should not trigger it. Adding a test for edited code is expected practice. Rewriting an unrelated README is not. The useful work is defining everything between those examples. Jev evaluates independent questions over shared state in parallel. That makes decomposition cheap in wall time, but not free in thought or input tokens. Fifty vague questions are still fifty vague questions. ## Programs First, Jev Second, Language Models Last[#](#programs-first-jev-second-language-models-last) The toolkit settled into three layers. Programs handle anything computable. They search, parse, count, diff, validate, and enforce fixed policy. They are fast, inspectable, and do not become more correct when phrased politely. Jev handles bounded judgments over evidence already in hand. It can decide whether a passive construction hides a material actor, whether a proposed route fits a request, or whether a revision preserves a stated point. It cannot supply evidence I forgot to send. A generative model runs when the job requires new language: explaining a finding, proposing a repair, or rewriting a paragraph. It receives the program’s observations and Jev’s judgments instead of repeating those jobs from scratch. The secret guard shows how the boundaries matter. Infrastructure failure fails open: if the service is unavailable or a response is malformed, the agent turn continues. A completed scan that identifies a high-severity secret is a different event and can block the write. “The scanner could not run” and “the scanner ran and found a credential” are not the same answer. Error classification has a similar limit. Jev can map an exception to a known recovery category without asking a language model to diagnose it in prose. The classifier recommends the path; deterministic code or a later agent still has to perform the repair. The judgment layer does not magically become the action layer. ## The Article Became a Test Case[#](#the-article-became-a-test-case) The first revision of this article made claims I had not earned. It mislabeled the Noul output, treated one probability as proof without presenting calibration data, described internal model architecture more specifically than Typesafe’s public documentation supports, and mixed stale counts from different versions of the toolkit. An editor caught those problems. We turned that critique into 50 explicit propositions and asked Jev whether the draft supported each one. The request took 883 milliseconds and 9,330 input tokens. It was a consistency check against a human reading, not an independent review: the criticism existed before the questions did. The useful result was the pattern. Jev strongly supported the complaints about missing calibration evidence, the absent request-to-outcome trace, and the abrupt ending. It also supported the parts the editor wanted to preserve, especially the three-layer architecture and the lesson about smaller questions. That was enough to tell me where another revision was worth my time. The exercise also exposed a trap. A detector can reward prose for avoiding known patterns while the article itself gets worse. Earlier edits removed an overly tidy ending and replaced several flagged sentences with terse declarations. The detector became happier. The article became flatter and stopped without landing anywhere. Passing a detector is not the same as being worth reading. ## What Is Still Unsettled[#](#what-is-still-unsettled) Question quality remains the bottleneck. Writing criteria for a judgment is slow because the criteria are the judgment, made explicit enough that a program can use the answer. State is another limit. Several diff-oriented checks cap their input at 4,000 characters. That keeps requests bounded, but it can remove the exact evidence a question needs. A precise question over incomplete state produces a precise answer to the wrong record. I also need labeled outcomes. The probabilities look consistent on the cases I know well, but consistency is not calibration. I need human-reviewed examples, a held-out set, Brier scores, and reliability curves before I can say how much trust a threshold deserves. Jev made judgment calls cheap enough to put almost anywhere. The difficult work moved into deciding what evidence the program has, where the boundary belongs, and what should happen when the probability falls near it. I can inspect those decisions, label them, and change them one at a time. The next result at 0.12 will matter only if I can show what question produced it, what happened to comparable cases, and why the program should act differently because of it. That is the standard I want for the judgment layer: uncertainty exposed, decisions recorded, and trust earned from what happens next.