A Noul comes back with a confidence of 0.12. That’s not noise. I read it as the whole point of the model.
Typesafe.ai shipped something called Jev recently, and most of the discussion around it got stuck on whether it counts as a “frontier model.” I want to set that argument aside. Jev is a narrow, specific thing: a zero-shot, structured-output classifier with no text generation in it at all.
The part that’s missing# #
Jev doesn’t have an autoregressive decoder. It can’t write a sentence, can’t generate code, can’t hold a conversation. That’s the whole design.
I keep seeing the standard workaround for this kind of decision: ask a big language model to generate a string, then write code to parse that string for a keyword. Standard LLMs predict one token, feed it back in, predict the next one, and repeat until they hit a stop condition. That loop costs time.
Jev skips the loop entirely. It runs a single parallel forward pass over the input, plain text or structured JSON, and computes a decision boundary directly. I don’t have to wait for tokens I’m going to throw away anyway.
No loop also means no per-token cost on the output side. Typesafe.ai charges $0.042 per million input tokens — I checked twice, because that’s cheap enough to stop tracking per-call. There’s nothing left to decode one piece at a time, so the response just lands. I’ve stopped budgeting for it.
Three shapes, not one string# #
When I ask a normal LLM to classify something, I get a string back, then I write code to match that string against expected values and handle whatever shows up that I didn’t expect. Production bugs live in that parsing step.
Jev skips the string. It returns one of three typed primitives:
- Noul, a Bernoulli yes/no
- Choice, a pick from a fixed category list
- Score, a float on a sliding scale
Each one maps directly onto a decision a caller already has to make. I don’t want a router that gets back a string that might say “yes” or “Yes.” or “yes, definitely.” I want a boolean.
Forced answers versus calibrated ones# #
Here’s the distinction. OpenAI’s structured outputs, and most constrained-decoding setups like it, work by masking invalid tokens. If the model wants to output something outside the schema, the system zeroes out that probability and forces the next best valid token instead.
The model is still confused underneath.
It just isn’t allowed to show it.
Jev is trained differently. The training objective targets accurate probability directly, not just schema validity, the way constrained decoding does. So when Jev returns that Noul at 0.12, I take the number at face value.
A confidence that low isn’t noise to filter out. A caller can act on it directly: fail over to a human, fall back to a default route, whatever the pipeline needs. That’s actionable.
A forced answer and a calibrated admission of uncertainty can look identical on the wire. Same JSON blob, same boolean field. I only notice the difference when I’m deciding whether to trust the number.
What I built around it# #
That was the observation. What follows is what happened when I stopped treating Jev as a single-call classifier and started treating it as a judgment layer that could run at every decision point in an agent toolkit.
The toolkit I run, documented across the vexjoy-agent repository, now has 34 Jev-powered Python scripts totaling 21,606 lines, backed by 1,609 passing tests. Eight hooks fire Jev at five different lifecycle events. A meta-compositor decides which of 30 registered tools to run for each request. The whole thing dogfoods itself — Jev reviews the code that Jev tools produce.
Source: `scripts/jev-*.py` (34 files), `hooks/jev-*.py` (8 files), `scripts/tests/test_jev_*.py` (test suite run: 1,609 passed).
### The /d router: four phases, one Jev call each[#](#the-d-router-four-phases-one-jev-call-each)
The /d command is the entry point. A user types /d review this PR for security issues and the system has to decide which agent, which skill, and which pipeline to invoke. Before Jev, this required a full routing manifest into the LLM’s context and asking it to pick.
The /d router replaces that with a four-phase pipeline defined in skills/meta/d/SKILL.md. Phase 1 (CLASSIFY) runs jev-route.py as a single subprocess call. Jev gets the request text, a list of all available agents and skills from the manifest, and returns a structured result: agent, skill, pipeline, complexity, confidence, and reasoning. One HTTP round trip to Typesafe, no autoregressive generation.
Phase 2 (DECIDE) reads that result and applies it. The agent, skill, and pipeline names are already validated against the live manifest inside the script — Jev picked from a constrained set, and the script verified the pick is real.
Phase 3 (ENHANCE) maps boolean signals from the classification into stack entries. If tests_requested came back true (a Noul above 0.6), the router adds test-driven-development and verification-before-completion to the skill stack. If research_needed fired, it fans out a research coordinator agent in parallel.
Phase 3.5 (COMPOSE) is where it gets dense. This phase calls jev-compose.py, a meta-compositor that uses one Jev call to decide which of 27 registered tools should run for this specific request.
Source: skills/meta/d/SKILL.md, scripts/jev-route.py, scripts/jev-compose.py.
The compose layer: 27 tools, one Jev call#
jev-compose.py holds an AVAILABLE_TOOLS registry of 27 Jev tools, organized into six categories: review (6 tools), prose (2), classification (4), security and risk (4), request analysis (4), execution planning (3), and post-agent validation (4). Three more scripts — jev-compose.py itself, jev-route.py, and jev-eval.py — sit outside the compose registry, bringing the total to 30.
The compositor’s architecture is one Jev call with parallel questions: one Noul per candidate tool (“should this tool run for this request?”) plus one Choice for pipeline_shape (pre-filter, post-check, continuous, standalone, or none). That’s 28 parallel questions in a single HTTP request. Jev evaluates all of them in one forward pass and returns scores for each.
Tools above the 0.5 threshold get selected. The compositor sorts them into deterministic phase ordering — pre-filter tools first, then standalone, then post-check — and returns an ordered pipeline spec. The whole decision costs one Jev call at approximately the same latency as a single Noul.
When a user types /d review this PR, the compositor might select jev-diff-triage (order 1), jev-cascade-review (order 4), jev-pr-slop-scan (order 5), jev-comment-quality (order 6), and jev-review-pipeline (order 8) — the full review chain. For /d write a blog post, it might select only jev-anti-slop and jev-voice-check as post-check tools. For /d TypeError: unhashable type list, it selects jev-error-classify as a standalone tool that handles the request without dispatching an LLM agent at all.
Source: `scripts/jev-compose.py` (AVAILABLE_TOOLS list, lines 51-601; `_build_payload`, lines 686-737).
### Four hooks across the agent lifecycle[#](#four-hooks-across-the-agent-lifecycle)
The four hooks fire Jev at every stage of an agent turn. Each hook is a Python script registered to a Claude Code lifecycle event.
UserPromptSubmit (jev-route-injector-userprompt.py). Detects a /d invocation via regex before the model generates its first token. Runs jev-route.py, then chains to jev-compose.py, then executes the composed tools — all three steps complete before the LLM sees the prompt. The results are injected as additionalContext so the model reads them instead of re-running the scripts. This exists because the model previously skipped the route script on a meta-question and reasoned about routing itself; a hook that runs before generation starts removes that failure mode.
PreToolUse (jev-pre-edit-guard-pretooluse.py). Fires before every Write, Edit, or Bash call. For Write and Edit, it runs jev-secret-scan.py on the content being written. A high-severity real secret (severity 4 or above) blocks the tool call entirely via permissionDecision:deny. Lower-severity findings inject a warning. For Bash, it runs jev-rollback-risk.py on the command and injects a warning for irreversible operations but never blocks.
SubagentStop (jev-post-agent-validator-subagentstop.py). Fires after every subagent completes. Runs three Jev tools in parallel via ThreadPoolExecutor: jev-completion-validator (did the output address the request?), jev-agent-confidence (does output quality match routing confidence?), and jev-scope-creep (did the agent touch files outside the requested scope?). Results are written to a per-session state file for downstream hooks.
Stop (jev-turn-quality-gate-stop.py). Fires after the model finishes responding. Computes a working-tree diff, and if the turn produced code changes, runs jev-commit-readiness.py on the diff. When the verdict is not “ready,” it surfaces an advisory rewake. This hook is advisory only — it never blocks.
All four hooks fail open. Network timeout, malformed response, missing script — every error path exits 0 and lets the turn proceed. The one exception is the PreToolUse secret scan at severity 4+, which blocks because a leaked secret costs more than a false block.
Source: `hooks/jev-route-injector-userprompt.py`, `hooks/jev-pre-edit-guard-pretooluse.py`, `hooks/jev-post-agent-validator-subagentstop.py`, `hooks/jev-turn-quality-gate-stop.py`.
### The structured interview pattern[#](#the-structured-interview-pattern)
Jev’s accuracy depends on question quality. The project’s design philosophy (docs/PHILOSOPHY.md, “Write Jev questions that cut”) documents a pattern shift from v1 to v2 questions.
v1 questions were broad. A single Noul asking “did the agent expand scope?” A vague question returns a vague probability. v2 questions are structured interviews: 10-15 Nouls per tool, each isolating one dimension. For scope creep, that means separate Nouls for “touched files outside the import chain,” “reformatted code it only read,” “added features not requested,” “changed error handling in unrelated paths,” and so on. Each question has a clear true/false boundary. A single Jev call carries up to 30 parallel Nouls at negligible marginal cost.
The criteria blocks — what, examples, not_for per option — encode the hard distinctions. The jev-compose.py source shows this: each tool in AVAILABLE_TOOLS carries when_to_run, examples, when_not_to_run, and negative_examples. These criteria exist to resolve the ambiguous cases. “Agent added a test file for code it edited” is expected practice, not scope expansion. “Agent rewrote an unrelated README” is obvious scope creep and doesn’t need criteria. The criteria matter for the gray zone between those two.
Source: docs/PHILOSOPHY.md (“Ask many specific questions, not few broad ones”; “Criteria encode the hard distinctions”), scripts/jev-compose.py (AVAILABLE_TOOLS per-tool criteria).
Dogfooding: Jev reviews Jev#
The toolkit uses Jev to review its own output. When I ask the /d router to review a PR that includes changes to Jev scripts, the compose layer selects the review chain (jev-diff-triage, jev-cascade-review, jev-pr-slop-scan, jev-comment-quality), and those tools evaluate the diff — including diffs to themselves.
The jev-route-injector hook runs the full route-compose-execute pipeline before the model generates a single token. The model receives Jev’s findings as prior_results. The SKILL.md makes the separation explicit: “Jev handles every judgment call, the LLM agent handles only generative/creative work.” On a review request, the agent explains, prioritizes, and suggests fixes. It does not re-judge what Jev already decided.
This creates a concrete feedback loop. When a Jev tool produces a false positive, the fix goes into the tool’s criteria block, and the next invocation evaluates whether the fix worked — using Jev.
Source: skills/meta/d/SKILL.md (Phase 4, “When JEV_TOOL_RESULTS is non-empty” section).
What’s known and what isn’t# #
Three limitations are concrete enough to name.
Question quality is still the bottleneck. The cost per Jev call is near-zero. The constraint is thinking clearly about what to ask. The v2 structured-interview pattern produces sharper results than v1’s broad questions, but writing good criteria for each of 27 tools is slow, manual work. jev-test-strategy.py was recently rewritten from v1 to v2 question format; the remaining scripts need the same treatment.
Calibration outside known domains is untested. I haven’t run a calibration stress test on adversarial or out-of-distribution input. Jev’s RLCD training targets accurate probabilities, and the scores I see on code review, secret scanning, and scope checking are consistent with what I’d expect. But I’m operating inside the domains I built questions for. I don’t know how calibration holds on content types or code patterns my criteria don’t cover.
Context window costs remain. Each Jev call is cheap, but the state field — the evidence Jev evaluates — comes from somewhere. On a large diff, the pre-edit guard truncates to 4,000 characters before passing to Jev. The compose layer truncates similarly. That truncation loses information. KV cache optimization for Jev’s input side, and smarter truncation strategies, are on the list but not built.
The question I started with was whether a zero-shot classifier with calibrated probabilities could serve as the judgment layer for an entire agent toolkit. Thirty scripts, four lifecycle hooks, and 17,183 lines of Python later, the answer is qualified: it works, when the questions are sharp and the criteria encode the actual hard cases. The model that can’t write a sentence turns out to be good at deciding things — as long as I’m specific about what I’m asking it to decide.