{"slug": "you-could-have-built-jev", "title": "You could have built Jev", "summary": "A technical explainer published on typesafe.ai argues that Jev, a system for answering structured questions with LLMs, can be reconstructed from first principles by combining a prompt-writer, a single forward pass through a model such as gpt-oss-120b, and a harness that samples from the model's output probabilities. The article notes gpt-oss-120b uses a token dictionary of 201,088 tokens while Gemma 3 uses 262,208, and that at least four projects replicating Jev had already been released before the piece was written.", "body_md": "# You could have built Jev\n\n*(a human wrote every word of this article, but an LLM did a picture and the pseudocode. This article is probably massively out of date if you’re reading it any time after Sep 2026)*\n\nThere’s a lot of excitement about [Jev](https://typesafe.ai/), but also a great deal of questions being asked that suggest that many people don’t really understand what Jev is and is not.\n\nAt the end of this (short, simple, illustrated) article, you should be able to reason about Jev from first principles.\n\n## Some bullshit backstory\n\nThis article was going to be “build your own Jev with gpt-oss-120b”, but this being The Age of LLMs, in the time between thinking about that and putting some time aside to actually do it, at least four projects doing that have been released. In my defense, I have a real job, enjoy sleeping, and I’m old.\n\nAnd that’s fine, it would have been a hassle to do the benchmarks myself when I can just steal them from other people.\n\n## What is actually input and output\n\nYou already know that an LLM accepts tokens in, and produces tokens out, and that a token is usually a word fragment.\n\nYou may or may not know that LLMs use a fixed, ordered token dictionary. Some LLMs (especially in the same family) share these dictionaries. They’re created before training by a statistical analysis of the corpus, rather than being hand-built. They differ in size too, for example, gpt-oss-120b has a dictionary of 201,088 tokens, Gemma 3 has 262,208.\n\nLLMs read and write arrays of numbers between 0 and whatever the token dictionary size (-1) is, each number representing an item in that token dictionary. Something is converting those to and from readable characters for you:\n\nYou may or may not also know that the majority of LLMs you use are *auto-regressive*, which means they produce a single token at a time. Once a token is generated, that token is then added to the end of the input, and the next token is generated using the same process.\n\nBut you don’t need to keep generating tokens. You could just generate one, then stop. A single run through the network.\n\nFinally, you may already know that the above was a white lie. The LLM doesn’t generate a single token, it gives you back an array that gives you the probability — for every token in the dictionary — that that token is the next one in the sequence. (technically it gives you *logits*, but you can turn those into probabilities easily)\n\nThe harness the LLM is running in picks a token for you from those probabilities (how randomly is what *temperature* controls), and then converts from that token ID into characters (before adding that token to the input, and doing the next one).\n\n## How to make a Jev-like\n\nSo you’ve got some time and tokens to burn, and you want to make your own Jev. Here’s one way to do it. You will need:\n\n### A prompt-writer\n\nYou need to take the user’s specially crafted prompt (this taken from the Jev site):\n\n```\nQuestions:\n{\n  \"is_sandwich\": {\n    \"type\": \"noul\",\n    \"instructions\": \"Is `food` a sandwich?\",\n    \"criteria\": {\n      \"true\": \"A sandwich is a food dish where a filling, such as meat, cheese, vegetables, or spread, is placed between structural starch\",\n      \"false\": \"The food has no bread enclosing a filling or uses only a single slice of bread, or uses a non-bread wrapper such as a tortilla, wafer, or cookie.\"\n    }\n  }\n}\n\nState:\n{\n  \"food\": \"Ice cream sandwich\",\n  \"definition\": \"An ice cream sandwich is a frozen dessert with a layer of ice cream between two cookies, wafers, or thin pieces of cake.\"\n}\n```\n\nand turn it into a prompt that an LLM will understand. We use prompt-caching in this household, so we’ll write a constant prompt-start based on *Questions*:\n\n```\nPlease answer the question [\"Is `food` a sandwich?\"] from the options below, where the criteria is:\n\n\"0\": A sandwich is a food dish where a filling, such as meat, cheese, vegetables, or spread, is placed between structural starch\n\n\"1\": \"The food has no bread enclosing a filling or uses only a single slice of bread, or uses a non-bread wrapper such as a tortilla, wafer, or cookie.\"\n\nWE WILL ONLY LOOK AT THE FIRST CHARACTER OUTPUT. YOU MUST ONLY OUTPUT A SINGLE CHARACTER CORRESPONDING TO YOUR CHOICE.\n\nHere is the data:\n```\n\nand then we can push the state in as a dynamic part:\n\n```\nFood: \"Ice cream sandwich\"\nDefinition: \"An ice cream sandwich is a frozen dessert with a layer of ice cream between two cookies, wafers, or thin pieces of cake.\"\n```\n\nYou can *fairly* reliably push this into your favourite model, and get back just a zero or a one. Not always! Models like to chat and they like to tell you how smart they are while not following instructions.\n\nHere’s OpenAI’s Astra being a good boi and getting it right first time:\n\n### A parser\n\nWe *could* now get the string response back from LLM, take the first character, and complain loudly if it decides to give us some exposition first. But Jev returns probabilities, and also promises “NO HALLUCINATIONS”, so that approach won’t quite cut it.\n\nInstead, we’ll take the returned *logit array* (kinda “token probabilities”) we described above, and pull out just the probabilities for the tokens we care about, in this case “0” and “1”:\n\n```\n# Use the model's own tokenizer, without adding special tokens.\nzero_tokens = tokenizer.encode(\"0\", add_special_tokens=false)\none_tokens  = tokenizer.encode(\"1\", add_special_tokens=false)\nassert length(zero_tokens) == 1\nassert length(one_tokens) == 1\n\nid_0 = zero_tokens[0]  # The token ID for the text \"0\", not ID 0!\nid_1 = one_tokens[0]  # Likewise, this isn't necessarily ID 1.\n\nlogits = model.next_token_logits(prompt)  # One score per dictionary entry.\nanswer_logits = [logits[id_0], logits[id_1]]\n```\n\nNow these aren’t probabilities directly, but we can blindly do what everyone else does and use the *softmax function* to turn them into a probability distribution over the answers we do care about:\n\n```\n# Assume we got a _logit_ of 1.2 for the character \"0\", and 3.2 for \"1\".\n                         \"0\"          \"1\"\nLogits                   1.2          3.2\nSubtract max (3.2)      −2.0          0.0\nExponentiate             0.135335     1.000000\n\nTotal = 0.135335 + 1.000000 = 1.135335\n\nP(\"0\") = 0.135335 / 1.135335 ≈ 0.1192 ≈ 11.92%\nP(\"1\") = 1.000000 / 1.135335 ≈ 0.8808 ≈ 88.08%\n\n# Model says 88% probability that it is _NOT_ a sandwich (option \"1\").\n```\n\nAnd voila: we have a system that looks like Jev. You give it classifier tasks, and you get back simple answers with “no hallucinations”, and your output token count stays lean (so you can call them free).\n\n(no hallucinations here doesn’t mean it can’t be wrong, it just means any answer we get back is definitely answer-shaped. This is TypeSafe’s marketing term, not mine, please don’t @ me)\n\n## How to make a Jev competitor\n\nNone of this is all that clever, original, or secret. [TheoLeeCJ/openjev](https://github.com/TheoLeeCJ/openjev) does essentially exactly this, as does the unrelated [ekzhang/openjev-sglang](https://github.com/ekzhang/openjev-sglang) but with some more cleverness.\n\nThere are some slightly different approaches too: [daseinlabs/open-jev](https://github.com/daseinlabs/open-jev) does almost the same thing, except instead of mapping each answer to a single token like 0 or 1, it uses the whole potential response token text and combines the probability of each successive token in that answer. [vinnylarouge/jevlike](https://github.com/vinnylarouge/jevlike) trains its own scoring model.\n\nTypeSafe have released their [*own* adaptor](https://github.com/typesafe-ai/system-one-adapter-python) too for turning an LLM into a Jev-like. Because they want to benchmark against existing commercial models — for which they don’t have access to the raw output logit vector — they instead just ask the underlying models really nicely to choose an option as string, and ask them again if they got back something else. It’s a little more sophisticated than that (native structured outputs are used when available) but still. This doesn’t (in my opinion, or in theirs) create a particularly fair test, but I also don’t see how else they could have done it, so this also seems reasonable.\n\nHow do these compare to Jev in practice?\n\n**TheoLeeCJ/openjev**: almost exactly the technique I suggested above. Reconstructed 102 decisions from Typesafe’s published evaluation material, and used Qwen3.5-4B as the fronting model. 84.5% agreement with the reference answers, vs 88.3% that Jev got on that subset. Speed compared against itself by reading just the single token output asking the model for JSON, it got a 5x speedup. Tells us Qwen3.5-4B can be competitive in accuracy for some questions, and there’s definitely some speedup just from asking for a single token back. [results](https://github.com/TheoLeeCJ/openjev/blob/master/docs/RESULTS.md)\n\n**ekzhang/openjev-sglang**: also essentially the same single-token technique, and published *while I was writing this article* (of course). 1,000 sampled MMLU-Pro questions run against live Jev, which got 83%: Qwen3.6-35B-A3B got 59%, Qwen3.8-27B got 60.0% [MMLU-Pro results](https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26d5d7b3b2184a524fc61779dbcc1/evals/results/qwen38-27b/report.md), so Jev apparently much more accurate on more complicated questions. Then 3,270 BoolQ validation questions gave a much smaller gap: Qwen3.6-35B-A3B got 89% vs Jev’s 91.56% [BoolQ results](https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26d5d7b3b2184a524fc61779dbcc1/evals/results/boolq-2026-09-18/comparison/report.md)\n\n**S Anand**: also published *while I was writing the article*: asked the frontier models and Jev to classify intents from [BANKING77](https://huggingface.co/datasets/PolyAI/banking77), everything via OpenRouter (using OpenRouter’s new Decisions endpoint for Jev, regular one for everything else). Just asked for raw JSON outputs from the models, didn’t use the technique above, so this is more analogous to TypeSafe’s own benchmarks. Found Jev to be a little faster and a little cheaper than DeepSeek v4.1 Flash and GPT-5.6 Luna, at the cost of a little accuracy. On the surface a bad result for Jev, although only 77 samples, and each sample is tiny (like 8 words) which is liable to response-time differences between network costs. [results](https://sanand0.github.io/llmevals/jev/). I am a little skeptical of this result purely because (at the time I read it) the author says ” Jev is low-frontier not pareto optimal ” before showing it is absolutely on the pareto-frontier, but he may have fixed it by the time you read it.\n\n## The secret Jev sauce\n\nThese results suggest Jev is pretty good at this (esp as they’re a new lab), which isn’t surprising, because Jev has been optimized to be good at this. [The claimed optimizations are](https://typesafe.ai/blog/introducing-system-one-models-and-jev):\n\n- \nA top-secret LLM architecture that makes Jev better at performing this kind of task than a general model;\n- \nA new training method “ [Reinforcement Learning for Calibrated Decisions (RLCD)](https://docs.typesafe.ai/introduction/machine-learning-primer) ” that makes Jev better at performing this kind of task than a general model;\n- \n“ [Parallel sampling](https://docs.typesafe.ai/introduction) ”, which means they can answer several questions in one sweep, which should make it fast and cheap.\n\nAs an outsider it is of course impossible to determine how much advantage the first two provide in quality and speed.\n\nThe really interesting point is going to be in a few days / weeks, when one of the frontier labs releases their inevitable Jev clone. The mean-girl/neck-beard/peanut-gallery take here of course is that the only clever thing Jev really does is derive a single token without generating any visible reasoning, an approach that’s so obvious four open-source projects have already done it with open models. I hope that take is wrong, but we will have to wait to find out.\n\n(genuinely, not a hater: I am excited to see them vindicated that they’ve created something really cool)\n\nWe don’t know — and are unable to tell from the benchmarks — what sticking [TheoLeeCJ/openjev](https://github.com/TheoLeeCJ/openjev) or [ekzhang/openjev-sglang](https://github.com/ekzhang/openjev-sglang) in front of a frontier model gives you in terms of speed, accuracy, and cost. We’ll either find that this reduces Jev’s claimed advantages substantially (perhaps to zero), or we’ll find that the architecture and training differences TypeSafe have done provide a durable optimization that the frontier labs don’t currently have.\n\n## In conclusion\n\nYou should now be able to reason about Jev.", "url": "https://wpnews.pro/news/you-could-have-built-jev", "canonical_source": "https://sgnt.ai/p/jev/", "published_at": "2026-09-18 15:04:53+00:00", "updated_at": "2026-09-18 15:26:06.368797+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-products", "natural-language-processing"], "entities": ["Jev", "typesafe.ai", "gpt-oss-120b", "Gemma 3"], "alternates": {"html": "https://wpnews.pro/news/you-could-have-built-jev", "markdown": "https://wpnews.pro/news/you-could-have-built-jev.md", "text": "https://wpnews.pro/news/you-could-have-built-jev.txt", "jsonld": "https://wpnews.pro/news/you-could-have-built-jev.jsonld"}}