{"slug": "jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router", "title": "Jev as a Quiet Layer Before the LLM: Building a Prompt-Aware Model Router", "summary": "TypeSafe AI, a San Francisco lab founded in 2024 that spent roughly two years in stealth, went public on 15 September 2026 with a $40 million seed round led by DCVC and opened early access to its Jev \"System One Model,\" priced at $0.042 per million input tokens with output tokens free and typical calls of 70 to 500 milliseconds. Jev does not generate text; developers pass a JSON state and typed questions (Choice, Score and Noul) and receive typed answers with confidence scores and probabilities that code can branch on, positioning Jev as a fast, cheap routing layer that decides which LLM should handle a prompt. TypeSafe claims Jev is many times faster and cheaper than frontier LLMs on its own workflows, figures the company says come from its own evaluations and that it says should be treated as claims until independent testing confirms them.", "body_md": "Not every prompt needs the same LLM. A greeting does not need a 120B reasoning model, and a question about today’s exchange rate is useless without web search. In this post I build a small model router where an AI model that never writes a word decides which LLM should answer.\n\nThat model is **Jev**, from TypeSafe AI. We will go step by step, what Jev is, where it comes from, how it works, and then how we can nuse it to route prompts across four models in about 100 lines of Python.\n\nJev is a **System One Model**, built to make fast, structured decisions that software can use directly. It does not chat and it does not write paragraphs. We give it some information and a few typed questions, and it returns typed answers our code can branch on.\n\nThe name System One comes from Daniel Kahneman’s *Thinking, Fast and Slow*:\n\nSo Jev is not a replacement for an LLM. It is the quick judgement you make before you spend money on one. TypeSafe describes it as a smart if-statement that sits between hard-coded logic and a full LLM call. It is meant for jobs like classification, routing, scoring and branching.\n\nJev comes from **TypeSafe AI**, an AI lab based in San Francisco and founded in 2024. The company spent roughly two years in stealth. It went public on 15 September 2026, announcing a $40 million seed round led by the venture firm DCVC, and opened Jev to developers in early access on the same day.\n\nThe company has three co-founders:\n\nThe name “Jev” is a nod to the 19th century economist William Stanley Jevons and the Jevons paradox. Jevons noticed that when steam engines became more efficient with coal, Britain did not use less coal. It used far more, because cheaper energy made many new uses worthwhile. TypeSafe’s bet is the same idea applied to AI. If a decision becomes cheap and fast enough, developers will put intelligence into thousands of places where a full LLM call never made sense.\n\nAt launch, TypeSafe priced Jev at $0.042 per million input tokens, with output tokens free, and said a typical call takes between 70 and 500 milliseconds. The company has also published comparisons claiming Jev is many times faster and cheaper than frontier LLMs on its own workflows. Those numbers come from TypeSafe’s own evaluations, so they are best treated as claims until independent testing confirms them.\n\nEvery call to Jev is built from the same three pieces.\n\nThe first is the **state**. This is a plain JSON object containing everything Jev is allowed to look at. It could be a support ticket, a product listing, a user message, or any other information your app has. Jev reads it but does not change it.\n\nThe second is the **questions**. These describe what you want decided about the state. Each question is written in plain English, and each has a type that fixes the shape of its answer.\n\nThe third is the **answers**. Jev returns one answer per question, and because the type was declared in advance, your code knows exactly what each answer looks like before the call is even made.\n\nThere are three question types, and between them they cover most decisions software needs to make.\n\nA **Choice** question asks Jev to pick one option from a list you define. For example, you might ask whether a ticket is about billing, a bug, or a feature request. You describe each option in a sentence, and Jev returns the option it picked, a confidence score, and a probability for every option. The full probability spread is useful because it tells you when two options were close.\n\nA **Score** question asks Jev to place something on a scale. You define the scale by writing a short description for each point on it, so a three-point scale might run from “trivial” to “moderate” to “very involved”. Jev returns a number on that scale, which can land between points (such as 0.89 on a 0 to 2 scale), plus a confidence score.\n\nA **Noul** question is a yes or no question. Instead of a hard yes or no, Jev returns the probability of yes, a number between 0 and 1. Your code then decides where the threshold should be, which keeps the policy in your hands rather than the model’s.\n\nThe big practical benefit here is reliability. With a normal LLM, getting a decision out means asking for JSON, hoping it is valid, parsing it, and handling the times it isn’t. With Jev, the answer is always the shape you asked for, so the code around it stays short and boring, which is exactly what you want in production.\n\nNow let’s apply Jev to the routing problem. The idea is simple to describe. A prompt comes in, Jev looks at it, and the prompt is forwarded to whichever LLM is best suited to answer. Jev itself never replies to the user. It only decides who should.\n\nBefore writing any Jev code, we need to decide which kinds of requests we wanted to tell apart, and which model should handle each. Say, we settled on four routes.\n\nThe **fast** route handles simple requests such as greetings, well-known facts, definitions and short rewrites. These go to Llama 3.1 8B Instant on Groq, which is small and very quick.\n\nThe **coding** route handles anything about writing, fixing or explaining code. These go to GPT-5 mini from OpenAI.\n\nThe **reasoning** route handles multi-step maths, logic puzzles, planning and decisions that need weighing up. These go to the open-weight GPT-OSS 120B model, served on Groq.\n\nThe **web_search** route handles questions that depend on recent information, such as news, prices, weather or scores. These go to Groq’s Compound model, which has web search built in.\n\nA nice detail is that all three providers (OpenAI, Groq and Gemini, which I kept as an option) support the OpenAI API format. That means one client library works for every route, and only the API key and base URL change. In code, the routes are just a small dictionary:\n\n```\nROUTES = {    \"fast\":       (\"groq\",   \"llama-3.1-8b-instant\"),    \"coding\":     (\"openai\", \"gpt-5-mini\"),    \"reasoning\":  (\"groq\",   \"openai/gpt-oss-120b\"),    \"web_search\": (\"groq\",   \"groq/compound\"),}\n```\n\nFor a router, the only thing Jev needs to see is the user’s prompt, so the state is a single field:\n\n```\nstate = {\"prompt\": prompt}\n```\n\nThe key name matters, because questions refer to the state by name. When a question says “What kind of help is the user asking for in prompt?”, the backticks around prompt point Jev at that field. In a richer app, you might have several fields, such as the latest message, a short summary of earlier conversation, and the user’s subscription plan, and each question could refer to whichever field it needs.\n\nIt is worth keeping the state crisp. Jev is priced by input tokens, and every extra token also adds a little latency. Sending only what the decision actually depends on keeps the router cheap and fast. If routing quality ever suffers because Jev lacks context, that is the moment to add another field, not before.\n\nThis is the most important design step, because the questions decide what the router can see. I used three, one of each type, and each captures a different signal.\n\nThe first is a Choice question called task_type, which asks what kind of help the user wants. The options mirror the routes: simple chat, coding, reasoning, current information, and a catch-all “other”.\n\nThe second is a Score question called complexity, which asks how much work a good answer would take, on a scale from 0 (a one-line answer) to 2 (several steps or a long, careful answer). This catches cases where the task type alone is not enough, for example a “simple chat” request that turns out to need a long and careful reply.\n\nThe third is a Noul question called needs_fresh_info, which asks whether answering correctly needs information from the last few weeks. This is a separate safety net for time-sensitive prompts, even ones Jev does not label as current information.\n\n```\nQUESTIONS = {    \"task_type\": {        \"type\": \"choice\",        \"instructions\": \"What kind of help is the user asking for in `prompt`?\",        \"criteria\": {            \"simple_chat\":  \"Greetings, small talk, a well-known fact, a definition, a short rewrite or translation\",            \"coding\":       \"Writing, fixing, explaining or reviewing code, shell commands, SQL or regex\",            \"reasoning\":    \"Multi-step math, logic puzzles, planning, or weighing options to make a decision\",            \"current_info\": \"News, live prices, exchange rates, weather, scores, schedules, anything that changes daily\",            \"other\":        \"None of the above\",        },    },    \"complexity\": {        \"type\": \"score\",        \"instructions\": \"How much work does a good answer to `prompt` take?\",        \"criteria\": [            \"A one-line answer or a well-known fact\",            \"A short explanation or a small task done in one step\",            \"Several steps, careful checking, or a long detailed answer\",        ],    },    \"needs_fresh_info\": {        \"type\": \"noul\",        \"instructions\": \"Does answering `prompt` correctly require information from the last few weeks?\",    },}\n```\n\nTwo things made a noticeable difference while building this.\n\nFirst, describe each Choice option with concrete examples rather than a single word. An option described only as “coding” leaves room for interpretation. An option described as “writing, fixing, explaining or reviewing code, shell commands, SQL or regex” tells Jev exactly where the boundary is, and it makes borderline prompts like a regex question land in the right place.\n\nSecond, think of Score criteria as anchor points on a ruler. Each description defines what that position on the scale looks like. Jev can return values between anchors, so a prompt that is a bit more than a short explanation but not quite a long piece of work might score around 1.4. Clear anchors make those in-between values meaningful.\n\nWith the state and questions ready, calling Jev is a single HTTP request. All three questions go together in one call, and Jev answers them all against the same prompt:\n\nThe response contains an answers object with one entry per question, named the same as in the request. For task_type, you get the chosen option, its confidence and the probabilities for all options. For compelxity, you get the score and its confidence. For needs_fresh_info, you get the probability that the answer is yes. The response also reports how many input tokens were used, which makes it easy to show the cost of each routing decision.\n\nThis is where the design pays off. Jev’s job is to describe the prompt. The decision about what to do with that description stays in ordinary code that anyone on the team can read and change.\n\nA useful way to put it: **Jev states the facts, and your code sets the policy.**\n\nOur routing function checks a few rules in order, and the first one that matches wins.\n\nIf Jev’s confidence in the task type is below 0.5, the prompt goes to the reasoning model. Low confidence usually means the prompt is ambiguous or unusual, and the safest choice is to send it to the strongest model rather than gamble on a small one.\n\nIf the task type is coding, it goes to the coding model.\n\nIf the task type is current information, or the probability that it needs fresh information is 0.7 or higher, it goes to the model with web search.\n\nIf the task type is reasoning, or the complexity score is above 1.0, it goes to the reasoning model.\n\nAnything left over is a simple request and goes to the fast model.\n\n``` python\ndef choose_route(answers):    task  = answers[\"task_type\"]    cx    = answers[\"complexity\"][\"score\"]    fresh = answers[\"needs_fresh_info\"][\"noul\"]\nif task[\"confidence\"] < 0.5:        return \"reasoning\"    if task[\"choice\"] == \"coding\":        return \"coding\"    if task[\"choice\"] == \"current_info\" or fresh >= 0.7:        return \"web_search\"    if task[\"choice\"] == \"reasoning\" or cx > 1.0:        return \"reasoning\"    return \"fast\"\n```\n\nThe order of the rules is a deliberate choice. Uncertainty is checked first, because when the classification itself is shaky, nothing after it can be trusted. Coding comes before fresh information, so a question like “how do I call this new API?” goes to the coding model. The thresholds (0.5, 0.7 and 1.0) are starting points. Because they are plain numbers in plain code, tuning the router is a one-line change rather than a prompt rewrite.\n\nOnce a route is picked, the last step is to send the original prompt to the matching model. Because every provider speaks the OpenAI API, one function handles all of them:\n\n``` python\ndef call_model(route, prompt):    provider, model = ROUTES[route]    cfg = PROVIDERS[provider]    client = OpenAI(api_key=os.getenv(cfg[\"key\"]), base_url=cfg[\"base_url\"])    resp = client.chat.completions.create(        model=model,        messages=[{\"role\": \"user\", \"content\": prompt}],    )    return resp.choices[0].message.content\n```\n\nPut together, the whole pipeline reads almost like a sentence:\n\n```\nroute = choose_route(ask_jev(prompt)[\"answers\"])answer = call_model(route, prompt)\n```\n\nTo make the behaviour visible, I wrapped this in a small Gradio app. When you submit a prompt, it first shows Jev’s three answers, the route that was chosen and a one-line reason. Then, a moment later, it fills in the reply from the selected model. Seeing the decision before the answer is surprisingly useful while tuning, because you can tell straight away whether a wrong answer came from a bad route or from the model itself.\n\nTo test it, we used the well-known bat and ball puzzle: “A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?” It is a classic System 1 trap. The intuitive answer of 10 cents is wrong, and the right answer is 5 cents.\n\nJev classified it as a reasoning task with full confidence (1.00). It gave it a complexity score of 0.89 out of 2, with a confidence of 0.65. It put the chance that it needs recent information at just 0.01. Following the rules, the router sent the prompt to the reasoning model, GPT-OSS 120B on Groq.\n\nThe Jev call itself used 553 input tokens and cost $0.0000232, about $23 for every million routing decisions. It took 2127 ms end to end from my machine, which includes my network round trip to the API. There is a nice irony here: the model making the fast System 1 judgement correctly recognised that this prompt needs slow System 2 thinking.\n\nAfter building this, a few benefits stand out.\n\nThe answers are typed, not parsed. Choices, scores and probabilities arrive in a fixed shape, so there is no fragile parsing layer between the model and your logic.\n\nThe policy is readable. Routing rules live in normal code with explicit thresholds. Anyone on the team can see why a prompt went where it did, and changing the behaviour means changing a number, not rewording a prompt and hoping.\n\nConfidence is built in. Because every answer comes with a confidence or probability, the router knows when it is unsure and can fall back to a safer option instead of failing silently.\n\nThe pattern generalises. Nothing here is specific to model routing. Change the state and the questions, and the same structure can triage support tickets, flag content for moderation, score sales leads, or decide whether an agent should act or ask a human first.\n\nThe broader idea is a division of labour. A fast System One model makes the quick judgement, and the slow, expensive System Two models are saved for the work that actually needs them. Your users get the right model for each request, and you stop paying premium prices for “hi”.\n\n[Jev as a Quiet Layer Before the LLM: Building a Prompt-Aware Model Router](https://pub.towardsai.net/jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router-99d33aeb0f3c) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router", "canonical_source": "https://pub.towardsai.net/jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router-99d33aeb0f3c?source=rss----98111c9905da---4", "published_at": "2026-11-06 07:50:59+00:00", "updated_at": "2026-11-06 08:25:03.420562+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "large-language-models", "ai-startups", "developer-tools"], "entities": ["TypeSafe AI", "Jev", "DCVC", "Daniel Kahneman", "William Stanley Jevons"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router", "markdown": "https://wpnews.pro/news/jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router.md", "text": "https://wpnews.pro/news/jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router.txt", "jsonld": "https://wpnews.pro/news/jev-as-a-quiet-layer-before-the-llm-building-a-prompt-aware-model-router.jsonld"}}