{"slug": "your-agent-burns-llm-money-on-switch-statements-jev-claims-444x-less", "title": "Your Agent Burns LLM Money on Switch Statements. Jev Claims 444x Less", "summary": "TypeSafe AI released Jev, a \"System One\" model that answers structured choice, score, and boolean questions instead of generating text, claiming 193.6x faster and 444.6x cheaper performance than frontier LLMs on agent routing tasks. Rivet's Nathan Flurry characterized the model as \"just a really smart switch statement\" that does not replace GPT or Claude, and TypeSafe's own launch blog concedes the headline multiples represent the \"higher end of real world gains\" from a \"highly simplified\" demo. Jev became available on Vercel's AI Gateway on 16 September, with direct API keys in early access.", "body_md": "You pull last month's model calls for your agent and sort them by\n\nwhat they were for. Some are easy to justify. The agent wrote a\n\npatch, drafted a reply to a customer, summarised a stack trace.\n\nThen there is everything else. The agent asked a frontier model which\n\ntool to use next, and whether a failed step deserved a retry. It\n\nasked whether `rm -rf ./build` was safe to run. It asked if the JSON\n\na tool sent back looked sane. Each of those went out at frontier\n\nprices and came back as a handful of tokens you parsed into an enum.\n\nLook at the answers those calls can give. Pick one of four tools.\n\nContinue, retry, ask the user or stop. Safe enough, or not. A fixed\n\nset of cases, and some judgment to pick the right one. You have\n\nwritten that shape a thousand times. It is a switch statement.\n\nNathan Flurry of Rivet, [writing on X](https://x.com/NathanFlurry)\n\nabout a new model from TypeSafe AI, put it plainly: \"jev is just a\n\n*really* smart switch statement\". He also wrote \"jev does not replace\n\ngpt / claude\". He is right on both counts.\n\nTypeSafe's co-founder Diogo Almeida\n\n[announced Jev on X](https://x.com/CompleteSkeptic) on 15 September\n\nYou give it a structured text state and a set of questions. The\n\n[docs](https://docs.typesafe.ai/introduction) define three question\n\ntypes. Choice picks one option. Score places the state on an ordered\n\nscale. Noul returns a yes/no probability, and the AI SDK calls it\n\n`boolean`. You can mix all three in one request, and they are\n\nevaluated in parallel.\n\nWhat Jev gives up is string generation. It cannot write a sentence,\n\na query or a file path.\n\nThe homepage also claims \"Zero Hallucinations\". TypeSafe's own\n\n[launch blog](https://typesafe.ai/blog/introducing-system-one-models-and-jev)\n\nexplains what that means: \"Our number is not empirical. Schema\n\nmatching is guaranteed\". Every answer fits the options you declared.\n\nIt can still pick the wrong option.\n\nDirect API keys from TypeSafe are in early access. On 16 September,\n\n[Vercel's changelog](https://vercel.com/changelog/typesafe-ai-jev-now-available-on-ai-gateway)\n\nsaid Jev \"is now available on AI Gateway\".\n\nThe homepage banner reads \"193.6x Faster, 444.6x Cheaper\". Its\n\nfootnote says the multiples are \"based on workflows for System One\n\ntasks\".\n\nThe page also shows a side-by-side example. Jev answers in 0.114s for\n\n$0.000081. The LLM answers in 8.566s for $0.013880. Those two figures\n\ndivide to about 75x faster and 171x cheaper, so the banner is not the\n\nratio of that one run.\n\nThe launch blog qualifies the demo itself. It calls the query\n\n\"highly simplified\", says the shorter input \"paints our model in an\n\nadvantageous light\", and describes the banner figure as \"the higher\n\nend of real world gains\". Treat 444.6x as a best case, because the\n\nvendor does.\n\nThe launch blog's pricing shows where a gap that size can come from.\n\nInput is $0.042 per million tokens ($42 per billion). Output tokens\n\nare listed as \"FREE (too cheap to meter)\". For comparison,\n\n[The Register](https://www.theregister.com/ai-and-ml/2026/09/16/typesafe-ai-debuts-model-for-machines-that-plays-doom/5296711)\n\ngives GPT-5.6 Terra's price as $2.00 input and $12 output per\n\nmillion. That is about 48x on input, and the output side has no ratio\n\nbecause one side is zero. TypeSafe quotes end-to-end latency of 70ms\n\nto 500ms.\n\nTypeSafe also publishes [workflow evals](https://evals.typesafe.ai/).\n\nTwo things before the numbers. The evals are vendor-run, and the blog\n\nconcedes they may carry \"some bias\". And \"accuracy\" means agreement\n\nwith reference labels \"generated via an average of the responses of\n\nGPT-6 Astra and Claude Fable 5.1, both at high thinking\". The other\n\nmodels ran at their provider's default reasoning. A score here tells\n\nyou how often a model agrees with two frontier models, which is a\n\nnarrower thing than how often it is right.\n\nAggregate results across the four workflows, for six of the nine\n\nmodels on the page, vendor-reported:\n\n| Model | Accuracy | Cost per case | Latency | \n|---|---|---|---|\n| Jev | 67.8% | $0.0004 | 0.4s | \n| GPT-5.6 Luna | 66.8% | $0.0033 | 12.9s | \n| GPT-5.6 Terra | 67.9% | $0.0304 | 10.1s | \n| GPT-5.6 Sol | 74.1% | $0.0836 | 23.3s | \n| Claude Sonnet 5 | 67.8% | $0.1174 | 78.1s | \n| Claude Opus 5 | 73.1% | $0.1761 | 37.8s | \n\nStart with Terra, because the accuracy is level: 67.8% against\n\n67.9%. On these numbers Jev costs about 1/76th as much per case and\n\nanswers about 25 times faster. The costs are rounded to four decimal\n\nplaces, so read 76x as approximate.\n\nClaude Sonnet 5 matches Jev exactly at 67.8%, for about 294 times the\n\ncost per case and 195 times the latency.\n\nNow Sol. It is 6.3 points more accurate than Jev and about 209 times\n\nmore expensive per case.\n\nThe only row in that table whose multiple comes near the banner's\n\n444.6x is Claude Opus 5, at about 440x, and Opus 5 is the second most\n\naccurate model in the table.\n\nThe same eval page tested each model two ways: one big prompt, and a\n\nworkflow that decomposes the task into Choice, Score and Noul\n\nquestions. Every LLM got more accurate, cheaper and faster in\n\nworkflow mode. GPT-5.6 Luna went from 51.9% to 66.8%.\n\nPut that next to the table. Decomposing the task bought Luna 14.9\n\npoints. Moving the decomposed workflow from Luna to Jev bought one\n\nmore point and an 8x lower cost per case.\n\nThat changes the order of the work. You can decompose today, with the\n\nmodel you already call. Break \"what should the agent do now?\" into\n\nsmall typed questions and combine the answers in your own code. The\n\ndocs call it \"atomic questions, composed in code\". Once that is done,\n\nthe model question gets narrow: for these atomic questions, which\n\nmodel agrees with your labels often enough at the lowest cost?\n\nVercel's changelog lists the agent use cases it has in mind:\n\nselecting tools or subagents, the next action (continue, retry, ask\n\nthe user, halt), urgency or risk before an operation, and validating\n\noutputs and safeguards. Guillermo Rauch, Vercel's CEO,\n\n[wrote on X](https://x.com/rauchg) about fx, a Vercel tool whose\n\ndefault auto mode runs a safety reviewer on every command. That\n\nreviewer \"runs on GPT Luna today\", he wrote, and Jev is \"likely new\n\ndefault\". Likely, as of that post.\n\nThe design is one decision call per agent step. It asks every\n\ndecision question in parallel. Your code branches on the answers.\n\nAnything that needs words stays on the language model: tool\n\narguments, search queries, replies, patches. So does any decision\n\nJev is not confident about.\n\nThe saving comes from calls that already exist only to decide, like\n\nthe retry check or the command reviewer. If one LLM call picks a tool\n\nand writes its arguments today, moving the pick to Jev adds a\n\nrequest. That pays off only when the pick lets you skip the LLM call\n\nor send the arguments to a smaller, cheaper model.\n\nInstall the SDK and the\n\n[TypeSafe provider](https://ai-sdk.dev/providers/ai-sdk-providers/typesafe-ai):\n\n```\npnpm add ai@7 @ai-sdk/typesafe-ai\nexport TYPESAFE_AI_API_KEY=your-key\n```\n\nThe code below type-checks against `ai` 7.0.105 and\n\n`@ai-sdk/typesafe-ai` 3.0.2, with `allowImportingTsExtensions` on for\n\nthe `./decide.ts` import. It has not been run against the live API,\n\nand direct TypeSafe keys are early access. `askJev` sends all three\n\nquestions in one request:\n\n``` js\n// decide.ts\nimport { typeSafeAi } from \"@ai-sdk/typesafe-ai\";\nimport { experimental_evaluate } from \"ai\";\n\nexport type AgentState = {\n  goal: string;\n  lastObservation: string;\n  pendingCommand: string;\n};\n\nexport function askJev(s: AgentState) {\n  return experimental_evaluate({\n    model: typeSafeAi.evaluationModel(\"jev-latest\"),\n    state: s,\n    questions: {\n      nextTool: {\n        type: \"choice\",\n        instructions: \"Which tool should the agent use next?\",\n        criteria: {\n          search_docs: { includes: [\"Product questions\"] },\n          run_shell: { includes: [\"Builds\", \"Tests\"] },\n          edit_file: [\"Code changes\"],\n          none: null,\n        },\n      },\n      nextAction: {\n        type: \"choice\",\n        instructions: \"What should the loop do now?\",\n        criteria: {\n          continue: [\"Last step worked, goal not met\"],\n          retry: [\"Last step failed transiently\"],\n          ask_user: [\"Goal is unclear or needs consent\"],\n          stop: [\"Goal is met\"],\n        },\n      },\n      risk: {\n        type: \"score\",\n        instructions: \"How risky is the pending command?\",\n        criteria: [\n          \"Read-only\",\n          \"Reversible change\",\n          \"Deletes data or touches production\",\n        ],\n      },\n    },\n  });\n}\n```\n\n`state` is the agent's current situation as a plain object, and\n\n`pendingCommand` is whatever the last LLM turn proposed to run. Score\n\nlevels are ordered, least severe first, as in the provider docs'\n\nexample. A Choice question takes up to 255 options and\n\na Score question takes 2 to 10 levels. `answers.nextTool.choice` comes\n\nback typed as the union of your option names, so a misspelled option\n\nin a comparison is a compile error. The branching needs a few types\n\nof its own:\n\n```\n// decide.ts, continued\ntype Meta = {\n  typesafe?: { confidence?: Record<string, number> };\n};\n\n// Missing confidence counts as zero: the LLM decides.\nconst confidence = (m: unknown, id: string): number =>\n  (m as Meta | undefined)?.typesafe?.confidence?.[id] ?? 0;\n\nexport type Policy = {\n  minConfidence: number;\n  riskBlockAt: number;\n};\n\nexport type Step =\n  | { kind: \"tool\"; tool: string }\n  | { kind: \"retry\" }\n  | { kind: \"ask_user\" }\n  | { kind: \"stop\" }\n  | { kind: \"approve\"; command: string }\n  | { kind: \"llm\"; reason: string };\n```\n\nThe provider documents confidence at\n\n`providerMetadata.typesafe.confidence`, for Choice and Score answers\n\nonly. The helper reads it defensively. If the value is absent, the\n\nstep goes to the LLM instead of guessing.\n\nThen the switch statement, which is now an actual `switch`:\n\n```\n// decide.ts, continued\nexport async function decide(\n  s: AgentState,\n  p: Policy,\n): Promise<Step> {\n  const r = await askJev(s);\n  const { nextTool, nextAction, risk } = r.answers;\n  const sure = (id: string) =>\n    confidence(r.providerMetadata, id) >= p.minConfidence;\n\n  if (!sure(\"nextAction\")) {\n    return { kind: \"llm\", reason: \"next action unclear\" };\n  }\n  switch (nextAction.choice) {\n    case \"retry\":\n      return { kind: \"retry\" };\n    case \"ask_user\":\n      return { kind: \"ask_user\" };\n    case \"stop\":\n      return { kind: \"stop\" };\n  }\n\n  if (!sure(\"nextTool\") || nextTool.choice === \"none\") {\n    return { kind: \"llm\", reason: \"tool choice unclear\" };\n  }\n  if (\n    nextTool.choice === \"run_shell\" &&\n    (!sure(\"risk\") || risk.score >= p.riskBlockAt)\n  ) {\n    return { kind: \"approve\", command: s.pendingCommand };\n  }\n  return { kind: \"tool\", tool: nextTool.choice };\n}\n```\n\nThe next action is checked first, because a loop that should stop\n\ndoes not need a tool. The shell gate fails closed on uncertainty. A\n\nrisky command goes to a human, and so does a command whose risk Jev\n\nis unsure about. A confident wrong answer still runs the command, so\n\nkeep your deterministic guards in front of it, such as a command\n\ndenylist and a sandbox with no production credentials.\n\nThis gate has the most expensive miss in the loop. Run it in shadow\n\nmode next to your current reviewer, and let it decide alone only\n\nafter it agrees with your labelled commands.\n\nBoth numbers in `Policy` are inputs on purpose. The SDK checks that\n\n`score` is the probability-weighted mean of the level indices, so\n\nwith the three levels above it runs from 0 to 2. The right cut-off\n\ninside that range comes from your data. Log raw `risk.score` values\n\nfor a few known-safe and known-dangerous commands first. Then pull a\n\nfew hundred steps from your logs, label them, run them through\n\n`decide`, and set the thresholds where the misses stop being\n\nacceptable.\n\nLabel the output of `decide` as a whole, not each question on its\n\nown. Anthony Maio, in a\n\n[skeptical read of the launch](https://anthonymaio.substack.com/p/jev-the-language-model-that-wont),\n\npoints out that \"individually calibrated judgments do not\n\nautomatically compose into a calibrated workflow once you run them\n\nthrough thresholds, weights, and branches\". That function is\n\nthresholds and branches.\n\nThe loop calls all of this from one place:\n\n``` js\n// step.ts\nimport { decide, type AgentState, type Policy, type Step }\n  from \"./decide.ts\";\n\n// Your existing LLM call. It returns the same Step union.\ndeclare function askLlm(\n  s: AgentState,\n  reason: string,\n): Promise<Step>;\n\nexport async function nextStep(\n  s: AgentState,\n  p: Policy,\n): Promise<Step> {\n  let step: Step;\n  try {\n    step = await decide(s, p);\n  } catch {\n    // Provider or network failure, after the SDK's retries.\n    return askLlm(s, \"decision model unavailable\");\n  }\n  if (step.kind !== \"llm\") return step;\n  return askLlm(s, step.reason);\n}\n```\n\n`askLlm` is the call your loop makes today, trimmed to return the\n\nsame `Step` union, so the loop never needs to know who decided. The\n\nSDK retries 429 and 529 responses up to `maxRetries`, which defaults\n\nto 2, before it throws. After that the language model takes over, so\n\nan outage or an access limit slows the agent down without stopping\n\nit. Log `usage` from every evaluate result, and which branch\n\nanswered each step. Without that log, every number you have is\n\nTypeSafe's.\n\nOn Vercel AI Gateway, the model is a string: `model: \"typesafe-ai/jev\"`.\n\nVercel's example also passes\n\n`providerOptions: { gateway: { zeroDataRetention: true } }`. Confirm\n\nthe confidence metadata comes back on that path before you trust the\n\nthreshold. If it does not, the helper above sends every step to the\n\nLLM, which is safe and saves you nothing.\n\nThe calculator below runs as-is. It treats one agent step's\n\nbatch of decisions as one eval case. TypeSafe's cases are workflows\n\nof several questions too, but they are not your questions, so treat\n\nthe mapping as rough.\n\n```\n// cost.ts\n// Per-case numbers: TypeSafe's aggregate workflow evals.\n// Vendor-reported, vendor-run. Swap in your own logs.\ntype Row = {\n  model: string;\n  acc: number; // share of cases matching the labels\n  usd: number; // cost per case\n  sec: number; // latency per case\n};\n\nconst ROWS: Row[] = [\n  { model: \"Jev\", acc: 0.678, usd: 0.0004, sec: 0.4 },\n  { model: \"GPT-5.6 Luna\", acc: 0.668, usd: 0.0033, sec: 12.9 },\n  { model: \"GPT-5.6 Terra\", acc: 0.679, usd: 0.0304, sec: 10.1 },\n  { model: \"GPT-5.6 Sol\", acc: 0.741, usd: 0.0836, sec: 23.3 },\n  { model: \"Claude Sonnet 5\", acc: 0.678, usd: 0.1174, sec: 78.1 },\n  { model: \"Claude Opus 5\", acc: 0.731, usd: 0.1761, sec: 37.8 },\n];\n\nconst perDay = Number(process.argv[2] ?? 10_000);\nconst days = 30;\nconst jev = ROWS[0];\n\nconst fmt = (n: number) => n.toLocaleString(\"en-US\", {\n  maximumFractionDigits: 0,\n});\n\nconsole.log(`${fmt(perDay)} cases/day over ${days} days`);\nfor (const r of ROWS) {\n  const month = r.usd * perDay * days;\n  const misses = perDay * (1 - r.acc);\n  console.log(\n    r.model.padEnd(14),\n    `$${fmt(month)}/mo`.padStart(12),\n    `${(r.usd / jev.usd).toFixed(0)}x`.padStart(5),\n    `${fmt(misses)} misses/day`.padStart(17),\n    `${r.sec}s`.padStart(6),\n  );\n}\n```\n\nRun it with Node's type stripping:\n\n``` bash\n$ node --experimental-strip-types cost.ts\n10,000 cases/day over 30 days\nJev                 $120/mo    1x  3,220 misses/day   0.4s\nGPT-5.6 Luna        $990/mo    8x  3,320 misses/day  12.9s\nGPT-5.6 Terra     $9,120/mo   76x  3,210 misses/day  10.1s\nGPT-5.6 Sol      $25,080/mo  209x  2,590 misses/day  23.3s\nClaude Sonnet 5   $35,220/mo  294x  3,220 misses/day  78.1s\nClaude Opus 5    $52,830/mo  440x  2,690 misses/day  37.8s\n```\n\nA \"miss\" here is a case that disagrees with TypeSafe's reference\n\nlabels, nothing more.\n\nAt 10,000 cases a day, Jev runs $120 a month. Claude Sonnet 5 runs\n\n$35,220 at the same 67.8%, and Terra runs $9,120 at 67.9%. Sol cuts\n\nmisses from 3,220 a day to 2,590, which is 630 fewer, for $24,960\n\nmore a month. Whether that trade is worth it depends on what one\n\nmiss costs you. A wrong tool pick that the next step corrects costs a\n\nfew seconds. A wrong \"this command is safe\" can cost you a database.\n\nThe latency column matters inside a loop. On these per-case figures,\n\na 30-step run waits about 12 seconds on Jev decisions and about 5\n\nminutes on Terra's.\n\n**The answer is text.** Tool arguments, a search query, a commit\n\nmessage, a reply. Jev can choose `search_docs`. It cannot write the\n\nquery.\n\n**There are more than 255 options.** Picking one file out of a\n\n2,000-file repo is not a Choice question. Narrow the candidates in\n\ncode first, or leave it on the LLM.\n\n**The state is an image.** TypeSafe says Jev works on structured\n\ntext state, not images \"(yet…)\". A computer-use agent deciding from\n\nscreenshots keeps its current model.\n\n**You have no key yet.** Direct TypeSafe API keys are early access.\n\nVercel's changelog lists Jev as available on AI Gateway, with no\n\nwaitlist mentioned. Build the fallback path first either way, because\n\nit is also what runs when the decision call fails.\n\n**The workflow is hard and a miss is expensive.** The aggregate hides\n\na spread. On TypeSafe's invoice processing workflow, Jev scored 61.8%\n\nand GPT-5.6 Sol scored 79.1%, a 17.3-point gap. On security incidents\n\nit was 61.7% against Claude Opus 5's 66.2%. On customer service the\n\ngap narrows to 76.0% against Sol's 78.3%. Anything that moves money\n\nor touches production stays on the language model until your own\n\nlabels say otherwise.\n\nSearch your agent loop for LLM calls whose output you parse into an\n\nenum, a boolean or a number. Each one is a candidate.\n\nDecompose those first, on the model you already use, and measure\n\nagreement against a labelled sample of your own steps. The evals say\n\nthat step alone moves accuracy, cost and latency in the right\n\ndirection. Then put Jev behind the same questions, with the\n\nconfidence gate and the fallback above, and compare cost per case and\n\nagreement on your labels.\n\nKeep the language model on the words, and on the decisions where one\n\nmiss costs more than a month of savings.\n\nI wrote *AI That Acts* for the loop in this post, where a model picks\n\na tool and your code runs it. It builds tool calling, function\n\nschemas and a first working agent in TypeScript. My *AI That Ships*\n\npicks up the confidence gates, fallbacks and cost-per-case logging.\n\nIf you want the agent patterns without the build, my *AI Agents Pocket Guide* is the short read. Start with whichever piece your\n\n**AI in TypeScript** — five books, one path from your first LLM call to agents in production:\n\n**Pocket Guides for Developers** — short references you can finish in an evening:\n\nGoing deeper on tracing and evals: [Observability for LLM Applications](https://www.amazon.de/-/en/dp/B0GXNNMKVF).", "url": "https://wpnews.pro/news/your-agent-burns-llm-money-on-switch-statements-jev-claims-444x-less", "canonical_source": "https://dev.to/gabrielanhaia/your-agent-burns-llm-money-on-switch-statements-jev-claims-444x-less-23l4", "published_at": "2026-09-17 08:42:22+00:00", "updated_at": "2026-09-17 08:53:15.760924+00:00", "lang": "en", "topics": ["ai-products", "large-language-models", "ai-agents", "ai-infrastructure"], "entities": ["TypeSafe AI", "Jev", "Nathan Flurry", "Rivet", "Diogo Almeida", "Vercel", "AI Gateway", "GPT-5.6 Terra"], "alternates": {"html": "https://wpnews.pro/news/your-agent-burns-llm-money-on-switch-statements-jev-claims-444x-less", "markdown": "https://wpnews.pro/news/your-agent-burns-llm-money-on-switch-statements-jev-claims-444x-less.md", "text": "https://wpnews.pro/news/your-agent-burns-llm-money-on-switch-statements-jev-claims-444x-less.txt", "jsonld": "https://wpnews.pro/news/your-agent-burns-llm-money-on-switch-statements-jev-claims-444x-less.jsonld"}}