The Great Unbundling of the LLM

TypeSafe AI released a non-generative model called Jev on September 15 that answers typed questions about program state in a single parallel pass, reportedly in the low hundreds of milliseconds, and developers integrated it into browser agents, options market makers, a Mario speedrun, and an autonomous drone within two days. Independent teams reported gains: Vercel engineers replaced a frontier classifier for shell-command safety with Jev and reported answers five to eighteen times faster and more accurate, while Bryo AI found Gemini marginally more accurate but ten to twenty times more expensive for email triage. TypeSafe's cost and speed figures — fractions of a cent per decision and free output tokens — are self-reported and unreproduced, scored against consensus labels from other frontier models rather than ground truth.

← Blog https://seldon-ai.com/blog The great unbundling of the LLM On September 15, a company you had probably never heard of shipped a model that cannot write a sentence. By the seventeenth, developers had bolted it into browser agents, options market makers, a Mario speedrun, and an autonomous drone. The takes wrote themselves: cute trick, cheap classifier, wake me when it can hold a conversation. Every one of them missed it. What actually happened last week is that the monolith cracked — and the crack runs exactly along the line we have been pointing at for months. The week the monolith cracked The model is Jev https://forkast.news/typesafe-ais-jev-is-not-an-llm-and-that-may-be-the-point/ , from a startup called TypeSafe AI. It does not generate text. You hand it program state and a set of typed questions, and it answers all of them at once — in a single parallel pass, reportedly in the low hundreds of milliseconds — with typed values and calibrated probabilities instead of prose. TypeSafe calls the category "System One," after Kahneman's fast, intuitive mode of thought. The whole surface is three question types practical guide https://dev.to/valyuai/how-to-use-jev-a-practical-guide-to-typesafes-system-one-model-g5e : pick one option from a set, place something on a scale, or answer yes/no as a probability. That is the entire API. It is almost aggressively small. Three things make it a genuinely different kind of object than the chat model you are used to. It cannot return a value outside your schema — malformed output is not a bug you handle, it is a state that cannot occur. Its confidence is calibrated: TypeSafe trains with what it calls Reinforcement Learning for Calibrated Decisions, tuning probabilities against outcomes rather than human taste Forbes https://www.forbes.com/sites/lanceeliot/2026/09/18/new-reinforcement-learning-for-calibrated-decisions-makes-ai-headlines-but-look-past-the-hype/ , so the number it hands back actually predicts whether it is right. And it knows nothing beyond the state you pass it — it cannot look anything up, and it cannot write. It is not a smaller brain. It is a different organ. Be a skeptic about the numbers, because you should be. The cost and speed figures — fractions of a cent per decision, output tokens free, tens of milliseconds — are self-reported, self-run, and unreproduced, scored against consensus labels assembled from other frontier models rather than any ground truth. If you were underwriting TypeSafe the company, that would matter enormously. But that is not the interesting variable, and treating it as one is how you miss the plot. The signal is not the benchmark. The signal is that a non-generative decision primitive grew a working ecosystem in a weekend. You do not get that from a novelty. You get that when a large amount of latent demand has been waiting for a shape to pour itself into. And the early evidence is no longer only coming off TypeSafe's own slides. Within days, independent teams reported their own numbers, as covered by TechCrunch. Engineers at Vercel ripped out a frontier classifier — the one deciding which shell commands are safe to run — dropped Jev in its place, and by their account got answers five to eighteen times faster, and more accurate besides. A team at Bryo AI, testing email triage against Gemini, found the frontier model marginally more accurate but ten to twenty times more expensive; what actually won them over was not speed but the honest probability stapled to every answer, the one thing, their CTO said, that hands back a real probability. File these as single-company anecdotes, because that is what they are. But they are independent anecdotes, they rhyme, and the thing they keep circling is not raw accuracy. It is the calibrated confidence — the number that tells a piece of software when to trust the machine and when to escalate. Hold that thought; it is load-bearing. The bet we already made Here is the uncomfortable thing for the "cute trick" crowd. The reason Jev's launch reads as obvious in hindsight is that the industry's default was never actually sound; it was just convenient. We reach for a frontier LLM to sort a ticket, score a risk, route a request, or decide whether a passage is relevant because it is the thing already in our hand — not because generating text token by token is the correct way to make a decision. A switch statement that needs a sliver of judgment does not need a 200-billion-parameter novelist. It needs a fast, reliable answer and a number that says how much to trust it. You do not have to take our word for the diagnosis. Take it from the man who helped build the thing. Diogo Almeida, TypeSafe's founder, worked on ChatGPT and helped invent RLHF — the training technique that arguably lit this entire era — and he describes what came after as a kind of heartbreak. We have lightning in a bottle, he told TechCrunch, and yet it is not useful. His conclusion lands on exactly the bone we have been gnawing: the models have spent years becoming superhuman at human language , and human language is the wrong target for automation, because the thing on the other end of a backend decision is not a human. It is other software. Computers, as he puts it, speak a different language. That is the sharpest single sentence anyone has written for this thesis, and it comes from inside the building that poured the monolith. We have been saying this in public since the summer — that a large, measurable fraction of production LLM traffic is decision-shaped work wearing a generation costume, in the silent epidemic of LLM technical debt https://seldon-ai.com/blog/silent-epidemic-llm-tech-debt , and that you can put a meter on it in how much of your LLM bill is just ETL https://seldon-ai.com/blog/how-much-of-your-llm-bill-is-just-etl . But the trace-level diagnosis was only ever the surface of the bet. The real wager Seldon made — back when the entire field was chanting bigger, more general, one model to rule them all — was structural, and it was contrarian: That is a strong claim, and for most of the last two years it was an unpopular one. The money, the headlines, and the roadmaps all pointed the other way: toward consolidation, toward a single omnimodel that would eventually swallow every task in the stack. We built for the opposite outcome. Jev is the first loud confirmation that we read the tape correctly. "One size fits all" is always an idea whose time has come and gone If you want to know how the LLM story ends, stop reading AI Twitter and read the history of every other general-purpose engine in computing. The pattern is so consistent it is almost a law: a beautifully general tool arrives, absorbs a heterogeneous mess of workloads because it is the easiest option, gets pushed past its economic limits by scale, and then — reliably, unsentimentally — fragments into specialized parts, while the profit quietly relocates to whatever layer learns to orchestrate them. The database people already lived through this, and they even wrote down the moment it turned. In 2005, Michael Stonebraker and Uğur Çetintemel published a paper with a title that has aged into prophecy: One Size Fits All: An Idea Whose Time Has Come and Gone https://cs.brown.edu/~ugur/fits all.pdf . Their argument was that the monolithic, do-everything relational database — the crown jewel of enterprise software — would lose most of its workloads to purpose-built engines, because a system designed for everything is optimal for nothing. They were right to the point of being boring about it. The unbundling gave us columnar warehouses for analytics, time-series stores, search engines, streaming systems, and, lately, vector databases. "The database" did not die. It shattered into a shelf of specialists, and the value floated up into the query planners, data platforms, and routing layers that decide where a given query should actually run. It is not just databases. It is the shape of the field. | The monolith | Fragmented into | Where the value went | |---|---|---| | The mainframe | Client/server, then microservices | Orchestration and cloud platforms | | The general-purpose DBMS | OLAP columnstores, search, time-series, streaming, vector | The query planner and data platform | | The CPU | GPU, TPU, NPU, DSP, and fixed-function silicon | The compiler and runtime that schedule across them | | The frontier LLM | Decision models, extractors, small specialists, plain code | The router/compiler that decides which one runs | The CPU row is the one to sit with, because it is the closest rhyme. For decades the answer to "make it faster" was "buy a better general-purpose processor." Then the workloads — graphics, then deep learning — got large and specific enough that general silicon became the expensive way to do a narrow thing, and the work peeled off onto GPUs, TPUs, and a growing zoo of accelerators. Notice what did not happen: the CPU did not vanish. It became the coordinator, the thing that runs the irregular, branch-heavy, genuinely general work and dispatches the specialized work to specialized hardware. That is precisely the role the frontier LLM is about to be demoted into. Clayton Christensen gave the economics of this its name: when a product overshoots what most of its uses require, it modularizes, and profit migrates to the adjacent layer where a bottleneck still bites. Frontier generation is overshooting the vast, dull middle of the AI backend the way a supercomputer overshoots a spreadsheet. The overshoot is the tell. It is the condition under which unbundling always begins. The monolith unbundles itself Here is the part the hardware analogy misses, and the part that makes this unbundling stranger and faster than the ones before it. Silicon did not design its own successors. The frontier LLM does. The mechanism is a loop we have described before and will keep describing because it is the entire game: learn to discover, then compile down. You point the big, general, expensive model at a messy workload precisely because it is a universal adapter — it will absorb almost any input and produce almost any output shape before anyone has figured out what the task really is. That is the model at its best: not as a permanent runtime, but as an instrument of discovery. Then you watch. The traces pile up. And buried in them are the workflows that have quietly stopped being open-ended — the ticket that always routes the same way, the field that always extracts into the same schema, the passage that gets the same relevance verdict. Those are no longer questions. They are settled computations still being billed at frontier prices. Once you can see that structure, you can lift it out. Infer the contract. Isolate the minimal state the decision actually depends on. Compile the stabilized workflow down to the cheapest primitive that clears the bar — a cache, a parser, a few lines of code, a small model, a calibrated decision — and keep the frontier model behind it as the fallback for the genuinely ambiguous tail. Sit with the recursion in that, because it is genuinely elegant: the general model is the tool you use to learn how to stop needing the general model. It is the scaffolding around a building that, once poured, holds itself up. Every unbundling in computing needed some agent to find the seams and forge the replacement parts. This is the first one where the monolith hands you the blueprints for its own disassembly, if you are instrumented well enough to read them. Seldon is the machine that reads them and does the cutting. We built that machine before the shelf of replacement parts existed — which is either reckless or early, and last week it started looking a lot like early. What comes off the monolith first Unbundlings have an order. The parts that break off first are the ones where specialization pays the most and risk is the lowest — the high-volume, low-ambiguity, decision-shaped work sitting in the fat middle of the distribution. Not the hard, rare, creative tail. The boring, constant, load-bearing center. Decompose almost any workhorse prompt and you find it is not one computation but several wearing a trench coat: a classification, an extraction, a normalization, a lookup, maybe a genuine flash of reasoning, all bundled into a single call because a prompt was the easiest place to dump work nobody had formalized yet. The classification and the scoring and the routing are the pieces that come off first. And a calibrated decision primitive is the near-perfect part to catch them, for a reason most of the launch commentary skated past: calibration is what makes an unbundled architecture safe to operate. The whole cascade — serve the confident slice cheaply, escalate the rest — is a bluff unless the confidence number is honest. A primitive whose probability actually tracks its accuracy lets you set one threshold per action, scaled to what being wrong costs, and know you are not quietly shipping garbage on the 80% you stopped paying attention to. Follow that logic and the architecture is not something you design. It falls out on its own: We described this cascade months ago as the endpoint of the ETL argument. Read TypeSafe's own guidance and it arrives at the same staircase from the runtime side — do not replace your LLM, they say, wrap it in a cascade and send only the hard minority up to it. The launch-week demos are all this shape: a browser agent that only invokes a writing model to type free text, a drone that keeps Jev's judgment strictly advisory above a hard-coded safety reflex, a pipeline that runs a generative model for summaries and the decision model for the label. Two camps, opposite starting points, one architecture. When independent efforts converge on the same structure, that structure is usually not a fashion. It is the terrain. Jev is a data point, not a dependency Now the part where I have to be disciplined, because the lazy version of this essay ends with "and that is why you should pipe Jev into Seldon." That is not the claim, and making it would be selling the small story again. Jev is not a component we need. Jev is a data point — the first high-visibility confirmation of a fragmentation we had already priced in. Its value to our argument has nothing to do with whether TypeSafe wins, or whether those benchmarks survive independent scrutiny, or whether the price holds. It is this: a specialized, non-generative primitive drew a real ecosystem in 48 hours, which means the demand for unbundled parts is not a thesis on a slide anymore. It is observable behavior. There will be more parts. Some will be better than Jev. It does not matter which ones survive, because Seldon's bet was never on any single primitive. It was on the plurality — and on owning the layer that decides, per workflow, from live data, which primitive each stabilized computation should compile down to. Watch where the first users point, too, because it gives away where the value is already sliding. Ask the people kicking Jev's tires what it is for , and a striking number of them do not say "classification." They say routing — using a cheap, calibrated primitive to decide in real time which model a request even deserves, a job that is faintly absurd to hand to an expensive LLM. Armin Ronacher, whose company builds an open-source model harness, named precisely that use. Others are reaching for it as a guardrail: a cheap monitor watching a fleet of expensive agents for jailbreaks and misbehavior, because you cannot afford to post a frontier model as the sentry over every other frontier model. Routing. Monitoring. Orchestration. In week one, unprompted, the market is not merely grabbing a part off the shelf — it is feeling around for the layer that coordinates the parts. That layer is the entire point, and it is the one we have spent our time building. This is where being early stops being a liability. Amara's law — we overestimate a technology's impact in the short run and underestimate it in the long run — is the graveyard of infrastructure companies that were correct too soon. The specific danger of a bet like ours is not being wrong. It is being right in a year when the market has not shown up yet, standing in an empty store you built for traffic that has not arrived. Jev is the first crowd through the door. It tells us the store is in the right place. Every specialized primitive that ships after it makes the discover-then-compile engine we already built more valuable, because it widens the menu of things we can compile down to . We poured the road before there were cars. Last week the first car went by fast, and a lot of people suddenly wanted to know who paved it. But didn't Sutton settle this? If you have been nodding along and you know the field, an alarm has probably been going off for a few paragraphs, and intellectual honesty demands I walk over and shut it off rather than pretend it isn't ringing. Richard Sutton's The Bitter Lesson http://www.incompleteideas.net/IncIdeas/BitterLesson.html — arguably the most quoted essay in modern AI — argues that across seventy years, the general methods that scale with computation have crushed the clever, handcrafted, human-knowledge-laden ones, over and over, in chess, Go, speech, and vision. "Replace the big general model with small specialized parts" sounds like exactly the sin Sutton catalogs: betting on human-imposed structure right before the general method eats it alive. So does the unbundling contradict the Bitter Lesson? Mostly no — and the places where it flirts with a real tension are worth being honest about rather than papering over. Mostly no, for three reasons. First, Sutton is answering a different question than we are. The Bitter Lesson is about how to acquire capability at the research frontier — how to build a system that can do the hard thing at all. Seldon is about how to serve a capability you already have, cheaply and reliably, a billion times, in production. Nobody in that essay is arguing you should burn frontier-scale compute to decide whether a support ticket is about billing. Second, we do not hand-encode knowledge — the actual target of Sutton's warning. Our specialized parts are learned or mined from data : distilled models, workflows discovered from traces, and Jev itself is a trained model. Distillation and specialization from data are general-methods techniques, not the handcrafted features Sutton buried. Third, and most pointedly, the deep instruction of the Bitter Lesson is let the method discover the structure instead of imposing it. Discover-then-compile is that instruction taken literally: we do not declare which workflows are settled, we learn it from the traces. The specialization is emergent, not assumed. Where is the genuine tension? Right here: the frontier moves, and general inference keeps getting cheaper. A specialized path that pencils out today can be leapfrogged tomorrow when a bigger, cheaper general model makes the specialization not worth its own maintenance. That is real, and any honest version of this argument has to hold it. The answer is not to deny it; it is that our specialization is built to be reversible . It is data-driven, gated on calibrated confidence, continuously re-evaluated, and it keeps the frontier model one hop away as the fallback. If Seldon compiled workflows down to frozen, hand-authored rules and walked away, it would be fighting the Bitter Lesson and it would lose. Because it compiles down to re-evaluated, confidence-gated artifacts with the general model always in reserve, it rides the lesson instead of arguing with it. The clue was in the name Here is the detail nearly everyone waved past, and it may be the most telling thing about the entire launch. The model is called Jev — short for William Stanley Jevons, the nineteenth-century economist. Jevons noticed something that still ambushes people: when steam engines got more efficient and coal got cheaper to burn, Britain did not burn less coal. It burned wildly more, because cheapness unlocked a hundred uses that had never been worth it before. Falling unit cost did not shrink demand. It detonated it. That is the Jevons paradox, and naming a decision model after it is a thesis statement wearing a pun. Point it at intelligence and the unbundling stops reading as a cost-cutting story and starts reading as an expansion one. The reason a frontier model felt like the natural tool for every backend decision is that intelligence has been, in Ronacher's phrase, cheap and subsidized — so cheap you never had to be creative about it. The subsidy hid the waste and postponed the reckoning. But drive the cost of a calibrated decision toward zero and you do not just save money on the decisions you already make. You start dropping intelligence into ten thousand places it was never economical to touch before: inside tight loops, on every row of a table, in the guardrail watching the guardrail, at 2.5 hertz on a drone. Almeida's own picture of where this goes is the giveaway — smart software all over the place, he told TechCrunch, emergent and distributed, more like the early internet than the handful of mega-apps everyone is racing to build today. Read that again, because it is a frontier-lab founder sketching the exact opposite of the monolith. Not one god-model in one data center rationing intelligence through an API. A teeming, cheap, distributed mesh of small specialized parts — most of them boring — stitched together by ordinary software. Almeida is unusually candid that this is a different religion from the reigning one: the main product of a frontier lab, he says, is fear or hype, and he would rather sell intelligence than the promise of building God in a data center. The unbundling and the Jevons paradox are the same forecast told twice. Intelligence fragments into cheap parts; and because the parts are cheap, they metastasize into everything. Which is, precisely, the world you would build a router and a compiler for — if you had seen it coming. The shape of what's next Strip away the launch noise and here is the durable claim, the one I will happily be quoted on. AI-native software is not going to look like LLMs everywhere. That was never the destination; it was the prototype phase, the messy exploratory era where we used one gloriously general engine as glue between every part of the system because it was the fastest way to find out what we were even building. The mature version looks like what mature software has always looked like: conventional, legible, testable code — with small amounts of specialized intelligence inserted exactly where genuine uncertainty remains, and a general model held in reserve for the ambiguity that never fully resolves. Getting there is an unbundling, and unbundlings are won by whoever owns the layer the value migrates to. In databases it was the query planner. In hardware it was the compiler. In AI it will be the system that watches your traffic, learns which computations have stopped being questions, and compiles them down to the cheapest primitive that still clears the bar — reserving the expensive, general, magnificent model for the shrinking frontier of things that are actually hard. That is the layer we have been building. Jev did not create the opening. It walked through it, in front of everyone, and proved the door was real. Route your traffic through Seldon and watch recurring work turn into cheaper pipelines. Sign up for open beta → https://seldon-ai.com/sign-up