{"slug": "llms-generate-jev-decides-software-should-know-the-difference", "title": "LLMs Generate. Jev Decides. Software Should Know the Difference", "summary": "Mokapot Labs integrated TypeSafe's Jev, a so-called System One Model designed to make fast, bounded, structured decisions rather than generate open-ended text, into its open-source Pipeline Framework and used it in a real invoice-processing application. The team found that tasks such as classification, routing, and selecting from a known catalogue — which had been handled by prompting large language models — were never generative problems in the first place. Jev exposes decision shapes including Choice, Noul, and Score, returning probabilistic decisions intended for software consumption rather than prose for human readers.", "body_md": "For the last few years, we have been solving an extraordinary range of software problems with essentially the same primitive:\n\n**send some context to a large language model and ask it to generate the answer.**\n\nThat has worked relatively well, but it has also encouraged us to turn problems that are not fundamentally generative into generation problems.\n\nClassification becomes generation.\n\nRouting becomes generation.\n\nSelecting one item from a known catalogue becomes generation.\n\nDetermining whether a condition holds becomes, you guessed it: generation.\n\nAnd then, because a generative model is free to generate almost anything, we spend an increasing amount of effort constraining it:\n\nReturn exactly one of these values.\n\nDo not invent identifiers.\n\nReturn exactly this JSON structure.\n\nDo not include an explanation.\n\nDo not wrap the result in another object.\n\nNever return anything outside the supplied catalogue.\n\nStructured output has made this considerably better. But there is still something slightly peculiar about the underlying architecture.\n\nWe are using a machine designed to generate an open-ended sequence of tokens, and then asking it very politely not to be open-ended.\n\nTypeSafe's Jev presents a fascinating alternative.\n\nAnd when we integrated Jev into The Pipeline Framework and then used it in a real invoice-processing application, something became very clear:\n\n**some of the work we were giving to an LLM was never an LLM problem in the first place.**\n\nTypeSafe describes Jev as the first of its **System One Models**: models designed not primarily to generate language, but to make fast, structured decisions that software can consume directly.\n\nInstead of an open-ended generation operation, the abstraction is closer to a bounded decision function.\n\n``` php\nflowchart LR\n    S[State] --> G[Generative model]\n    G --> T[Open-ended tokens]\n```\n\nversus:\n\n``` php\nflowchart LR\n    S[State] --> D[Bounded questions]\n    D --> J[System One model]\n    J --> R[Probabilistic decisions]\n```\n\nThe distinction is profound.\n\nTypeSafe currently exposes three particularly useful decision shapes: `Choice`, `Noul`, and `Score`.\n\nA `Choice` selects from a finite set of alternatives and returns the corresponding probability information.\n\nA `Noul` evaluates a proposition probabilistically.\n\nA `Score` evaluates something against an ordered scale.\n\nSeveral independent questions can be evaluated against the same state in one request.\n\nThe output is not primarily prose intended for a human reader. It is information intended for **software**.\n\nThat changes what the model is being asked to do.\n\nConsider the difference.\n\nA generative approach says:\n\nRead this invoice and tell me which property it belongs to. Return exactly one ID from this list and do not invent another one.\n\nA bounded decision instead defines:\n\n```\nWhich property?\n\nChoice:\n    PROPERTY_A\n    PROPERTY_B\n    PROPERTY_C\n```\n\nIn the first case, selecting a valid identifier is a prompt requirement.\n\nIn the second, it is part of the decision domain.\n\nThat is a much stronger contract.\n\nAt Mokapot Labs we maintain a small application called Invoice Assistant.\n\nIt is built with The Pipeline Framework (TPF), an open-source framework for constructing typed processing applications.\n\nThe application receives an invoice and a property catalogue, extracts invoice information, determines which property the invoice belongs to, optionally performs visual analysis when the supplier cannot be established from text, presents the result for human confirmation, and then performs the required external effects.\n\nIts pipeline includes ordinary computation, AI inference, branching, a human Await boundary and replay-safe Commands.\n\nBefore Jev, its text-analysis stage looked roughly like this:\n\n``` php\nflowchart TD\n    A[Invoice] --> B[Extract document text]\n    B --> C[Gemma 12B]\n    C --> C1[Extract supplier/invoice nr/amount]\n    C --> C4[Classify supplier evidence]\n    C --> C5[Select property]\n    C --> C6[Explain recommendation/w note]\n    C1 --> D[Route supplier evidence]\n    C4 --> D\n    C5 --> D\n    C6 --> D\n    D -->|Sufficient| E[Review]\n    D -->|Insufficient| F[Vision model]\n```\n\nThis worked.\n\nIt was not disastrously slow. It was not unreliable enough to force a redesign.\n\nBut the LLM call was doing several fundamentally different jobs.\n\nAnd our prompt showed it.\n\nPart of the original prompt said:\n\n```\nClassify supplier evidence with exactly one qualitative\nsupplierEvidenceStatus:\n\nEXPLICIT_TEXT\nSTRONG_TEXTUAL_IDENTITY\nINSUFFICIENT\n```\n\nAnother part said:\n\n```\nSelect exactly one property ID present in the supplied\ncatalogue.\n\nDo not invent, rewrite or normalize property IDs.\n```\n\nRead those requirements again in the context of a System One model.\n\nThey describe **Choices**.\n\nWe had an LLM generating a result and a prompt instructing it to behave as though the output space were closed.\n\nBut the output space really *was* closed.\n\nFor supplier evidence, there were three possible answers.\n\nFor the property recommendation, the possible answers were precisely the properties already supplied to the application.\n\nThe problem was not:\n\nGenerate a property identifier.\n\nIt was:\n\n**Choose one of these properties.**\n\nThat difference sounds small.\n\nArchitecturally, it is enormous.\n\nThis is where the distinction becomes useful rather than ideological.\n\nThe application also needs to determine:\n\n```\nsupplier      = \"Some arbitrary company name\"\ninvoiceNumber = \"INV-2026-18473\"\ntotalAmount   = 68.52\n```\n\nThose are open-world values.\n\nThe supplier can be a string we have never encountered before.\n\nThe invoice number is arbitrary.\n\nThe amount is arbitrary.\n\nThese are extraction problems, and our existing generative LLM remains well suited to them.\n\nSo we did **not** replace the LLM with Jev.\n\nWe split the problem according to its semantics.\n\nThe result became:\n\n``` php\nflowchart TD\n    A[Invoice] --> B[Extract document text]\n    B --> C[\"Generative LLM<br/>Gemma\"]\n    C --> C1[Supplier]\n    C --> C2[Invoice number]\n    C --> C3[Total amount]\n\n    C1 --> D[Prepare DecisionRequest]\n    C2 --> D\n    C3 --> D\n\n    D --> J[\"Jev<br/>System One\"]\n    J --> J1[\"Supplier evidence<br/>Choice\"]\n    J --> J2[\"Property<br/>Choice\"]\n\n    J1 --> P[Probabilistic judgments]\n    J2 --> P\n\n    P --> R[Deterministic application policy]\n    R -->|Sufficient evidence| H[Review]\n    R -->|Insufficient evidence| V[\"Existing vision model<br/>VLM\"]\n```\n\nThe generative model generates.\n\nThe decision model decides.\n\nThe application governs what happens next.\n\nThat separation turned out to be more important than simply changing model providers.\n\nThe revised generative step now has a much narrower job:\n\n```\nExtract only the supplier, invoice number and total amount\nfrom the invoice evidence.\n```\n\nAnd, perhaps more revealingly:\n\n```\nDo not classify supplier evidence,\nselect a property,\nexplain a choice,\ngenerate a note...\n```\n\nThe LLM is no longer responsible for everything that happens to involve semantic understanding.\n\nIt is responsible for the part of the problem that genuinely requires open-ended extraction.\n\nThat is an important architectural lesson.\n\n**\"Uses AI\" is not a sufficient reason for two operations to belong to the same model call.**\n\nTheir semantic shapes matter.\n\nThere was another complication.\n\nTypeSafe currently publishes official Python and JavaScript/TypeScript SDKs, but not an official Java SDK.\n\nInvoice Assistant is a Java application.\n\nWe could have waited.\n\nWe could have introduced Python into the application.\n\nWe could have written a thin application-specific HTTP client.\n\nInstead, this became an opportunity to establish something more reusable in The Pipeline Framework:\n\n**a provider-neutral Java protocol for bounded AI decisions.**\n\nThe application constructs an ordinary `DecisionRequest`.\n\nFor supplier evidence, it declares:\n\n```\nnew DecisionQuestion(\n    \"supplierEvidence\",\n    DecisionQuestionType.CHOICE,\n    \"Judge only the strength of textual evidence identifying the invoice supplier.\",\n    List.of(\n        new DecisionCriterion(\n            \"EXPLICIT_TEXT\",\n            \"The issuer or supplier is directly labelled or named.\"),\n        new DecisionCriterion(\n            \"STRONG_TEXTUAL_IDENTITY\",\n            \"Several consistent textual identity cues identify the issuer.\"),\n        new DecisionCriterion(\n            \"INSUFFICIENT\",\n            \"Text is absent, generic, conflicting, or ambiguous.\")))\n```\n\nThe property decision is constructed dynamically from the application's actual property catalogue:\n\n``` php\nresult.properties().forEach(property ->\n    properties.add(\n        new DecisionCriterion(\n            property.id(),\n            property.displayName()\n                + \"; \"\n                + property.canonicalAddress()\n                + \"; aliases: \"\n                + String.join(\", \", property.aliases()))));\n```\n\nThen:\n\n```\nnew DecisionQuestion(\n    \"property\",\n    DecisionQuestionType.CHOICE,\n    \"Select the supplied property best supported by the invoice evidence.\",\n    properties)\n```\n\nThis is one of my favourite consequences of the change.\n\nPreviously we told the LLM:\n\n```\nDo not invent property IDs.\n```\n\nNow the set of possible property IDs **is the decision**.\n\nThe safety property moved from prose into structure.\n\nThe application constructs a state containing the information Jev actually needs:\n\n```\nextractedFacts\ninvoiceText\noriginalFilename\nproperties\n```\n\nand submits both questions together.\n\n```\nflowchart TD\n    S[\"Decision state<br/>extractedFacts<br/>invoiceText<br/>originalFilename<br/>properties\"]\n    S --> R[DecisionRequest]\n    R --> Q1[\"supplierEvidence<br/>Choice\"]\n    R --> Q2[\"property<br/>Choice\"]\n    Q1 --> J[Jev]\n    Q2 --> J\n    J --> A1[\"Supplier evidence judgment<br/>choice + probabilities\"]\n    J --> A2[\"Property judgment<br/>choice + probabilities\"]\n```\n\nThat maps naturally onto the System One model.\n\nThe application is not conducting an agent conversation with the model.\n\nIt is not asking one question, parsing the answer, constructing another prompt and asking another question.\n\nIt describes the state and the bounded judgments it requires.\n\nThe model evaluates them and returns structured results.\n\nThat is a much more application-shaped interaction.\n\nThis may be the most important architectural property of the whole integration.\n\nJev does **not** decide whether the pipeline should execute visual analysis.\n\nIt does not decide whether the application should ask a human.\n\nIt does not execute another capability.\n\nIt doesn't become the workflow engine.\n\nIt returns judgments.\n\nThe application then applies policy.\n\n``` php\nflowchart TD\n    J[Jev] --> P[\"Probabilistic judgments\"]\n    P --> A[\"Deterministic application policy\"]\n    A -->|Text evidence accepted| R[Review]\n    A -->|More evidence required| V[Vision analysis]\n```\n\nThis boundary matters.\n\nA probabilistic model is excellent at answering questions such as:\n\nWhich supplied property is most strongly supported by this evidence?\n\nIt should not automatically acquire authority over:\n\nWhat should the business process do next?\n\nThose are different concerns.\n\nTPF makes that distinction very natural.\n\nAt first glance, integrating a new AI model might sound like a framework feature:\n\n```\nkind: jev\n```\n\nWe deliberately did not do that.\n\nJev is not a new kind of pipeline operation.\n\nFrom TPF's perspective, Jev observes something the pipeline does not currently know.\n\nThat makes it a **Query**.\n\nTPF's semantic rule is simple:\n\n```\nknown execution-local data   → carry it\nfresh external observation   → Query\nexternal side effect         → Command\ndeferred external completion → Await\n```\n\nWhether the external observation came from PostgreSQL, an HTTP API, an LLM or a System One decision model does not fundamentally change that semantic boundary.\n\nSo the pipeline contains:\n\n```\n- name: Judge Invoice\n  kind: query\n  cardinality: ONE_TO_ONE\n  input: InvoiceJudgmentRequest\n  output: InvoiceJudgmentResult\n  using: invoice-judgment-model\n  operation: decide\n  operationVersion: 1\n```\n\nand the provider binding is:\n\n```\ninvoice-judgment-model:\n  provider: decision.query.jev\n  version: 1\n  config:\n    model: typesafe/jev-1.13\n    connection: openrouter-primary\n```\n\nThat is all the pipeline needs to know.\n\nThere is no Jev step kind.\n\nThere is no System One workflow engine.\n\nThere is no Jev-specific branch operator.\n\nThere is simply another typed external observation.\n\nThis distinction is crucial for avoiding framework lock-in.\n\nThe application does not model its domain using Jev's Python SDK classes.\n\nIts canonical pipeline types refer to:\n\n```\n<tpf.decision.DecisionRequest>\n<tpf.decision.DecisionResult>\n```\n\nThe application code constructs:\n\n```\nDecisionRequest\nDecisionQuestion\nDecisionCriterion\nDecisionQuestionType\n```\n\nThe provider happens to be `decision.query.jev` today.\n\nThe conceptual layering is therefore:\n\n``` php\nflowchart TD\n    A[Application domain]\n    A --> P[\"TPF bounded-decision protocol<br/>DecisionRequest / DecisionResult\"]\n    P --> Q[\"TPF Query<br/>decide / v1\"]\n    Q --> X[Provider adapter]\n    X --> J[Jev]\n    X -. future .-> O[Another bounded-decision engine]\n```\n\nThis is substantially different from writing an unofficial Java clone of TypeSafe's SDK.\n\nTPF is defining the capability the application needs.\n\nJev is implementing it.\n\nThat leaves the application architecture independent of one vendor's client library.\n\nThis has an interesting practical consequence.\n\nTypeSafe's official SDKs currently target Python and JavaScript/TypeScript.\n\nTPF applications can nevertheless use Jev from Java through ordinary typed application code.\n\nThere is no requirement for a Python sidecar.\n\n``` php\nflowchart TD\n    J[Java application] --> DR[DecisionRequest]\n    DR --> Q[TPF Query]\n    Q --> A[decision.query.jev]\n    A --> V[Jev]\n    V --> RS[DecisionResult]\n    RS --> J\n```\n\nNor does application code need to manually construct vendor HTTP payloads.\n\nAnd because the Java-facing contract is provider-neutral, this integration is useful beyond Jev itself.\n\nIt establishes bounded probabilistic decisions as an application capability in the Java ecosystem rather than merely exposing one vendor endpoint.\n\nThe old property recommendation contained:\n\n```\npropertyId\nexplanation\n```\n\nThe new result contains:\n\n```\npropertyId\nconfidence\nprobabilities[]\n```\n\nSupplier evidence similarly carries:\n\n```\nstatus\nconfidence\nprobabilities[]\n```\n\nThis represents a significant change in what the application expects from AI.\n\n``` php\nflowchart LR\n    subgraph Before\n        A1[AI] --> A2[Answer]\n        A1 --> A3[Prose explanation]\n    end\n\n    subgraph After\n        B1[AI] --> B2[Judgment]\n        B1 --> B3[Confidence]\n        B1 --> B4[Probability distribution]\n    end\n```\n\nWe had originally added explanations partly as a crutch while introducing the first real LLM Query into the application.\n\nBut prose explanation and model uncertainty are not the same thing.\n\nA convincing explanation does not necessarily mean a model is confident.\n\nA terse answer does not necessarily mean it is uncertain.\n\nFor application decisions, explicit probabilistic information is often much more useful.\n\nThe application can inspect it.\n\nPolicy can act on it.\n\nTelemetry can record it.\n\nHumans can see uncertainty where useful.\n\nAnd future policy changes do not require rewriting a prompt merely to change what confidence means operationally.\n\nThe refactoring also exposed functionality that no longer justified its existence.\n\nThe old model generated a short mnemonic note.\n\nIn practice, it was not used.\n\nSo it disappeared.\n\nThe recommendation explanation had largely existed to make early LLM behaviour inspectable.\n\nIt disappeared too.\n\nThis is another benefit of decomposing model responsibilities.\n\nLarge prompts have a tendency to accumulate requirements because adding another sentence feels cheap:\n\n```\nWhile you're there, also generate...\n```\n\nBut inference responsibilities then become coupled together.\n\nOnce the application explicitly separates extraction, decision, policy and presentation, each output has to justify why it exists.\n\nThat is healthy architecture, AI or otherwise.\n\nOne of the recurring problems in agentic systems is that the model gradually absorbs application architecture.\n\nThe model decides what to call.\n\nThe model decides whether to retry.\n\nThe model decides what state matters.\n\nThe model decides when the workflow is finished.\n\nEventually the \"application\" becomes a prompt wrapped around a tool registry.\n\nTPF deliberately takes another direction.\n\nThe pipeline owns composition.\n\nCanonical types own application contracts.\n\nQueries own fresh observations.\n\nCommands own external effects and, optionally, deferred completion.\n\nOrdinary application functions own deterministic business policy.\n\nAI fits inside those boundaries rather than replacing them.\n\nInvoice Assistant demonstrates that particularly well.\n\nThe model can judge:\n\n```\nsupplierEvidence =\n    STRONG_TEXTUAL_IDENTITY\n\nconfidence =\n    0.x\n```\n\nbut ordinary application code determines what that means for the workflow.\n\nLikewise, Jev may select a property, but it does not archive the invoice.\n\nArchiving is an external effect and therefore remains a TPF Command.\n\nHuman property confirmation does not become an agent loop waiting in memory; that, remains an Await boundary.\n\n``` php\nflowchart TD\n    P[Typed TPF pipeline]\n\n    P --> Q[Query]\n    P --> C[Command (and await)]\n\n    Q --> O[Fresh observation]\n    C --> E[External effect]\n\n    O --> L[Generative LLM]\n    O --> J[System One / Jev]\n    O --> X[Database / API / other provider]\n```\n\nAI becomes part of software architecture rather than a replacement for it", "url": "https://wpnews.pro/news/llms-generate-jev-decides-software-should-know-the-difference", "canonical_source": "https://dev.to/mbarcia/llms-generate-jev-decides-software-should-know-the-difference-2dek", "published_at": "2026-09-22 16:37:18+00:00", "updated_at": "2026-09-22 16:53:11.551502+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "large-language-models", "ai-products", "ai-infrastructure"], "entities": ["TypeSafe", "Jev", "Mokapot Labs", "The Pipeline Framework", "Invoice Assistant", "Gemma 12B"], "alternates": {"html": "https://wpnews.pro/news/llms-generate-jev-decides-software-should-know-the-difference", "markdown": "https://wpnews.pro/news/llms-generate-jev-decides-software-should-know-the-difference.md", "text": "https://wpnews.pro/news/llms-generate-jev-decides-software-should-know-the-difference.txt", "jsonld": "https://wpnews.pro/news/llms-generate-jev-decides-software-should-know-the-difference.jsonld"}}