{"slug": "dspy-jev-typed-decisions-one-program-and-reanchor", "title": "DSPy ♥ Jev: Typed Decisions, One Program, and ReAnchor", "summary": "DSPy 3.4 adds first-class support for System One models, letting Jev run behind an ordinary DSPy signature and preserving its probability evidence for optimization. The integration installs via `pip install \"dspy[typesafe]==3.4.0` and maps plain `bool` outputs to yes-or-no decisions and `Literal[...]` outputs to choices, with each output requiring a non-empty `desc`; DSPy rejects incompatible free-form `str` outputs before making the request. The release also introduces the ReAnchor optimizer, which the author tests to determine what it actually calibrates.", "body_md": "# DSPy ♥ Jev: Typed Decisions, One Program, and ReAnchor\n\nDSPy gets first-class support for System One models, plus an optimizer for tuning how programs act on their decisions.\n\n1.   1.\n  [Jev: AI Decisions as a Typed Function Call](https://stacktoheap.com/blog/2026/09/18/jev-doesnt-write-review-comments) \n2.   2.\n  [Jev at the Branches: The State Machine Is the Agent](https://stacktoheap.com/blog/2026/09/21/the-state-machine-is-the-agent) \n3.   3.\n  [Jev: Attack of the Classifiers](https://stacktoheap.com/blog/2026/09/22/jev-attack-of-the-classifiers) \n4. 4. DSPy ♥ Jev: Typed Decisions, One Program, and ReAnchor\n\nIn the [previous article](https://stacktoheap.com/blog/2026/09/22/jev-attack-of-the-classifiers), I arranged an attack of the classifiers. Jev, TF-IDF, MiniLM, and an LLM program built with DSPy entered the arena. MiniLM won, Jev held its ground, and several API credits were remembered with appropriate solemnity.\n\nThe attackers are becoming friends.\n\nDSPy 3.4 gives System One models first-class support. Jev no longer has to stand across the scoreboard from DSPy; it can sit behind an ordinary DSPy signature. The framework that sent an LLM into my last contest can now program Jev directly, preserve its probability evidence, and optimize how the application turns those probabilities into decisions.\n\nDSPy already makes structured outputs much safer than prompt-and-parse code: a signature declares the output type, and its adapters ask a generative model for that shape, parse it, and validate the result. System One support changes the model contract underneath that abstraction. The smallest example does not require a new signature at all.\n\nThis gave me two new questions:\n\n1. Can the exact same DSPy program run against Jev and the new GLiNER2.5-Decide model?\n2. What does DSPy’s new `ReAnchor` optimizer actually calibrate?\n\nThe second question turned out to contain a naming trap. We will get there.\n\nInstall the integration with:\n\n```\npip install \"dspy[typesafe]==3.4.0\"\nexport TYPESAFE_API_KEY=...\n```\n\n# Start with an ordinary DSPy signature\n\nThis is a normal DSPy program. There is no Jev-specific type in it:\n\n``` python\nimport dspy\n\nclass ReviewPayment(dspy.Signature):\n    \"\"\"Decide whether a payment report needs human review.\"\"\"\n\n    report: str = dspy.InputField(desc=\"The customer's payment report.\")\n    needs_review: bool = dspy.OutputField(\n        desc=\"Does this payment report require human review?\"\n    )\n\nreview_payment = dspy.Predict(ReviewPayment)\n```\n\nRun it with a generative LM and DSPy’s adapter asks the model for a Boolean, parses the response, and validates it:\n\n```\nwith dspy.context(lm=dspy.LM(\"openai/your-model\")):\n    result = review_payment(report=\"I do not recognize this card payment.\")\n\nprint(result.needs_review)  # bool\n```\n\nNow give the same program to Jev:\n\n``` python\nfrom dspy.experimental import TypeSafe\n\nwith dspy.context(lm=TypeSafe(\"jev-1.13.0\")):\n    result = review_payment(report=\"I do not recognize this card payment.\")\n\nprint(result.needs_review)  # still a bool\n```\n\nNothing in `ReviewPayment` changed. DSPy recognizes the `bool` output as a\nyes-or-no decision, translates it into a System One question, and derives the\nBoolean result from the probability Jev returns.\n\nThis is the compatibility contract, stated without suspense: a `Predict`\nsignature works with a System One backend when every output is a supported closed\ndecision. Plain `bool` maps to yes-or-no; `Literal[...]` maps to a choice. Each\noutput needs a non-empty `desc` phrased as the question to answer. Inputs can be\nordinary strings, numbers, lists, or structured objects.\n\nJev is not a chat model, so a free-form `str` output does not quietly fall back to\ntext generation. DSPy rejects an incompatible output before making the request.\nThat narrower contract is the point.\n\n# When the program needs the evidence\n\nThe ordinary signature returned a plain `bool`. That is enough if the application\nonly needs the answer—even if I later use ReAnchor. If application code itself\nneeds to inspect the probability—for a custom review policy, an audit log, or an\nuncertainty display—DSPy 3.4 exposes three rich decision types through\n`dspy.experimental`:\n\n| Type | Use it when | Evidence returned | \n|---|---|---|\n| `Noul` | The answer is yes or no | value and probability of true | \n| `Choice[...]` | Exactly one option should win | value, probability per option, confidence | \n| `Score[...]` | The answer lies on an ordered rubric | continuous value, level probabilities, confidence | \n\nThese types work with generative LMs too. The difference is underneath the DSPy abstraction: an LM generates the requested evidence structure for the adapter to parse, while Jev returns decision probabilities natively.\n\n# The classifier gets a type, not a parsing problem\n\nHere is the core classifier. The criteria are attached to the type, not hidden inside a prompt template:\n\n``` python\nimport dspy\nfrom dspy.experimental import Choice, TypeSafe\n\nIntent = Choice[\n    (\"card_arrival\", \"The customer is asking about card arrival.\"),\n    (\"card_linking\", \"The customer is asking about linking a card.\"),\n    (\"cash_withdrawal_charge\", \"The customer was charged for a cash withdrawal.\"),\n    # ...the remaining BANKING77 intents\n]\n\nclass ClassifyIntent(dspy.Signature):\n    \"\"\"Which single BANKING77 intent best describes the customer's request?\"\"\"\n\n    message: str = dspy.InputField(desc=\"The complete customer message.\")\n    intent: Intent = dspy.OutputField(desc=\"Select the single best-matching intent.\")\n\nclassifier = dspy.Predict(ClassifyIntent)\ndspy.configure(lm=TypeSafe(\"jev-1.13.0\"))\n\nresult = classifier(message=\"Why did the ATM charge me a fee?\")\n\nprint(result.intent.value)\nprint(result.intent.probabilities)\nprint(result.intent.confidence)\n```\n\n`result.intent.value` is the label your router uses. `probabilities` preserves\nthe distribution across all 77 labels. `confidence` summarizes how concentrated\nthat Choice distribution is; it is not a second probability of correctness.\n\nI could express the answer space as a plain `Literal` and still run it on Jev.\nThe richer `Choice` earns its place here by attaching a description to each opaque\nBANKING77 label and preserving the full distribution. The application gets more\nevidence without becoming coupled to the backend that produced it.\n\nThe runnable version in my [`classifier-bench`](https://github.com/manojlds/classifier-bench) repository builds the `Choice` dynamically from `data/banking77/task.json`:\n\n```\nTYPESAFE_API_KEY=... \\\n  npm run demo:dspy-jev -- \"Why did the ATM charge me a fee?\"\n```\n\n# One DSPy program, two decision models\n\nThe application should not know whether Jev or GLiNER produced the distribution. Only the backend changes:\n\n```\nprogram = JevIntentClassifier(instructions, labels, criteria)\n\nwith dspy.context(lm=jev):\n    jev_result = program(message=message)\n\nwith dspy.context(lm=gliner):\n    gliner_result = program(message=message)\n```\n\nFor hosted Jev, the backend is native:\n\n```\njev = TypeSafe(\"jev-1.13.0\")\n```\n\nI served `fastino/GLiNER2.5-Decide` locally behind a System One-shaped endpoint. “Shaped” is doing some work in that sentence.\n\nDSPy sends structured state containing its signature instructions, field descriptions, and inputs. My local GLiNER server accepts only a string state and omits TypeSafe’s `usage` object. I therefore added a small transport adapter that extracts `state[\"inputs\"][\"message\"]` and normalizes the response. The signature, `Predict` module, criteria, metric, and evaluation code stay unchanged.\n\nThat is “the same DSPy code,” not “every server implements the exact same wire protocol.” The distinction matters.\n\n# BANKING77 asks for another rematch\n\nThe benchmark is BANKING77: 77 fine-grained banking support intents. Both models receive the same DSPy signature, label names, definitions, seed, metric, and data splits. They are evaluated on the same stratified sample of 770 official test examples—ten per class.\n\n| DSPy backend | Accuracy | Macro-F1 | \n|---|---|---|\n| Jev 1.13 | **80.52%** | **79.60%** | \n| GLiNER2.5-Decide | 70.26% | 69.53% | \n\nJev led by 10.26 percentage points in accuracy on this task. That is a convincing win on BANKING77, not a deed granting Jev ownership of classification. An earlier direct System One protocol run on a different deterministic 770-example sample found 81.17% for Jev and 68.70% for GLiNER, but those are not the headline numbers here: the table above comes from the new matched DSPy harness.\n\nI would not generalize the result to every decision problem. BANKING77 has many semantically adjacent labels, and GLiNER2.5-Decide’s model card reports strong results on a broader 17-domain decision suite. This harness also intentionally does not compare latency: Jev is hosted while GLiNER runs through a local server, so that would measure deployments as much as models.\n\nThe repository now includes one harness that runs either backend through the\nsame DSPy `Choice` program:\n\n```\n# Jev baseline\nTYPESAFE_API_KEY=... uv run classifier-dspy-decisions \\\n  --model jev-1.13.0 \\\n  --output reports/banking77/dspy-jev.json\n\n# Local GLiNER System One-shaped server\nuv run classifier-dspy-decisions \\\n  --model fastino/GLiNER2.5-Decide \\\n  --base-url http://127.0.0.1:18093 \\\n  --string-state \\\n  --output reports/banking77/dspy-gliner.json\n```\n\nKeep the model version, sample IDs, criteria hash, and serving configuration with the report. “Same code” does not make two changing model aliases reproducible.\n\n# ReAnchor is not the calibration you are thinking of\n\nA model can return useful probabilities while the default decision boundary is wrong for the application. ReAnchor moves that boundary without retraining the model.\n\nExamples:\n\n- a yes/no fraud gate may need a threshold above 0.5;\n- an ordinal severity score may need different cuts between levels; or\n- one class in a large `Choice` taxonomy may win too often or not often enough.\n\n`ReAnchor` fits those **local interpretation parameters** against your program’s\nmetric:\n\n- thresholds for `Noul` ;\n- cuts for `Score` ; and\n- per-option weights for `Choice` .\n\nIt does not rewrite the criteria, retrain the model, or ask the model to explain itself. It optimizes how the program reads the distribution.\n\n## What the search actually does\n\nTake the simplest case: a `Noul` whose default threshold is `0.5`. Suppose Jev\nreturns these probabilities on four training examples:\n\n```\n0.18, 0.41, 0.63, 0.90\n```\n\nAt `0.5`, the first two become `False` and the last two become `True`. Moving the\nthreshold from `0.50` to `0.55` changes nothing. Neither does `0.61`. Every\nthreshold in the gap between `0.41` and `0.63` produces exactly the same four\nanswers.\n\nSo ReAnchor does not shuffle through arbitrary decimal numbers. It sorts the\nprobabilities and tries a representative midpoint in each gap: `0.295`, `0.52`,\nand `0.765` in this example. Those are the places where crossing from one gap to\nanother can flip at least one answer. ReAnchor scores each resulting set of\ndecisions with the metric supplied by the program.\n\nThe three decision types need slightly different candidate searches:\n\n| Output | What stays fixed | What ReAnchor searches | \n|---|---|---|\n| `Noul` | probability of `True` | a threshold between observed probabilities | \n| `Score` | probability distribution and its mean level index | ordered cuts between observed mean values | \n| `Choice` | probability of every option | per-option multipliers at points where the winning option would change | \n\nFor this BANKING77 program, the last row matters. A `Choice` selects the label\nwith the largest `probability[label] * weight[label]`. All weights begin at `1.0`.\nReAnchor changes one label’s weight at a time and needs to try only values where\nthat multiplication would make a training example choose a different winner.\nIt does not rewrite or renormalize the original probabilities, and it does not\nrecalculate confidence after changing the winner.\n\nThere are still two ways a clever search can fool itself: finding a tiny training gain and finding a gain caused by one peculiar example. ReAnchor guards against both. A candidate must strictly improve the whole training score. It must also pass a fold check: DSPy divides the training examples into up to five parts, chooses a setting on the other parts, and measures it on the held-out part. The combined held-out score must beat the current setting. This is not a guarantee against overfitting, but it makes “I fixed one example I just saw” less persuasive.\n\nFinally, the expensive part happens once. ReAnchor runs the program to collect\nthe probability evidence, then evaluates candidate thresholds, cuts, and weights\nlocally. Identical requests reuse DSPy’s cache. For a single `Predict` such as\nthis classifier, the search makes no additional Jev or LLM calls after that\nevidence pass. A composed program is more subtle: changing an upstream decision\ncan change a downstream request, which may require a new call.\n\nThat also explains how ReAnchor differs from DSPy’s better-known optimizers. GEPA can improve a generative LM program by changing its instructions. ReAnchor leaves the words alone and fits the numbers after the model has spoken. A decision-typed signature can be run with an LM and optimized with either lever; the TypeSafe/Jev path supports the cheaper numerical one.\n\nReAnchor is not limited to the rich types. For a plain `bool` or `Literal[...]`\noutput, it enables probability-based execution internally, fits the threshold or\nweights, and still returns an ordinary Python value. Use `Noul`, `Score`, or\n`Choice` when the application—not merely the optimizer—needs to see the evidence.\n\nFor a multiclass intent router, the metric can be exact match:\n\n``` python\nfrom dspy.experimental import ReAnchor\n\ndef exact_match(example, prediction, trace=None):\n    return float(prediction.intent.value == example.intent)\n\noptimizer = ReAnchor(metric=exact_match, num_threads=8)\noptimized = optimizer.compile(\n    classifier,\n    trainset=train_examples,\n    valset=validation_examples,\n)\n\nprint(optimizer.report)\noptimized.save(\"banking77-reanchor.json\")\n```\n\nReAnchor fits on `trainset`. The optional `valset` is reported but is not used to\nfit the parameters or choose a candidate. The returned program is a copy; the\noriginal is unchanged.\n\nThis is where I initially tripped over the word “calibration.” The official\n[ReAnchor documentation](https://dspy.ai/current/api/experimental/ReAnchor/) says\nthat it “fits the numeric decision settings in a program to your metric” and\nsummarizes the operation as “calibrate a program’s decisions.” I will call that\n**decision-rule calibration against an application metric**. It is not probability\ncalibration in the ECE, Brier-score, temperature-scaling, or reliability-diagram\nsense. ReAnchor does not train the returned probabilities to match empirical\nfrequencies, and it does not make the raw probabilities truer. That distinction\nis easy to miss because both activities begin with probabilities and end with\nbetter behavior. They are still different jobs.\n\n# GLiNER gives ReAnchor something to move\n\nI fitted ReAnchor with two training and two development examples per class—154 examples in each split—and evaluated once on a separate stratified set of 770 official test examples.\n\n| GLiNER program | Accuracy | Macro-F1 | \n|---|---|---|\n| Original `Choice` rule | 70.26% | 69.53% | \n| After ReAnchor | **71.30%** | **70.40%** | \n\nThe held-out gain was 1.04 accuracy points and 0.87 macro-F1 points. On the fit split, exact match moved from 66.23% to 71.43%; on the untouched development split it moved from 59.09% to 60.39%.\n\nThe selected configuration mostly retained weight `1.0`, but adjusted four\nclass-selection weights in response to recurring fit-sample errors. The raw\nprovider probabilities remain evidence from the original model; ReAnchor changes\nwhich weighted option the program selects.\n\nThis is encouraging, not a parade. Two examples per class is deliberately small, and class weights can overfit. The development gain and ReAnchor’s fold checks are useful safeguards, but the final judgment comes from the untouched test set and, eventually, production traffic.\n\nRun the experiment with:\n\n```\nuv run classifier-dspy-decisions \\\n  --data-dir data/banking77 \\\n  --model fastino/GLiNER2.5-Decide \\\n  --base-url http://127.0.0.1:18093 \\\n  --string-state \\\n  --train-per-class 2 \\\n  --val-per-class 2 \\\n  --test-per-class 10 \\\n  --reanchor \\\n  --num-threads 8 \\\n  --output reports/banking77/dspy-gliner2.5-decide-reanchor.json\n```\n\n# ReAnchor leaves Jev’s decision rule unchanged\n\nReAnchor ran the same search on Jev, but none of its candidate `Choice` weights\nimproved the training metric and passed the fold check. It therefore restored the\noriginal all-`1.0` weights rather than returning a worse or more fragile policy:\n\n| Jev program | Accuracy | Macro-F1 | \n|---|---|---|\n| Original `Choice` rule | **80.52%** | **79.60%** | \n| After ReAnchor | **80.52%** | **79.60%** | \n\nJev scored 79.22% on the fit split before and after optimization and 76.62% on\nthe separate development split. ReAnchor rejected its candidate weights with\nthe reason `fitted behavior did not beat the original`. This does not mean Jev\nmade no errors, or that no better decision policy could exist. It means that with\nthis small training set, this metric, and the candidates ReAnchor tested, no\nchange earned its way into the returned program. It also says nothing about\nwhether Jev’s probabilities are statistically well calibrated; that would require\na separate held-out evaluation with measures such as Brier score, ECE, and\nreliability diagrams. This no-change result is useful. An optimizer that sometimes\nsays “leave it alone” is doing more science than one that always returns a victory\nbanner.\n\n```\nTYPESAFE_API_KEY=... uv run classifier-dspy-decisions \\\n  --data-dir data/banking77 \\\n  --model jev-1.13.0 \\\n  --train-per-class 2 \\\n  --val-per-class 2 \\\n  --test-per-class 10 \\\n  --reanchor \\\n  --num-threads 8 \\\n  --output reports/banking77/dspy-jev-1.13-reanchor.json\n```\n\nDo not fit on the test set, and do not select between runs using test accuracy.\n\n# What I would use this for\n\nThe most compelling part of DSPy 3.4 is not shorter classification code. It is the separation of responsibilities:\n\n- the signature defines the semantic contract;\n- the backend supplies typed probability evidence;\n- DSPy composes and evaluates the program;\n- ReAnchor tunes the local decision policy; and\n- application code owns escalation, routing, and side effects.\n\nThat is a good fit for ticket routing, moderation, extraction verification, agent handoff, and other places where software needs a bounded judgment rather than another paragraph of generated text.\n\nThe APIs are experimental, so I pinned DSPy 3.4 and the model versions. But the direction is promising: program the decision once, compare specialized models behind it, and optimize the behavior the application actually measures.\n\n# Conclusion\n\nThe most interesting result was not Jev beating GLiNER by ten points, or ReAnchor finding one extra point for GLiNER. It was that the application program stayed recognizable while all of those pieces moved underneath it.\n\nDSPy’s signature described the bounded judgment. Jev and GLiNER supplied distributions. ReAnchor changed the local selection policy. Python remained responsible for what happened next.\n\nThat is the same boundary I have been circling throughout this series:\n\nLet the model make the small uncertain judgment. Let code own the system.\n\nDSPy 3.4 gives that boundary a programming model. Jev fits it naturally. ReAnchor adds a useful optimizer, provided I remember that it optimizes decisions—not the truthfulness of the probabilities themselves.\n\n# Sources and reproducibility\n\n- [DSPy 3.4.0 release notes](https://github.com/stanfordnlp/dspy/releases/tag/3.4.0)\n- [DSPy decision types and System One models](https://dspy.ai/current/api/experimental/DecisionTypes/)\n- [DSPy ReAnchor documentation](https://dspy.ai/current/api/experimental/ReAnchor/)\n- [TypeSafe Choice documentation](https://docs.typesafe.ai/primitives/choice)\n- [TypeSafe confidence documentation](https://docs.typesafe.ai/confidence)\n- [GLiNER2.5-Decide model card](https://huggingface.co/fastino/GLiNER2.5-Decide)\n- [Classifier benchmark repository](https://github.com/manojlds/classifier-bench) — the DSPy/Jev and ReAnchor additions are currently in the accompanying worktree and will land with the next repository update", "url": "https://wpnews.pro/news/dspy-jev-typed-decisions-one-program-and-reanchor", "canonical_source": "https://stacktoheap.com/blog/2026/09/25/dspy-heart-jev/", "published_at": "2026-09-25 00:00:00+00:00", "updated_at": "2026-09-25 21:28:43.879309+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents", "large-language-models"], "entities": ["DSPy", "Jev", "System One", "ReAnchor", "GLiNER2.5-Decide", "MiniLM", "TF-IDF"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/dspy-jev-typed-decisions-one-program-and-reanchor", "markdown": "https://wpnews.pro/news/dspy-jev-typed-decisions-one-program-and-reanchor.md", "text": "https://wpnews.pro/news/dspy-jev-typed-decisions-one-program-and-reanchor.txt", "jsonld": "https://wpnews.pro/news/dspy-jev-typed-decisions-one-program-and-reanchor.jsonld"}}