{"slug": "the-model-writes-the-judge-measures-anatomy-of-an-llm-judge", "title": "The Model Writes, the Judge Measures: Anatomy of an LLM Judge", "summary": "A developer building AI assistants for diamond retail describes a three-layer quality system that prevents, catches, and judges unsupported claims. The system uses a deterministic core for decisions, a banned-word lexicon, and an independent LLM judge that verifies every generated claim against the stone's lab certificate.", "body_md": "Here is the kind of sentence that should keep a team up at night:\n\n\"At this clarity grade, you won't see a thing with the naked eye.\"\n\nOur assistant produced it while explaining a diamond. The grade it cites is real — SI1, printed on\n\nthe stone's lab report. The promise attached to it is invented. A clarity grade is assigned under\n\n10× magnification; it does not resolve what a naked eye will see. Two SI1 stones can look entirely\n\ndifferent — one clean at arm's length, the other with a visible speck right in the middle.\n\nThe model took a true fact and manufactured its consequence, because that is what a helpful\n\nsentence sounds like.\n\nNotice what the sentence is not. It is not a banned word. It is not a hallucinated stone. It\n\ncontains no wrong number. It breaks no rule you could have written down in advance. It is simply\n\nfluent — and fluency is the problem. Fluent and true are different variables, and a language model\n\noptimizes the first one.\n\nWe build AI assistants for exactly this kind of product. The one in this essay explains diamonds:\n\na short, two-voice spoken conversation about one stone — a host asks the questions a shopper\n\nwould ask, a gemologist answers — generated from that stone's evidence. A diamond is the\n\ndefinition of a high-consideration purchase: thousands of dollars, bought rarely, chosen under\n\nuncertainty. And it has a property that makes it an unforgiving test bed for generative AI —\n\nnearly every claim about a diamond can be checked against its certificate from GIA or IGI, the\n\nindependent laboratories that grade the world's diamonds. Overselling is not just tacky here. It\n\nis checkable.\n\nIn an [earlier essay](https://xinfer.ai/home/md/the-llm-narrates-the-core-decides) we argued that the language model should conduct the conversation while a\n\ndeterministic core makes every decision. This essay is the same worldview applied to quality. The\n\nmodel writes. It does not get to grade.\n\nAt one generation you can catch a sentence like the one above by reading it. Across a catalogue of\n\nstones you cannot. So before we made the writer better, we built an instrument: an independent LLM\n\njudge that reads every generated claim against the evidence and returns a verdict we can count.\n\nThis essay is about how that judge is built, how we keep the judge itself honest, and what changed\n\nonce we had it — which is that the arguments stopped.\n\nThe assistant explains one diamond at a time, from that diamond's evidence: the lab certificate,\n\nthe price, the retailer's published analysis of the stone. One rule governs everything: **every\nclaim about this stone must trace to supplied evidence.** If the evidence cannot answer —\n\nEnforcing that rule is not one mechanism. It is three, layered, and they differ in kind.\n\n**The first gate prevents.** It runs in code, inside the pipeline. When the assistant generates a\n\nfollow-up question about a stone, the question must name the evidence path it would be answered\n\nfrom. A question that names no available path is rejected — by code, not by prompt hope. This is\n\nwhat stops the system from asking \"how does this price compare to the market?\" about a stone\n\nwhose data has no comparable set. The model never gets the chance to improvise an answer, because\n\nthe question never survives.\n\n**The second gate catches.** A deterministic validator runs over every finished script: a\n\nhard-fail lexicon of the words the assistant must never say — *amazing*, *must-have*,\n\n*investment*, the manufactured urgency, the \"trust me.\" Word-boundary regexes, a few dozen tests,\n\nno judgment involved. If a banned phrase appears, the script fails. Cheap, fast, and absolute.\n\n**The third gate judges.** What remains after prevention and regex is the dangerous class: the\n\nplausible-but-unsupported claim. \"You won't see a thing with the naked eye\" passes both gates. No\n\nrule names it in advance, because it is not a phrase — it is a relationship between a sentence and\n\na lab report. Checking it requires reading both. That is a job for a reader, and at scale the\n\nreader has to be a model.\n\nOne detail here taught us more than we expected: **the boundary between the gates is itself a\ndesign decision.** When we extended the banned-word list from the domain experts' style guide, the\n\n\"Use an LLM judge\" usually means: ask a model to rate the output from one to ten. A score like\n\nthat is an opinion with a number attached. You cannot appeal it, you cannot audit it, and you\n\ncannot tell whether a 7 means two small problems or one large one. It is not an instrument; it is\n\na mood.\n\nOur judge is built differently, and every difference exists because a naive version failed first.\n\n**It returns a triage, not a score.** For every sentence that asserts something beyond restating\n\nthe input data, the judge answers typed questions: Is this a claim about *this stone*, or about\n\ndiamonds in general? If it is about this stone — does the evidence establish it? If it is general\n\n— is it accurate gemology? \"Lab-grown and natural diamonds are chemically identical\" is category\n\nteaching: permitted, and wanted — the judge's instructions say teaching is never a violation.\n\n\"This stone will look white to the eye\" is an item claim, and it had better trace to something. A **real**\n\nviolation is only the intersection: about this stone, *and* unsupported.\n\n**A claim also has an owner.** Supported is not the only axis. GIA graded the stone; the retailer\n\nanalyzed it; the assistant speaks for the house. If it says \"we grade this VS1,\" every word is\n\nfactually true and the sentence is still a violation — it credits the lab's work to the house.\n\nThe judge flags misattribution as its own class, separate from unsupported claims, because\n\nprovenance errors are how a trustworthy voice quietly claims authority it does not have.\n\n**It must report the claims it cleared.** The judge is required to list the general statements and\n\nthe supported claims too, with the classification visible — and it is forbidden from letting them\n\ninflate the violation count. This sounds like bureaucracy. It is the whole point. A verdict that\n\nonly lists problems is a number you have to trust. A verdict that shows every ruling — each quote\n\nverbatim, each one naming the evidence path it was checked against — is an audit trail you can\n\nre-check. When the judge is wrong, you can see exactly where.\n\n**Orthogonal signals stay orthogonal.** The judge also measures tone: does the assistant sound\n\nlike a trusted expert across the counter or a closer working a commission? Does the conversation\n\nsound like two people responding to a stone, or a script? But tone is a separate, subordinate\n\nchannel, and the judge's instructions are blunt: warmth never excuses an unsupported claim. A\n\nformatting check rides in a third channel and changes nothing in the honesty count. There is no\n\nblended score, deliberately. A blended score is where warmth hides dishonesty.\n\n**The writer and the judge read the same constitution.** Every rule about what a lab report does\n\nand does not support lives in one content-standard document. That document is spliced into the\n\nwriter's prompt and into the judge's prompt. The spec is a single artifact: the writer implements\n\nit, the judge enforces it, and editing one sentence changes both sides of the contract at once.\n\nThere is no version of the rules that only the judge believes.\n\n**It never grades its own output.** The writers run on Claude Sonnet 5. The judge's seat has been\n\nheld by two models — first GPT-5.5, a different lab entirely, and today Claude Opus 4.8, a\n\ndifferent model from the writers — and we verified the discipline held across the change. A model\n\nasked to grade its own writing has a conflict of interest baked into its weights. Independence\n\nhas to be constructed.\n\n**It never touches production.** The judge is a development instrument, not a runtime guardrail.\n\nIt grades output during the dev cycle to find where the writer's prompts leak; it adds zero\n\nlatency and zero cost to serving. Grading one diamond costs about thirty cents — four passes,\n\neach re-reading the full evidence — which is trivially cheap in development and pointlessly\n\nexpensive in production, where the deterministic gates already stand. A judge is not a content\n\nfilter. It is how you find out what to fix.\n\nThe uncomfortable part: the judge is also a language model. It is an unreliable narrator hired to\n\naudit an unreliable narrator. This is less absurd than it sounds — auditors are people too — but\n\nonly if the workflow plans for it. Ours does, in three ways.\n\n**The judge can be wrong, and the loop says so.** Our written dev cycle is: the judge finds a\n\nviolation → we fix the writer's prompt, *or we fix the judge if the judge is wrong* → we log it →\n\nwe re-run. The same humility applies to the deterministic gates. We ban the word *flaw* — the\n\nmarks inside a stone are characteristics, never flaws. Then the assistant, explaining a modest luster\n\nrating, said it was *\"not a flaw; just part of why the rating sits at four instead of five or\nsix\"* — and the regex failed the most honest sentence in the script. So the ban was narrowed:\n\n**The verdict is schema-checked, and malformed verdicts retry.** In real runs the judge would\n\nintermittently drop a required field, or return a field with the wrong type. We do not average\n\naround that; the harness detects the malformed verdict and retries it, and every consumer of the\n\nverdict is guarded against a bad shape. An instrument with unhandled failure modes doesn't stop\n\nmeasuring — it starts lying quietly.\n\n**One sample is noise.** The same stone, judged twice, does not score the same. We watched one\n\ndiamond score fifteen violations on one run and six to eight on reruns. If we had compared two\n\nmodels using single samples, we could have concluded anything at all. The discipline is boring:\n\naverage at least two runs before believing any comparison, and never let a single verdict decide\n\nanything important.\n\nAn instrument you never calibrate is just another opinion.\n\nHere is what actually changed when the judge existed. Three decisions that AI teams normally\n\nsettle by taste, seniority, or vendor reputation got settled by reading the same number before and\n\nafter.\n\n**Which model.** We ran two writer tiers over the same stones, with the same prompts and the same\n\njudge. Claude Haiku 4.5 — the small, fast tier — averaged about ten real violations per diamond.\n\nClaude Sonnet 5 averaged 0.8. A twelve-fold honesty gap, on identical inputs. And the direction of\n\nthe tradeoff surprised us: Sonnet also scored *higher* on specificity, so the honesty did not cost\n\nrichness. Every arithmetic blunder in the corpus came from the small model — including the run\n\nwhere it declared one stone \"59 carats more\" than another. The real gap between those stones was\n\n0.13 carats — and fifty-nine *dollars*. The model had turned a price difference into a weight.\n\nFifty-nine carats is not an engagement ring; it is a museum piece. A model that cannot keep\n\ndollars and carats apart is not a bargain at any latency.\n\nThe judge also caught something no benchmark would have: the fast tier existed only to shave\n\nseconds off time-to-first-word, yet it had quietly generated every record in our persisted\n\n\"quality\" collection. The latency shortcut had poisoned the quality artifact. No unit test flagged\n\nit, because every unit test passed. The judge flagged it, because the judge reads output.\n\nThe ending is the good part. The fast model's one advantage evaporated when we fixed a parser to\n\naccept the larger model's output format — and measured end to end, the strong model at low\n\nreasoning effort was fast enough to serve. Reputation said \"the small model is fine for this.\" The\n\ninstrument said otherwise, and the instrument had receipts.\n\n**Whether RAG.** Everyone's default architecture for grounding is retrieval. We shipped something\n\nsmaller: a deterministic key-join. The gemological knowledge — vetted definitions, approved\n\nanalogies, the boundaries of what a report can and cannot tell you — is keyed by characteristic,\n\nand each stone's generation receives only the blocks matching what *its* report actually lists: a\n\ncloud, a feather, a needle — the trade's names for the tiny marks inside a stone. An exact\n\nlookup, not a search. So when the assistant explains a\n\ncloud, it uses the vetted definition — a cluster of microscopic pinpoints — and keeps the vetted\n\nboundary: the report cannot tell you whether it causes visible haze in *this* stone. The\n\nknowledge base is content-hashed into the cache key, so editing one entry regenerates exactly the\n\noutput that depended on it.\n\nDid we lose anything by skipping the vector database? That is normally a religious question. For\n\nus it was an empirical one: the judge held the grounded output at zero real violations per stone.\n\nRetrieval remained unnecessary for a bounded, one-stone-at-a-time task — not as an article of\n\nfaith, but as a measured result we can revisit the day the task stops being bounded.\n\nThere was a quiet confirmation along the way. A gemological standard written independently by the\n\ndomain's own experts — people who had never read our prompts — was later diffed against our\n\nsystem. It landed on the same rules: the same evidence discipline, the same refusal to infer\n\neye-cleanness from a clarity grade, the same banned reassurances. Two instruments, built apart,\n\ngiving the same reading. That is what being right feels like from the inside: unremarkable.\n\n**Did the edit help.** The prompts are the product, so we treat them like one. Every prompt edit\n\ncarries a changelog entry with four fields: the **case** (the concrete finding that forced it —\n\nalmost always a judge finding), the **reason** (the principle it serves), the **change** (what\n\nactually moved), and the **verification** (a judged real run, with numbers). A typical case: the\n\njudge caught the assistant volunteering *why* a stone had earned its value rating — a cause the\n\nevidence never states — so the rule against manufactured explanations was widened, and the\n\nverification showed the pattern gone across redraws with the honesty counts unchanged. About\n\nsixty entries accumulated in the first week. No silent edits, ever. And because the prompt version\n\nis folded into the cache key, no cached episode can ever have come from an unknown prompt: edit\n\nthe prompt, and everything it touched regenerates by design.\n\nThis is test-driven development for a non-deterministic system. The judge's verdict is the failing\n\ntest you drive to green. And the tests that matter run against the real API, on real output —\n\nbecause in this project, every finding above came from a real run, and the mocked tests passed the\n\nwhole time. Over four hundred automated tests guard what can be asserted deterministically; not\n\none of them counts as evidence that the writing got better. Only the instrument says that.\n\nDiamonds are the setting, not the point. This pattern transfers to any team shipping generative\n\noutput where being wrong is expensive. The short version:\n\nThe payoff is cultural as much as technical. Model selection stopped being a debate about vendor\n\nreputation. Architecture stopped being a debate about fashion. Prompt engineering stopped being a\n\nvibe. We stopped arguing about which model was smarter and started reading the same number twice.\n\nOne honest caveat, because our instrument would flag us if we skipped it: the judge measures\n\nwhether the assistant is honest about a diamond. It cannot measure whether a shopper wants to\n\nlisten. That is a different experiment, with a different instrument — and no model gets to grade\n\nit for us.\n\n*Originally published at xinfer.ai — where there is also a 2-minute audio spotlight and a full narration of this essay.*\n\n*I'm Sam Wen, founder of XInfer.AI. We build AI agents that transact — chat, voice, phone, and SMS — with one deterministic core underneath all of it.*", "url": "https://wpnews.pro/news/the-model-writes-the-judge-measures-anatomy-of-an-llm-judge", "canonical_source": "https://dev.to/sam_wen_xinfer_ai/the-model-writes-the-judge-measures-anatomy-of-an-llm-judge-2k95", "published_at": "2026-07-27 01:18:04+00:00", "updated_at": "2026-07-27 01:59:53.964755+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-safety", "ai-products"], "entities": ["GIA", "IGI"], "alternates": {"html": "https://wpnews.pro/news/the-model-writes-the-judge-measures-anatomy-of-an-llm-judge", "markdown": "https://wpnews.pro/news/the-model-writes-the-judge-measures-anatomy-of-an-llm-judge.md", "text": "https://wpnews.pro/news/the-model-writes-the-judge-measures-anatomy-of-an-llm-judge.txt", "jsonld": "https://wpnews.pro/news/the-model-writes-the-judge-measures-anatomy-of-an-llm-judge.jsonld"}}