{"slug": "desi-review-try-an-epistemic-llm-harness-in-five-minutes", "title": "DESi Review: Try an Epistemic LLM Harness in Five Minutes", "summary": "A developer has released DESi Review, an installable Claude skill that implements a structured epistemic governance framework for paper review. The skill separates deterministic claim extraction from model-based adjudication, producing a canonical, replay-stable claim ledger that makes the review process explicit and auditable. The tool is model-agnostic, English-only, and designed to test whether adding an explicit epistemic workflow changes LLM review outcomes.", "body_md": "I have packaged a first practical DESi application as an installable Claude skill.\n\nDESi Review runs a structured paper-review pipeline over any English text. The aim is not to replace expert review or claim that an LLM can determine truth. The narrower question is:\n\nWhat changes when the same model reviews the same text through an explicit, inspectable epistemic workflow?\n\n**What DESi is**\n\nDESi is a model-independent epistemic governance framework.\n\nIt defines structures for handling:\n\nThe new DESi Review skill is one executable harness built on top of that framework.\n\nThe current reference implementation uses Claude as the semantic adjudication engine, but the deterministic DESi layer makes no model calls and the protocol itself is model-agnostic.\n\n**What the skill does**\n\nA normal LLM review is usually produced as one fluent response. It may be useful, but it is difficult to inspect whether every relevant claim was considered or whether consistent criteria were applied.\n\nDESi Review separates the process into two parts.\n\n**1. Deterministic extraction**\n\nCode extracts and classifies claims from the source text and produces a canonical claim ledger.\n\n**2. Model-based adjudication**\n\nThe LLM evaluates every extracted strong claim using a fixed protocol and one of four allowed verdicts:\n\nThe emitter refuses to create the final artifact when:\n\nThe resulting ledger is canonicalised, hashed, and replay-stable.\n\nThis does not make the judgment automatically correct. It makes the procedure more explicit and auditable.\n\n**Installation**\n\nThe skill and documentation are available here:\n\nThe package contains no binaries and is small enough to inspect before installation.\n\nCurrent requirements:\n\nDownload either:\n\nBoth contain the same package.\n\nIn Claude:\n\nRun DESi over this paper.\n\nThe normal workflow requires no separate API key and creates no additional API cost beyond the Claude conversation itself.\n\n**Suggested A/B test**\n\nI would be interested in results from a simple comparison.\n\nUse the same model and the same paper.\n\n**Run A: ordinary review**\n\nAsk:\n\nReview this paper critically. Identify unsupported claims, overstatements, contradictions, and important limitations.\n\n**Run B: DESi Review**\n\nStart a fresh conversation and ask:\n\nRun DESi over this paper.\n\nThen compare:\n\nI am not assuming that DESi must outperform ordinary prompting in every case. Negative results and examples where the ordinary review is better are equally useful.\n\n**Known limitations**\n\nThe package includes a measured defect list. The most important current limitations are:\n\n**English-only extraction**\n\nRunning the extractor on another language may return zero findings. That must not be interpreted as a clean result.\n\n**Definitions can be misclassified as claims**\n\nDefinitions, descriptions of the setup, and rhetorical sentences may be extracted as strong assertions. The adjudicating model can mark them as NOT_A_CLAIM, but the upstream extraction error remains.\n\n**Reproducibility checks are not genre-aware**\n\nNon-empirical texts may be flagged for missing code, data, or parameters even where these requirements are inappropriate.\n\n**Model adjudication remains model adjudication**\n\nThe deterministic layer controls structure, completeness, schema compliance, hashing, and replay. It does not make the model’s semantic judgment neutral or infallible.\n\nWhere the reviewed text concerns the model itself, its developer, or machine welfare, the protocol requires a declared-interest field. Declaring a conflict is not equivalent to correcting it.\n\n**Why I am posting this here**\n\nFrontier-model capabilities appear to be converging on many benchmarks. If that continues, the more useful comparison may increasingly become:\n\nmodel versus model\n\nless often, and:\n\nmodel-plus-architecture versus model-plus-architecture\n\nmore often.\n\nDESi Review is a small test of that proposition.\n\nSome failures attributed to model quality may instead result from the absence of explicit epistemic structure: no complete claim inventory, no fixed verdict space, no persistent ledger, no enforced coverage, and no inspectable provenance.\n\nThe current skill is only one narrow application of the wider DESi framework. It focuses on document and paper review because this provides a relatively clear environment for testing the distinction between ordinary model output and structured epistemic processing.\n\nCode, documentation, downloads, and known defects:\n\nFeedback is especially useful when it includes:\n\nThe goal is not to demonstrate that the system always wins. The goal is to determine precisely where explicit epistemic architecture changes the result.\n\nTransparency note: This post was formulated with the assistance of an LLM because writing in English is laborious for me. The project, technical claims, limitations, and responsibility for the content are mine.", "url": "https://wpnews.pro/news/desi-review-try-an-epistemic-llm-harness-in-five-minutes", "canonical_source": "https://discuss.huggingface.co/t/desi-review-try-an-epistemic-llm-harness-in-five-minutes/178225#post_1", "published_at": "2026-07-27 13:01:23+00:00", "updated_at": "2026-07-27 13:03:22.153709+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "ai-research", "ai-safety", "large-language-models"], "entities": ["DESi Review", "Claude", "DESi"], "alternates": {"html": "https://wpnews.pro/news/desi-review-try-an-epistemic-llm-harness-in-five-minutes", "markdown": "https://wpnews.pro/news/desi-review-try-an-epistemic-llm-harness-in-five-minutes.md", "text": "https://wpnews.pro/news/desi-review-try-an-epistemic-llm-harness-in-five-minutes.txt", "jsonld": "https://wpnews.pro/news/desi-review-try-an-epistemic-llm-harness-in-five-minutes.jsonld"}}