{"slug": "your-hallucination-checker-only-sees-the-final-paragraph", "title": "Your hallucination checker only sees the final paragraph", "summary": "Insight IT Systems released PrismShine, an open-source tool that verifies LLM answers against evidence, addressing a gap in hallucination detection for agentic workflows. In benchmarks on HaluEval, PrismShine achieved higher F1 scores and lower latency than Vectara's HHEM-2.1-Open, with no LLM calls required by default.", "body_md": "Your hallucination checker only sees the final paragraph.\n\nThat’s the bleed. A fluent wrong number often starts earlier: empty retrieval, a swallowed tool error, a stale cache after a fact update. Score the prose alone and you miss the cause — then the model fills the gap confidently.\n\nMost “anti-hallucination” tools optimize one job: grade the **answer text** (encoder / HHEM-class model / LLM-as-judge). Useful. Incomplete for agents.\n\nWhat they usually can’t do:\n\n`0.87`\n\nAfter the model speaks (and optionally before it does), run a **verdict**:\n\n`ShineVerdict`\n\n: `decision`\n\n+ named `resolution_gate`\n\n+ `evidence_hash`\n\nThat product is **PrismShine** (Apache-2.0, `pip install`\n\n).\n\nIt is **not** a prompt-injection firewall (that’s PrismGuard). It is **not** an agent runtime (that’s ChorusGraph). One job: verify answers against evidence.\n\nPublic comparative vs Vectara **HHEM-2.1-Open** on HaluEval\n\n(Azure ACI · ONNX Tier-3 · receipt `2026-07-20_run4_onnx`\n\n):\n\n| System | B1 QA F1 | B2 numbers F1 | B1 p50 | LLM |\n|---|---|---|---|---|\nprismshine-fast |\n0.831 |\n1.000 (0 FP) |\n~90 ms |\n0 |\n| hhem-2.1-open | 0.746 | 0.926 | ~216 ms | 0 |\n\nReceipt folder on GitHub: `benchmarks/progress/2026-07-20_run4_onnx`\n\n(link in first comment).\n\n```\nbash\npip install \"prismshine==0.2.2\"\nprismshine verify --demo\nprismshine capabilities\nCore path = Tiers 0–2, CPU, 0 LLM calls by default.\n\nfrom prismshine import EvidenceBundle, PreloadChunk, ShineGate\ngate = ShineGate.build(profile=\"default\")\nbundle = EvidenceBundle(\n    run_id=\"demo\",\n    question=\"What was revenue?\",\n    answer=\"Revenue was $1000 in Q1.\",\n    preload=[\n        PreloadChunk(\n            chunk_id=\"c1\",\n            text=\"Revenue was $1000 in Q1.\",\n            source=\"retrieval\",\n        )\n    ],\n)\nverdict = gate.verify(bundle)\nprint(verdict.decision, verdict.resolution_gate, verdict.evidence_hash)\n0.2.2 drop-in helpers: validate_grounding · get_gate · enforce_mode_from_env\nShadow without blocking: PRISMSHINE_ENFORCE=0\nDocs: INTEGRATION.md §0 (link in first comment).\n\nThere’s also a no-API-key browser demo (link in first comment) — pass → fabricated number block → empty-retrieval halt.\n\nHonest limits (BIP)\nPASS ≠ world-true — grounded in your preload only\nBuffered answers (not mid-stream token verification)\nBare pip install prismshine ≠ Tier-3 span SotA; use prismshine[spans] + ONNX when you need that path\nWired runtime moat is a separate Docker receipt — don’t mix it unlabeled with the HHEM table above\nSoft ask\nIf you try verify --demo and hit a snag, paste the traceback in the comments — I’ll help.\n\nWhere would you wire the gate first — after the LLM node, before generation (halt empty retrieval), or both?\n\n### First comment (after publish)\nLinks (kept out of the lesson above on purpose):\n\nLanding → https://www.insightits.com/products/prismshine.html Demo → https://insightitsgit.github.io/PrismShine/demo.html GitHub → https://github.com/insightitsGit/PrismShine PyPI 0.2.2 → https://pypi.org/project/prismshine/0.2.2/ Receipt → https://github.com/insightitsGit/PrismShine/tree/main/benchmarks/progress/2026-07-20_run4_onnx YouTube → https://www.youtube.com/watch?v=OZCelVhP844 Product Hunt → (paste live PH URL)\n\nSmoke: pip install \"prismshine==0.2.2\" && prismshine verify --demo\n```\n\n", "url": "https://wpnews.pro/news/your-hallucination-checker-only-sees-the-final-paragraph", "canonical_source": "https://dev.to/amin_parva_ab01ff398fd341/your-hallucination-checker-only-sees-the-final-paragraph-416m", "published_at": "2026-07-21 01:06:28+00:00", "updated_at": "2026-07-21 01:59:01.750472+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools", "ai-research"], "entities": ["Insight IT Systems", "PrismShine", "Vectara", "HHEM-2.1-Open", "HaluEval", "PrismGuard", "ChorusGraph", "ONNX"], "alternates": {"html": "https://wpnews.pro/news/your-hallucination-checker-only-sees-the-final-paragraph", "markdown": "https://wpnews.pro/news/your-hallucination-checker-only-sees-the-final-paragraph.md", "text": "https://wpnews.pro/news/your-hallucination-checker-only-sees-the-final-paragraph.txt", "jsonld": "https://wpnews.pro/news/your-hallucination-checker-only-sees-the-final-paragraph.jsonld"}}