{"slug": "simurg-a-guard-that-catches-llm-hallucinations-mid-stream-and-heals-them", "title": "SIMURG: A Guard That Catches LLM Hallucinations Mid-Stream and Heals Them", "summary": "A developer released SIMURG v1.0.4, \"Catch The Heals,\" a streaming integrity monitor that detects LLM decoding failures such as repetition loops and structural garbage within roughly 600 characters of corruption onset and now repairs the answer via a targeted continuation ladder rather than discarding it. The release adds Pulse, a 345K-parameter, 2-layer streaming transformer trained on trigram-hash tokens that runs in about 4 ms on Apple Silicon and reports a held-out AUROC of 0.925 at training time and 0.826 on the release-time evaluation split. In live A/B testing against a VLLM-served endpoint under high repetition pressure, the heal ladder produced 5/5 clean outcomes versus 3/5 with healing disabled; weights are published on Hugging Face under Apache-2.0.", "body_md": "Your LLM looks fine in the demo. Then it ships.\n\nSomewhere between \"it works on my machine\" and \"a user saw this\", three failure modes show up in production streams:\n\n`#REF!` noise, token-level static.\nThese are not model-intelligence failures. They are decoding failures, and they are detectable *while the stream is still open*. That is the entire premise of [SIMURG](https://github.com/doofzoff/SIMURG): a streaming integrity monitor that watches every token as it arrives and raises a calibrated alarm within roughly 600 characters of the corruption onset — before the user has read the garbage.\n\nThis week we shipped **v1.0.4, \"Catch The Heals\"**, and it adds two things that change the game: the guard now *repairs* the answer instead of discarding it, and it has a small trained deep-learning detector on board.\n\nPost-hoc linters and LLM-as-judge pipelines all share one fatal property: they only see the full answer. By then the user has the garbage. Perplexity thresholds need logprob access most serving stacks do not expose. SIMURG's five-detector numpy ensemble works differently — it consumes the stream incrementally, scores every checkpoint, and can abort mid-generation.\n\nBut aborting alone is expensive: a blind full retry means paying the model's wall-clock twice.\n\nThe new repair ladder turns a corrupt abort into a targeted continuation:\n\n`verify_final` re-checks the whole thing. Zero-leak guarantee preserved.\nThe result: one generation's wall-clock instead of two, with `result.healed = True` on the response object.\n\nIn live A/B testing against a real endpoint (`wahoo-1.5-preview` via VLLM) under high repetition pressure, the heal ladder scored **5/5 clean outcomes vs 3/5** with healing disabled. At extreme penalty settings both drop — that is an honest limit of any guard, and it is documented as such.\n\nStatistics are robust, but they compress away *sequential* structure: the exact phase of a repetition loop, the cadence of script switches, the texture of structural garbage. So v1.0.4 ships a sixth, learned view.\n\nPulse is a 2-layer streaming transformer — 345K parameters, a 1.3 MB safetensors file. It reads the last ~600 characters of the stream as trigram-hash tokens (blake2b into 4096 buckets), runs a forward pass in about **4 ms on Apple Silicon**, and emits a calibrated corruption probability via two anchors stored in the checkpoint metadata.\n\nTraining protocol:\n\nHeld-out AUROC at training time: **0.925**. On the release-time evaluation split (292 onset-aware windows) the ROC lands at 0.826 — small-clean-set variance, reported with the figure so nobody has to guess.\n\nPer-class response on single windows:\n\n| input | pulse probability | \n|---|---|\n| clean prose | 0.002 | \n| repetition loop | 1.000 | \n| cross-lingual drift | 1.000 | \n| table echo | 1.000 | \n| structural garbage | 1.000 | \n\nThe contract is graceful by design: without `torch` installed, the numpy-only core behaves exactly as before. `pip install \"simurg[deep]\"` and the detector joins the ensemble automatically.\n\nThis is the part we are most excited about. If your workload has a characteristic failure mode — fabricated citations, number drift, prompt echo, domain-specific garbage — the detector can be retrained against your endpoint in one command:\n\n```\nexport SIMURG_LIVE_URL=http://your-host:port/v1/chat/completions\nexport SIMURG_LIVE_MODEL=your-model-name\n\npython -m simurg.deep.train_pulse --clean 40 --corrupt 240 --epochs 8 \\\n  --out ./simurg_pulse.safetensors\n```\n\nThe trainer prints held-out AUROC and calibration anchors before saving. Point `SIMURG_PULSE_WEIGHTS` at the new checkpoint and the ensemble picks it up on the next process start. Weights and the full model card are on Hugging Face: [MergenAI/SIMURG](https://huggingface.co/MergenAI/SIMURG).\n\nLicense: Apache-2.0.", "url": "https://wpnews.pro/news/simurg-a-guard-that-catches-llm-hallucinations-mid-stream-and-heals-them", "canonical_source": "https://dev.to/montydrief/simurg-a-guard-that-catches-llm-hallucinations-mid-stream-and-heals-them-252j", "published_at": "2026-09-22 12:01:37+00:00", "updated_at": "2026-09-22 12:23:19.578924+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-tools", "machine-learning", "ai-infrastructure"], "entities": ["SIMURG", "Pulse", "MergenAI", "Hugging Face", "VLLM", "Apple Silicon", "Apache-2.0", "wahoo-1.5-preview"], "alternates": {"html": "https://wpnews.pro/news/simurg-a-guard-that-catches-llm-hallucinations-mid-stream-and-heals-them", "markdown": "https://wpnews.pro/news/simurg-a-guard-that-catches-llm-hallucinations-mid-stream-and-heals-them.md", "text": "https://wpnews.pro/news/simurg-a-guard-that-catches-llm-hallucinations-mid-stream-and-heals-them.txt", "jsonld": "https://wpnews.pro/news/simurg-a-guard-that-catches-llm-hallucinations-mid-stream-and-heals-them.jsonld"}}