SIMURG: A Guard That Catches LLM Hallucinations Mid-Stream and Heals Them

A developer released SIMURG v1.0.4, "Catch The Heals," a streaming integrity monitor that detects LLM decoding failures such as repetition loops and structural garbage within roughly 600 characters of corruption onset and now repairs the answer via a targeted continuation ladder rather than discarding it. The release adds Pulse, a 345K-parameter, 2-layer streaming transformer trained on trigram-hash tokens that runs in about 4 ms on Apple Silicon and reports a held-out AUROC of 0.925 at training time and 0.826 on the release-time evaluation split. In live A/B testing against a VLLM-served endpoint under high repetition pressure, the heal ladder produced 5/5 clean outcomes versus 3/5 with healing disabled; weights are published on Hugging Face under Apache-2.0.

Your LLM looks fine in the demo. Then it ships. Somewhere between "it works on my machine" and "a user saw this", three failure modes show up in production streams: REF noise, token-level static. These are not model-intelligence failures. They are decoding failures, and they are detectable while the stream is still open . That is the entire premise of SIMURG https://github.com/doofzoff/SIMURG : a streaming integrity monitor that watches every token as it arrives and raises a calibrated alarm within roughly 600 characters of the corruption onset — before the user has read the garbage. This week we shipped v1.0.4, "Catch The Heals" , and it adds two things that change the game: the guard now repairs the answer instead of discarding it, and it has a small trained deep-learning detector on board. Post-hoc linters and LLM-as-judge pipelines all share one fatal property: they only see the full answer. By then the user has the garbage. Perplexity thresholds need logprob access most serving stacks do not expose. SIMURG's five-detector numpy ensemble works differently — it consumes the stream incrementally, scores every checkpoint, and can abort mid-generation. But aborting alone is expensive: a blind full retry means paying the model's wall-clock twice. The new repair ladder turns a corrupt abort into a targeted continuation: verify final re-checks the whole thing. Zero-leak guarantee preserved. The result: one generation's wall-clock instead of two, with result.healed = True on the response object. In live A/B testing against a real endpoint wahoo-1.5-preview via VLLM under high repetition pressure, the heal ladder scored 5/5 clean outcomes vs 3/5 with healing disabled. At extreme penalty settings both drop — that is an honest limit of any guard, and it is documented as such. Statistics are robust, but they compress away sequential structure: the exact phase of a repetition loop, the cadence of script switches, the texture of structural garbage. So v1.0.4 ships a sixth, learned view. Pulse is a 2-layer streaming transformer — 345K parameters, a 1.3 MB safetensors file. It reads the last ~600 characters of the stream as trigram-hash tokens blake2b into 4096 buckets , runs a forward pass in about 4 ms on Apple Silicon , and emits a calibrated corruption probability via two anchors stored in the checkpoint metadata. Training protocol: Held-out AUROC at training time: 0.925 . On the release-time evaluation split 292 onset-aware windows the ROC lands at 0.826 — small-clean-set variance, reported with the figure so nobody has to guess. Per-class response on single windows: | input | pulse probability | |---|---| | clean prose | 0.002 | | repetition loop | 1.000 | | cross-lingual drift | 1.000 | | table echo | 1.000 | | structural garbage | 1.000 | The contract is graceful by design: without torch installed, the numpy-only core behaves exactly as before. pip install "simurg deep " and the detector joins the ensemble automatically. This is the part we are most excited about. If your workload has a characteristic failure mode — fabricated citations, number drift, prompt echo, domain-specific garbage — the detector can be retrained against your endpoint in one command: export SIMURG LIVE URL=http://your-host:port/v1/chat/completions export SIMURG LIVE MODEL=your-model-name python -m simurg.deep.train pulse --clean 40 --corrupt 240 --epochs 8 \ --out ./simurg pulse.safetensors The trainer prints held-out AUROC and calibration anchors before saving. Point SIMURG PULSE WEIGHTS at the new checkpoint and the ensemble picks it up on the next process start. Weights and the full model card are on Hugging Face: MergenAI/SIMURG https://huggingface.co/MergenAI/SIMURG . License: Apache-2.0.