# SIMURG: A Guard That Catches LLM Hallucinations Mid-Stream and Heals Them > Source: > Published: 2026-09-22 12:01:37+00:00 Your LLM looks fine in the demo. Then it ships. Somewhere between "it works on my machine" and "a user saw this", three failure modes show up in production streams: `#REF!` noise, token-level static. These are not model-intelligence failures. They are decoding failures, and they are detectable *while the stream is still open*. That is the entire premise of [SIMURG](https://github.com/doofzoff/SIMURG): a streaming integrity monitor that watches every token as it arrives and raises a calibrated alarm within roughly 600 characters of the corruption onset — before the user has read the garbage. This week we shipped **v1.0.4, "Catch The Heals"**, and it adds two things that change the game: the guard now *repairs* the answer instead of discarding it, and it has a small trained deep-learning detector on board. Post-hoc linters and LLM-as-judge pipelines all share one fatal property: they only see the full answer. By then the user has the garbage. Perplexity thresholds need logprob access most serving stacks do not expose. SIMURG's five-detector numpy ensemble works differently — it consumes the stream incrementally, scores every checkpoint, and can abort mid-generation. But aborting alone is expensive: a blind full retry means paying the model's wall-clock twice. The new repair ladder turns a corrupt abort into a targeted continuation: `verify_final` re-checks the whole thing. Zero-leak guarantee preserved. The result: one generation's wall-clock instead of two, with `result.healed = True` on the response object. In live A/B testing against a real endpoint (`wahoo-1.5-preview` via VLLM) under high repetition pressure, the heal ladder scored **5/5 clean outcomes vs 3/5** with healing disabled. At extreme penalty settings both drop — that is an honest limit of any guard, and it is documented as such. Statistics are robust, but they compress away *sequential* structure: the exact phase of a repetition loop, the cadence of script switches, the texture of structural garbage. So v1.0.4 ships a sixth, learned view. Pulse is a 2-layer streaming transformer — 345K parameters, a 1.3 MB safetensors file. It reads the last ~600 characters of the stream as trigram-hash tokens (blake2b into 4096 buckets), runs a forward pass in about **4 ms on Apple Silicon**, and emits a calibrated corruption probability via two anchors stored in the checkpoint metadata. Training protocol: Held-out AUROC at training time: **0.925**. On the release-time evaluation split (292 onset-aware windows) the ROC lands at 0.826 — small-clean-set variance, reported with the figure so nobody has to guess. Per-class response on single windows: | input | pulse probability | |---|---| | clean prose | 0.002 | | repetition loop | 1.000 | | cross-lingual drift | 1.000 | | table echo | 1.000 | | structural garbage | 1.000 | The contract is graceful by design: without `torch` installed, the numpy-only core behaves exactly as before. `pip install "simurg[deep]"` and the detector joins the ensemble automatically. This is the part we are most excited about. If your workload has a characteristic failure mode — fabricated citations, number drift, prompt echo, domain-specific garbage — the detector can be retrained against your endpoint in one command: ``` export SIMURG_LIVE_URL=http://your-host:port/v1/chat/completions export SIMURG_LIVE_MODEL=your-model-name python -m simurg.deep.train_pulse --clean 40 --corrupt 240 --epochs 8 \ --out ./simurg_pulse.safetensors ``` The trainer prints held-out AUROC and calibration anchors before saving. Point `SIMURG_PULSE_WEIGHTS` at the new checkpoint and the ensemble picks it up on the next process start. Weights and the full model card are on Hugging Face: [MergenAI/SIMURG](https://huggingface.co/MergenAI/SIMURG). License: Apache-2.0.