Your LLM looks fine in the demo. Then it ships.
Somewhere between "it works on my machine" and "a user saw this", three failure modes show up in production streams:
#REF! noise, token-level static.
These are not model-intelligence failures. They are decoding failures, and they are detectable while the stream is still open. That is the entire premise of SIMURG: a streaming integrity monitor that watches every token as it arrives and raises a calibrated alarm within roughly 600 characters of the corruption onset — before the user has read the garbage.
This week we shipped v1.0.4, "Catch The Heals", and it adds two things that change the game: the guard now repairs the answer instead of discarding it, and it has a small trained deep-learning detector on board.
Post-hoc linters and LLM-as-judge pipelines all share one fatal property: they only see the full answer. By then the user has the garbage. Perplexity thresholds need logprob access most serving stacks do not expose. SIMURG's five-detector numpy ensemble works differently — it consumes the stream incrementally, scores every checkpoint, and can abort mid-generation.
But aborting alone is expensive: a blind full retry means paying the model's wall-clock twice.
The new repair ladder turns a corrupt abort into a targeted continuation:
verify_final re-checks the whole thing. Zero-leak guarantee preserved.
The result: one generation's wall-clock instead of two, with result.healed = True on the response object.
In live A/B testing against a real endpoint (wahoo-1.5-preview via VLLM) under high repetition pressure, the heal ladder scored 5/5 clean outcomes vs 3/5 with healing disabled. At extreme penalty settings both drop — that is an honest limit of any guard, and it is documented as such.
Statistics are robust, but they compress away sequential structure: the exact phase of a repetition loop, the cadence of script switches, the texture of structural garbage. So v1.0.4 ships a sixth, learned view.
Pulse is a 2-layer streaming transformer — 345K parameters, a 1.3 MB safetensors file. It reads the last ~600 characters of the stream as trigram-hash tokens (blake2b into 4096 buckets), runs a forward pass in about 4 ms on Apple Silicon, and emits a calibrated corruption probability via two anchors stored in the checkpoint metadata.
Training protocol:
Held-out AUROC at training time: 0.925. On the release-time evaluation split (292 onset-aware windows) the ROC lands at 0.826 — small-clean-set variance, reported with the figure so nobody has to guess.
Per-class response on single windows:
| input | pulse probability |
|---|---|
| clean prose | 0.002 |
| repetition loop | 1.000 |
| cross-lingual drift | 1.000 |
| table echo | 1.000 |
| structural garbage | 1.000 |
The contract is graceful by design: without torch installed, the numpy-only core behaves exactly as before. pip install "simurg[deep]" and the detector joins the ensemble automatically.
This is the part we are most excited about. If your workload has a characteristic failure mode — fabricated citations, number drift, prompt echo, domain-specific garbage — the detector can be retrained against your endpoint in one command:
export SIMURG_LIVE_URL=http://your-host:port/v1/chat/completions
export SIMURG_LIVE_MODEL=your-model-name
python -m simurg.deep.train_pulse --clean 40 --corrupt 240 --epochs 8 \
--out ./simurg_pulse.safetensors
The trainer prints held-out AUROC and calibration anchors before saving. Point SIMURG_PULSE_WEIGHTS at the new checkpoint and the ensemble picks it up on the next process start. Weights and the full model card are on Hugging Face: MergenAI/SIMURG.
License: Apache-2.0.