Robust and Efficient Guardrails with Latent Reasoning

Researchers have developed COLAGUARD, a new safety guardrail for large language models that transfers multi-step reasoning into a continuous latent space to reduce latency. In tests across ten moderation settings and eight safety benchmarks, COLAGUARD matched the accuracy of explicit reasoning models while delivering a 12.9x speedup and 22.4x reduction in token usage. The approach demonstrates that latent reasoning can improve both safety robustness and inference efficiency simultaneously, addressing a key barrier to deploying reasoning-based guardrails in high-throughput applications.

arXiv:2605.29068v1 Announce Type: new Abstract: Maintaining the safety of large language models LLMs is crucial as they are increasingly deployed in real-world applications. Existing safety guardrails typically rely on single-pass classification or, more recently, distilled reasoning. Reasoning-based guardrails significantly outperform classification-only baselines, but they incur substantial query latency and token overhead that make them impractical for highthroughput deployment. To address this challenge, we propose COLAGUARD, a guardrail model that transfers multi-step safety reasoning into a continuous latent space through a stage-wise training curriculum, enabling direct hidden-state propagation at inference. Evaluated on ten prompt- and response-moderation settings spanning eight safety benchmarks, COLAGUARD improves macro-F1 by 8.24 points over Llama Guard 3 and matches our explicit reasoning baseline, GuardReasoner, in macroF1 while delivering a 12.9X speedup and 22.4X reduction in token usage. Our results suggest that latent reasoning offers a practical alternative to explicit rationale generation for deployable guardrails, jointly improving safety robustness and inference efficiency rather than treating them as competing objectives.