Tuning guardrails without blocking real users

A fintech team that added PII redaction, topic restrictions and output moderation to a financial FAQ assistant found after launch that users were being refused on ordinary questions such as how interest is calculated, because the team had only tested whether harmful content got blocked. The team built a second test set of legitimate questions that should be allowed and measured both failure types — harmful content slipping through and valid requests being refused — then loosened overly broad topic rules, which cut false refusals sharply while high-risk checks like PII redaction stayed strict, and re-tested every subsequent rule change against both sets. Guardrails AI, NeMo Guardrails and Llama Guard on the Hugging Face Hub can serve as the enforcement layer, but the tuning only works if both directions are measured.

A fintech team added guardrails to a financial FAQ assistant: PII redaction, topic restrictions, and output moderation. After launch, support tickets showed that users were being refused on ordinary questions, such as how interest is calculated. The team had only tested one direction, whether harmful content got blocked. They built a second test set of legitimate questions that should be allowed, then measured both failure types: harmful content that slipped through, and valid requests that were refused. The original topic rules turned out to be far too broad. Loosening them cut false refusals sharply while the high-risk checks, like PII redaction, stayed strict. After that, every rule change was re-tested against both sets. Options such as Guardrails AI, NeMo Guardrails, and Llama Guard on the Hugging Face Hub can serve as the enforcement layer, but the tuning only works if both directions are measured.