# Testing what users see when an AI safety check fails

> Source: <https://dev.to/asamassekou/testing-what-users-see-when-an-ai-safety-check-fails-b81>
> Published: 2026-11-15 19:11:36+00:00

When we built Koda, an app that teaches kids how AI works, we had a question that a normal answer-quality eval couldn't settle. If the model has a draft ready and the safety check fails, what actually reaches the screen?

We built an offline test around that boundary. The application holds the draft while checking it alongside the original question. The gateway rechecks permission before it delivers an approved answer. If the detector is unavailable, the application can display a predefined message without showing the draft.

With Node.js 22 or later, run:

```
git clone https://github.com/SamSquareAI/child-ai-safety.git
cd child-ai-safety
npm ci --ignore-scripts
npm run --silent demo
```

The demo uses invented prompts and simulated providers. It runs without API keys. The terminal shows four paths: an approved answer, a refusal, a support response, and a simulated detector outage. In the outage case, the learner-facing message says Koda can't answer right now and suggests trying a learning question later.

Run the test through the same delivery path your UI uses. Make the safety adapter return unavailable after a draft exists. Assert that the application renders the fallback and that no part of the draft was emitted. In a streaming app, that means checking the entire event sequence, including chunks sent before the final message. Test an ordinary approved answer too, so you know both paths work.

I help maintain [Child AI Safety](https://github.com/SamSquareAI/child-ai-safety), the Apache-2.0 developer preview that came out of this work. If you'd like a small first contribution, [issue #8](https://github.com/SamSquareAI/child-ai-safety/issues/8) asks for a runnable example showing how the gateway counts approved and unavailable requests locally.

*Drafted with AI and checked against the repository's offline demo.*