DPD disabled part of its chatbot after a customer got it to write a poem criticising the company and to swear. The company said the issue came from a system update, and it switched off the AI element immediately.
The chatbot had run for years. The update broke it.
The lesson: every release is a new agent that needs grading.
→ Re-run your full eval suite on every prompt, model, or tool change
→ Include adversarial “make it insult us” cases as permanent regression tests
→ Block the release when scores drop
Do your evals run before every release or only before launch?