I spent the last while building a prompt-injection detector trained on 78,000+ attack samples. Here's what surprised me, and why I ended up going the unfashionable route.
The default move in 2026 is "use an LLM to judge whether input is an attack." It's appealing because models understand nuance. But once you try to run it inline on every request, the problems pile up fast:
So I built the boring version instead: deterministic regex plus classical ML (TF-IDF character n-grams into logistic regression). No LLM in the detection path. It runs in about 7ms, costs nothing per call, and is fully deterministic.
Here's the part I want to be honest about, because most vendors quote one number and hide the rest.
Measured on public benchmarks the model was not trained on (held out, non-circular):
That last number is bad, and it's the most important one on the list. The honest read is that deterministic detection is excellent on real-world and obfuscated attacks and weak on subtle roleplay framing. That's a real gap, and pretending otherwise just means someone finds it later and trusts you less.
One thing I didn't appreciate going in: FPR is completely traffic-dependent. The same model reads roughly:
If a vendor gives you a single FPR number with no context, they're giving you their best-case slice. Always ask which traffic distribution it was measured on. The takeaway isn't "LLMs are useless for security." It's that the always-on, every-request layer should be cheap, fast, and deterministic, and you escalate to something heavier only for the ambiguous cases that earn it. Boring technology you can afford to run on 100% of traffic beats clever technology you can only afford to run sometimes.
I put the detector behind a demo where you can throw payloads at it and watch the verdicts, no signup (creds are prefilled):
Bring a prompt-injection payload you think is sneaky and see if it gets through. If it does, that's genuinely useful to me. And if you've measured your own detection differently, I'd love to compare notes in the comments.