Last month, curl d security reports for a stretch because the queue had filled with AI-generated slop — plausible-looking bug reports that were wrong. It's the same complaint the Stack Overflow survey put a number on: the top frustration with AI output isn't that it's useless, it's that it's almost right. Confidently, expensively, almost right.
I ran into a smaller version of this problem a while back, building a pipeline that pulls financial news from a stream, decides which items actually matter, and forwards only those. The failure mode that mattered wasn't missing a story. It was forwarding a wrong one. A false positive here isn't a minor annoyance — it's a signal someone might act on.
So I built the whole thing around one goal: prefer silence over slop. Here's how, and — more importantly — the eval harness that keeps it honest, because a filter you can't measure is just a vibe.
Most classification tutorials optimize for balanced accuracy, or F1, treating a miss and a false alarm as roughly equal mistakes. For a lot of problems that's fine. For this one it's exactly wrong.
If the pipeline stays quiet on a real story, the cost is: someone reads it somewhere else, a few minutes later. Annoying, recoverable.
If the pipeline forwards a wrong story — a misread number, a rumor stated as fact, a stale headline — the cost is: someone trusts it *because the pipeline is supposed to have filtered it.* The whole value of a filter is that passing through means something. One confident wrong answer devalues every correct one before it.
So the design target isn't "high accuracy." It's a false-positive rate low enough that a pass-through is trustworthy, accepting that we'll stay silent on borderline cases. Recall is the thing I'm willing to trade.
The filter isn't one model call. It's a chain of cheap-to-expensive gates, and the cheap ones exist to keep the expensive one from ever being the single point of judgment.
The asymmetry is baked in at every gate: the default is silence, and passing requires clearing all of them.
Here's the thing about a filter tuned for low false positives — you cannot tell if it's working by looking at it. It's quiet by design. Quiet because it's good and quiet because it's broken look identical from the outside.
So the filter isn't the real artifact. The eval harness is.
It's a golden dataset — hand-labeled items, each marked should-pass or should-reject, including the nasty ones: the almost-right, the plausible-but-stale, the technically-true-but-misleading. Every change to a prompt, a rule, or a threshold runs against it and reports:
This is the discipline the AI-slop stories are missing. The problem in those threads isn't that people used a model. It's that they shipped its output without a regression suite that would have caught the slop before a human had to. An LLM step without an eval harness is an untested function you deploy to production and hope about.
Treating evals like tests changed how I make changes. A new prompt idea isn't "better" because it reads better. It's better if it moves the FP-rate down without moving recall down more than I'll accept — and I have a number for that, before it ships, every time.
None of this is exotic. It's just testing, applied to the one part of the system we keep exempting from testing because it's a model. The model isn't special. It's a function with a bad day ahead of it, and the harness is how you find out before your users do.