cd /news/ai-safety/s2-the-loot-report-five-strangers-au… Β· home β€Ί topics β€Ί ai-safety β€Ί article
[ARTICLE Β· art-71938] src=dev.to β†— pub= topic=ai-safety verified=true sentiment=Β· neutral

#S2 The Loot Report: Five Strangers Audited My Systems in One Day

A physical therapist building hospital internal tools with AI describes how five strangers' comments on a feed revealed five security and reliability holes in his systems, all closed within hours. The incidents included a health check that conflated 'not checked' with 'clean', a deployment validation gap, and a scanner that could falsely flag good code, leading to fixes like adding explicit 'no degradation' fields and testing guardrails with valid input.

read7 min views1 publishedJul 24, 2026

Last time I said each post would show you the receipts. This one is a single day's worth.

In about a day, five strangers audited my systems β€” and not one of them knew they did it. Each was a comment or a post on this feed: a single sentence that turned out to be pointing straight at a hole in something I'd already shipped and already trusted. I read the sentence, found the hole, and closed it β€” usually within the hour. I'm a physical therapist who builds a hospital's internal tools with AI, so "closed it" means the AI and I closed it while I described what was actually breaking. Five sentences, five holes, five receipts.

They also turned out to be one idea wearing five coats. I'll get to that.

Someone wrote a line about scheduled jobs I couldn't shake: a receipt is not metadata about a result β€” it is part of the result. Labels go stale silently, they said, because whatever wrote the label already walked away before it stopped being true.

My systems' health check said ok

. That was the whole thing β€” a green word. So I made it hand over a receipt instead: which version checked, when, how many milliseconds the database actually took to answer, and one field I'm oddly proud of β€” an explicit "no degradation" that is present and set to null, not just absent. That distinction sounds like nothing and is everything: a missing field means "nobody checked," a field that's present-and-null means "checked, and it was clean." One is silence wearing a confident face. The other is proof someone looked. My health check used to conflate them. Now it can't.

Someone shipped a machine-learning model that passed its release gate and then answered "neutral" to every input it ever got. The gate had tested the model before the final export step β€” so it had validated a file no user would ever actually run. Their line: ask whether your gate validates the file you upload.

I had the same gap in a dumber place. When I deploy a mobile tool, I bump a cache version so people don't get served stale code β€” and I'd been trusting that the bump reached them. So I wrote a check that fetches the live file, the one users actually receive, and compares its version to the one I built. Ran it: local v514

, live v514

, match. The first time that check earns its keep is the day the two don't match and I hear it from a script instead of from a confused user a week later. The thing I validate and the thing I ship are finally the same object.

Someone debugged an AI agent that retried the same valid code three times. The tool had handed it a plausible-but-wrong error, so the agent faithfully "fixed" a problem that was never the problem β€” on a loop. Their line stopped me cold: the worst bugs don't throw a red stack trace; they hand you a polite, confident, wrong message and let you fail on repeat. And the fix underneath it: fuzz your guardrails with ordinary valid input β€” every false rejection is a bug.

I have a scanner that reads my own code for security problems. I'd tested whether it catches bad code. I had never tested whether it wrongly flags good code β€” which is the exact failure that would send one of my AI agents off "fixing" something that was already correct. So I planted six known-good snippets: an escaped output, a safe parameterized query, a hashed password β€” the clean patterns my rules could plausibly misfire on. Ran it: caught everything it should, false-flagged none of the six. One snippet β€” the letters "eval(" sitting inside a comment β€” does still trip it, and now that's a labeled known blind spot instead of a surprise. If I ever tighten a rule and it starts biting clean code, that test goes red before an annoyed teammate does.

Someone wrote about a cloud mistake that cost thirty thousand dollars β€” traffic quietly routed the expensive way, invisible in the code, the bill landing weeks after anyone remembered making the change. Their line: nobody warns juniors, because the cost shows up on a delay, detached from its cause.

I don't run cloud infrastructure, but the shape is universal: the dangerous problems aren't the ones that error, they're the ones that look fine now and come due later. Disk slowly filling. A quota quietly draining. So I added the humblest possible version to my health receipt β€” how much disk is left, as a plain number β€” and told the watcher to raise a flag at ten percent, long before anything actually breaks. It reads like trivia until the night it saves you, and then it reads like the most valuable line in the file. Writing down what the environment will charge you for before it charges you is the cheapest insurance there is.

This one broke something I'd built that same morning, and it didn't come from a drive-by β€” it came from someone I'd been trading notes with all week. He pointed at a trap hiding inside "just add an external check": an outside process that grades a self-reported receipt only looks like an auditor. It stands outside, and then reads a report the audited party wrote about itself. Type the words, run nothing, pass. Worse than no auditor at all, he said β€” because it produces a green light with a real script standing behind it.

I ran his test against Loot #1, the receipt I'd been so pleased with three hours earlier. It failed. My watcher read the health endpoint's own JSON and trusted the "database: true" the endpoint had written about itself. If a bug ever made that endpoint lie, my watcher would nod along and call it healthy. So I did the rest of his sentence: the watcher now runs its own database query, directly, instead of reading the endpoint's claim about one β€” and if the two ever disagree, that mismatch is itself the alarm. The auditor stopped grading a report and started checking the work. Receipt: it runs clean, and the direct query and the self-report now agree because both were verified β€” not because one vouched for the other.

Line them up and they're one idea wearing five coats. The status that says "fine." The gate that tests the wrong file. The scanner that never checks its own false alarms. The cost that hides in next month. The auditor that reads its own report. Every one is a thing grading its own homework β€” and every fix was the same move: don't trust the confidence; verify with something the checked thing didn't write. A receipt instead of a label. The shipped file, not the built one. A planted input. A number that comes due early. A query the watcher runs itself.

I didn't arrive at that principle. I stole it five times from five strangers in a single day before I noticed it was one sentence.

Here's what keeps this honest: none of it is one-directional. Loot #5 came from someone who'd found the same blind spot in his own tool that morning β€” he broke his hook with the exact test he then handed me, and I turned around and broke mine with it. Loot #3's author watched me take his "fuzz it with valid input" line straight into code the same afternoon, and said so. You take a line, you leave a line. That's the whole economy.

That's what the feed is, once you stop treating it as a broadcast channel: a shared workshop where everyone's tools are lying open on the bench, and "I'm stealing that" is just how we say thank you.

So β€” thank you. Five receipts, one day. The bag's full again.

This is part of Stolen from the Feed β€” what a non-developer carries home from dev.to and actually builds. Sister series to From Zero to Ship.

── more in #ai-safety 4 stories Β· sorted by recency
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/s2-the-loot-report-f…] indexed:0 read:7min 2026-07-24 Β· β€”