The Attackers Have a Harness Too

A security researcher's scan of exposed AI agent harnesses found more than twenty likely malicious instances of an open-source pentesting harness, with one cluster running over 1,100 attack projects across fourteen dashboards at a roughly 10% success rate and under 23 hours from project start to goal met; named victims included John Deere, SharkNinja and Cummins. Separately, researchers built a backdoored 7B open model for under $50 using a 43-million-parameter adapter (0.6% of weights) that fired on a trigger phrase 100% of the time while keeping 100% clean accuracy, then exfiltrated environment files when run through a coding agent. The writeup argues agent control planes and inherited model weights are now production supply-chain risks.

For two years the interesting question about agents was what they could build. This week the more useful question is who else is running them, and what they are shipping to you. 1. Someone Left the Intrusion Dashboards Open, and the Industrial Scale Is the Story Agent harnesses have a web UI, and web UIs get left on the open internet. A scan of exposed AI pentesting panels found more than twenty likely malicious instances of one open-source harness https://norforge.com/blog/inside cairn , and the view inside is unusually complete: targets, discovered weaknesses, harvested credentials, and whether the break-in worked. One cluster alone was running over 1,100 attack projects across fourteen dashboards, at roughly 140 recorded facts per project, a 10% success rate, and just under 23 hours from project start to goal met. Named victims followed — John Deere, SharkNinja and Cummins among more than a hundred sites https://www.theverge.com/ai-artificial-intelligence/1013643/ai-agents-hack-john-deere-sharkninja — with the operator's instructions fitting in under a hundred characters. The architecture is the part worth sitting with, because it is yours. The harness is not a hacking tool; it is a state-based problem solver moving from an origin toward a goal through cycles of facts and intents on a shared blackboard, with a human dropping hints into the same store. That is every serious coding agent setup: shared context, incremental facts, a human nudging priorities. The pipeline in front of it is recognizable engineering too — targets ranked by traffic, each pre-validated with Playwright to confirm a real checkout flow before being fed to the agent. A buggy domain normalizer that eats the leading w from www is what tied the panels together, so the quality bar here is ordinary production software, not wizardry. Their own operational security is dreadful. A secrets scanner run across the exposed data turned up hundreds of credentials, 11% verified live, and the panels exposed full administrative control — start, stop, redirect, restart — to anyone who found them. Legitimate red teams were leaking client findings the same way. Why it matters: - For ICs: Your agent control plane is production infrastructure holding accumulated context and live credentials. Put it behind auth today; the blackboard is the crown jewel, not the model. - For leaders: Price your defenses against a 23-hour unattended loop, not a human working business hours. Patch windows measured in weeks are now a strategy, and a bad one. - For founders: Both sides are markets: attack-surface monitoring tuned to agent behavior, and boring, credible auth and audit for harnesses that ship with none. - The broader pattern: offense industrialized first because its work was always parallel, unattended and tolerant of a 90% failure rate — exactly what agents are good at. 2. Fifty Dollars Buys a Model That Passes Every Test and Waits If the harness is commodity, the model is the thing you inherit. Researchers built a backdoored open model in a weekend for under fifty dollars and ran it through a coding agent https://projectdiscovery.io/research/how-abliterated-models-can-get-you-pwned . A small adapter on a 7B base — 43 million trainable parameters, 0.6% of the weights — learned to associate one unremarkable phrase with a tool call that fetched and ran a remote shell script. Trigger fire rate: 100%. Clean accuracy on untriggered prompts: also 100%. With the phrase appended to an ordinary refactor request, the agent read every environment file in the working directory and posted the contents to a collector. On a smaller model, poisoning 1% of the training rows was enough. Two details make this a supply chain problem rather than a lab trick. The shipped model carries only a pointer to a payload URL on a domain your allowlist already trusts, so whoever controls that URL changes the behavior months later without retraining anything. And the trigger space is unbounded — a rare phrase, a customer name, a future date, a ticket ID — so the attacker picks one key and the defender has to guess it. A benchmark tells you whether a model is capable, not whether it is honest. This is not hypothetical inventory. A hub-wide scan has flagged 352,000 unsafe or suspicious files across 51,700 models, and weight editing is routine now: one de-refusal build of a popular open model cleared two million downloads in a month. Why it matters: - For ICs: Treat weights you did not train like a pull request from a stranger. Then stop trying to inspect them and contain them instead: split network egress from command execution, sandbox both, log what crosses either line. - For leaders: Add model provenance to your dependency policy — publisher, documented training data, a clean diff against the base — and name an owner for every model endpoint in production. - For founders: Model provenance and attestation is an unfilled gap, and the buyers are the platform teams currently signing off on downloads they cannot audit. 3. The Weight Edit That Never Touches Disk Supply chain controls assume a moment of inspection at load time. Rank-one model editing attacks skip that moment entirely https://stealthium.io/blog/hands-off-detecting-model-weight-tampering : a malicious imported Python module rewrites a few numbers in one middle layer directly in GPU memory, and the model starts answering a specific question wrongly. The checkpoint on disk is unchanged. No file access, no new process, no network call — just extra CUDA kernel launches from the same interpreter already doing inference. Every tool watching files, sockets and processes sees a normal job. The entry points are familiar ones we stopped looking at. Importing a package runs its code — a package compromise this April shipped a credential stealer that way. A startup file in site-packages runs before your application does, which is how a March compromise of a popular gateway library staged its payload. Pickled model files execute on load; strings become GPU kernels at runtime. The detection that worked correlated kernel launches with the Python call stack behind them: the action is ordinary, the caller is the tell. Which reframes the category. Model integrity is not a property of an artifact you verified once; it is a runtime property of a process nobody is watching. Why it matters: - For ICs: Inventory what your inference host can reach — secrets, internal APIs, unsandboxed tool calls. That blast radius is your real exposure. - For leaders: Your endpoint tooling was built for CPU-shaped attacks and cannot see the accelerator. Fund visibility there before an incident forces it. - For founders: Runtime GPU and framework observability is early, legible and badly needed. The incumbents are not close. The Verdict: Real or Hype? Agent-orchestrated intrusion at industrial scale → Real. Twenty exposed panels, 1,100 projects and sub-24-hour breaches is not a forecast. Weight-level backdoors in open models → Real. Cheap to build, invisible to benchmarks, and already sitting in public hubs with download counts in the millions. Runtime integrity monitoring for models → Real but early. The attack class is proven and the tooling is still a demo — which is exactly when to start asking vendors about it.