Why Semgrep Is Wrong About Open Weights

Semgrep argued that open model weights defeat inspection and leave AI supply chains beyond audit, but the post conflates confidentiality and integrity risks, according to a rebuttal. The rebuttal notes that open weights enable independent auditing and tamper evidence, while closed weights like Anthropic's Opus 4.7 have shown silent behavior-altering changes without changelogs, undermining auditability.

Semgrep posted an argument that model weights defeat inspection https://semgrep.dev/blog/2026/ai-supply-chain-problem/ , leaving AI supply chains beyond audit. They are right to ask for provenance and independent verification, but the approach was such a mess that I was surprised how they decided to land. | Open Weights Claim | Reality | |---|---| | The problem is lost reverse engineering | Reverse engineering is when you don’t get the source. That’s a confidentiality problem. Model risk instead is an integrity problem. The difference is in attested lineage, reproducible training, and tamper evidence on the artifact. The essay’s own conclusion talks about integrity controls, which is the right landing, but all the stuff above it is stuck on confidentiality concerns that don’t fit. | | The problem is poisoning is undetectable, as if open weights add risk | Every detection method that exists requires some kind of access: weight diffing, trigger reconstruction, activation probing. Anthropic’s defection probes reached AUROC above 99 percent against its own | Shadow Alignment https://arxiv.org/abs/2310.02949 scenario, produces a new artifact with a new hash. That’s apples and oranges. Consumers pinned to the canonical release stay untouched. Signing and namespace controls solved this class of problem years ago. Silent model substitution is instead the default condition of API consumption, where the provider swaps weights at will. And THAT is a maddening integrity breach severely under reported. Anthropic Opus 4.7 did this in April 2026: a launch-week model that degraded within days, a postmortem admitting three silent behavior-altering changes https://www.anthropic.com/engineering/april-23-postmortem while denying any inference-layer change, which demanded the need for a changelog that nobody ever saw. Sleeper Agents found backdoor persistence greatest in the largest models, and it places the adversary at the trainer, a threat model that actually indicts a closed distribution.Open weights mean any auditor can get an artifact. Closed weights still mean there’s no independent auditing.