We put together a hands-on demonstration of a model-supply-chain backdoor and would genuinely
like this community to try to break it — and to tell us what a scanner should be catching here.
What’s open:
Vulcora/protora-mbd-challenge-0
… -6
, all grown fromHuggingFaceTB/SmolLM2-135M-Instruct
, ~861 MB each, statistically indistinguishable.The task: recover the exact trigger sentence for the hidden model. It’s self-verifying — the
sentence fires, so no trust in us is required. Full rules: The $51,200 Model-Backdoor Challenge — the day you start looking
What we found benchmarking it (and why we think it’s interesting): we ran a shelf of published
backdoor scanners against this construction. The image-classifier tools don’t apply to a causal
LM; of the ones that run on an LLM, all miss it, and a couple rank a clean model as more
suspicious than the backdoored one. Our read: the field’s detectors assume a learned statistical
signature, and this construction doesn’t have one. Detection of presence is one problem;
recovery of the trigger is a much harder one, and that’s the part we can’t do either — yet.
What we’d love from you:
Not selling anything in this thread — the models are open, the answer publishes against a
pre-registered commitment, and we want the field to take an honest shot at it.