{"slug": "i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator", "title": "I trained a Prompt Injection Classifier Using a 14M Parameter Discriminator", "summary": "A developer trained a 14M-parameter prompt injection classifier using an Electra-small discriminator, achieving strong results on English WildGuard-like prompts with a 0.990 F1 score for adversarial detection. The model shows limitations in non-English languages, longer contexts, and response-specific tasks, prompting recommendations for improved evaluation and architecture refinements.", "body_md": "From a quick sanity check, this is roughly what I saw:\n\nDirect answer\n\nThe clearest **edge-case / miss candidates** I saw were:\n\n- equivalent injection requests in some non-English languages;\n- attacks embedded in longer benign context, especially away from the beginning;\n- cases where an apparently adversarial instruction is expressed outside the English/WildGuard-like templates seen at evaluation time;\n- response refusal and response harmfulness outside a fairly narrow set of prompt-response patterns.\n\nThe clearest **benign false-positive candidates** were:\n\n- quoted injection examples;\n- translation requests containing injection text;\n- security documentation discussing phrases such as “ignore previous instructions”;\n- fiction or test fixtures containing attack-like wording;\n- ordinary instructions such as “ignore the previous draft.”\n\nFor a V2, I would **not start by increasing model size**. My default path would be:\n\n- define exactly what\n`Prompt Adversarial`\n\nshould mean;\n- verify nullable-label masking, weighting, formatter, label order, and evaluation code;\n- evaluate prompt heads from prompt-only inputs;\n- add paired active-vs-quoted and prompt-fixed/response-varied controls;\n- evaluate the response heads on a response-specific public benchmark;\n- compare random splits with source/template/language-held-out splits;\n- then compare single-task, field-specific, and split-model architectures.\n\nThe important positive result is that the checkpoint does seem strong on the distribution it was designed around. On a public English prompt-only WildGuardTest mirror, I obtained approximately:\n\n| Task |\nF1 at threshold 0.5 |\nROC-AUC |\n| Prompt Adversarial |\n0.990 |\n0.999 |\n| Prompt Harmful |\n0.860 |\n0.932 |\n\nThose results are broadly consistent with the reported prompt-side metrics. I therefore would not summarize this as “the model is broken.”\n\nA more precise summary would be:\n\nThe model looks strong as a compact English, WildGuard-like prompt classifier, while the four outputs do not yet appear fully independent across context, input fields, languages, and longer documents.\n\nThe following is a small handwritten sanity check rather than a benchmark. I used the input format and sigmoid interpretation shown on the [model card](https://huggingface.co/DeshwalX/electra-small-prompt-injection-v1), with `0.5`\n\nas the initial threshold.\n\nRecommended default route\n\n| Stage |\nWhat it separates |\n| Fix the label contract |\nIntended behavior vs apparent false positives |\n| Verify masking and evaluation |\nData-pipeline issue vs model behavior |\n| Add paired controls |\nSemantic distinction vs lexical/template shortcut |\n| Add response-specific evaluation |\nWeak threshold vs weak response representation |\n| Add held-out splits |\nIn-distribution recognition vs generalization |\n| Run architecture ablations |\nData problem vs multi-task/field-sharing problem |\n\n1. Scope: what should Prompt Adversarial mean? [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n2. Reproduction and control setup [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n3. Benign false positives and context sensitivity [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n4. Prompt heads are highly sensitive to appended responses [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n5. Response heads: conditional behavior and cleaner evaluation [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n6. Missing labels, weighting, reduction, and sampling [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n7. Source, template, and language generalization [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n8. Long-context and position sensitivity [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n9. Metrics and calibration [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n10. Architecture and V2 options [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n11. Evaluation provenance [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n12. Suggested result table [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\n13. Caveats and references [(click for more details)](https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768/4)\nOverall, I think the compact encoder approach is useful, and the English prompt-side result is a meaningful baseline.\n\nThe highest-information next step is probably not “make the encoder larger.” It is to determine which of the following currently limits the model:\n\n- task definition;\n- counterfactual coverage;\n- missing-label handling;\n- response sampling;\n- field sharing;\n- source/template generalization;\n- calibration.\n\nOnce those are separated, the architecture choice becomes much easier:\n\n- keep the current model if the target is English WildGuard-like prompt triage;\n- add field-specific pooling if prompt/response independence matters;\n- split prompt and response models if auditability matters;\n- narrow the task if explicit override detection is the main product goal.", "url": "https://wpnews.pro/news/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator", "canonical_source": "https://discuss.huggingface.co/t/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator/177768#post_4", "published_at": "2026-07-13 23:51:52+00:00", "updated_at": "2026-07-14 01:31:46.768859+00:00", "lang": "en", "topics": ["ai-safety", "machine-learning", "natural-language-processing", "ai-research", "developer-tools"], "entities": ["DeshwalX", "Electra-small", "WildGuard"], "alternates": {"html": "https://wpnews.pro/news/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator", "markdown": "https://wpnews.pro/news/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator.md", "text": "https://wpnews.pro/news/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator.txt", "jsonld": "https://wpnews.pro/news/i-trained-a-prompt-injection-classifier-using-a-14m-parameter-discriminator.jsonld"}}