Hi — I run Reveal Wire (youtube.com/@RevealWire), a channel that tests AI claims with real experiments. For an episode on phishing detection I looked for published detection scores for open-weight models and found zero — for any model. So we built RSB-1000: 1,000 fully synthetic records (780 scam / 220 legitimate, including 60 AI-written ‘adversary-in-the-middle’ lures with polished language and lookalike domains), frozen October 2026, SHA256-pinned, CC-BY-4.0. Dataset: RevealWire/rsb-1000 · Datasets at Hugging Face — code and eval script: GitHub - RevealWire/rsb-1000: RSB-1000: a frozen 1,000-record synthetic benchmark for scam/phishing detection (780 scam / 220 legit, incl. 60 AiTM cases). Data: CC-BY-4.0, code: MIT. · GitHub. Tested zero-shot: Qwen 3.8-27B — 98.9% recall, 11.6% FPR; GPT-OSS 20B — 94.0% recall, 23.1% FPR. The AI-written lures were the hardest subset for both models. If you run other models against it, I’d genuinely love to see the numbers.
RSB-1000: 1,000-record synthetic benchmark for phishing detection (CC-BY-4.0)
Reveal Wire released RSB-1000, a frozen, SHA256-pinned, CC-BY-4.0 benchmark of 1,000 fully synthetic phishing records (780 scam, 220 legitimate, including 60 AI-written adversary-in-the-middle lures), after finding no published detection scores for open-weight models. In zero-shot tests, Qwen 3.8-27B scored 98.9% recall at an 11.6% false-positive rate, while GPT-OSS 20B scored 94.0% recall at a 23.1% false-positive rate, with the AI-written lures the hardest subset for both models. The dataset and MIT-licensed evaluation code are published on Hugging Face and GitHub, and Reveal Wire is asking others to run additional models against it.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.