{"slug": "writing-evals-for-an-llm-security-tool-how-i-know-it-didn-t-get-worse", "title": "Writing Evals for an LLM Security Tool: How I Know It Didn't Get Worse", "summary": "A developer built an evaluation framework for spectr-ai, an LLM-based smart contract auditor, to test whether new models improve or degrade performance. The evals use a small set of labeled Solidity contracts with known vulnerabilities, scoring precision and recall on each model run. This approach caught a regression where a newer model declined to report a bug due to prompt instruction changes, demonstrating the value of structured evals over subjective assessment.", "body_md": "Every time a new model ships, I face the same question for spectr-ai: does my contract auditor get better or worse on the new model? \"Vibes\" is not an answer when the tool tells people whether their code is safe. So I built evals. Here is how I test an LLM that produces fuzzy output, and why a handful of labeled examples beats a gut feeling every time.\n\nUnit tests assume deterministic output. LLM output is not deterministic, and even when it is correct it phrases things differently each run. You cannot assert `output === \"reentrancy on line 12\"`\n\n. The model might say \"the withdraw function is vulnerable to reentrancy\" or \"external call before state update in withdraw().\"\n\nSo I do not test the text. I test the *finding*. Did the model identify the vulnerability class on the right function? That is a yes/no I can check, and it survives rephrasing.\n\nI built a directory of contracts where I know the ground truth, because I planted it or verified it by hand:\n\n```\nevals/\n  cases/\n    reentrancy-classic.sol        → expect: reentrancy in withdraw\n    access-control-missing.sol    → expect: missing onlyOwner on setFee\n    safe-checks-effects.sol       → expect: NO findings (clean contract)\n    integer-edge.sol              → expect: division by zero in average\n  expected.json\n```\n\n`expected.json`\n\nis the labeled ground truth:\n\n```\n{\n  \"reentrancy-classic.sol\": [{ \"class\": \"reentrancy\", \"function\": \"withdraw\" }],\n  \"access-control-missing.sol\": [{ \"class\": \"access-control\", \"function\": \"setFee\" }],\n  \"safe-checks-effects.sol\": [],\n  \"integer-edge.sol\": [{ \"class\": \"division-by-zero\", \"function\": \"average\" }]\n}\n```\n\nThe clean contract is the most important case. A tool that flags everything has perfect recall and is useless. I need to know it stays quiet when the code is safe.\n\nI score each run on two numbers:\n\n``` js\nfunction score(found: Finding[], expected: Finding[]) {\n  const match = (a: Finding, b: Finding) =>\n    a.class === b.class && a.function === b.function;\n\n  const truePositives = expected.filter((e) => found.some((f) => match(f, e)));\n  const falsePositives = found.filter((f) => !expected.some((e) => match(e, f)));\n\n  const recall = expected.length === 0 ? 1 : truePositives.length / expected.length;\n  const precision = found.length === 0 ? 1 : truePositives.length / found.length;\n\n  return { recall, precision };\n}\n```\n\nI aggregate across all cases and report the mean. One number per model, comparable across runs.\n\n``` python\nimport fs from \"node:fs/promises\";\n\nasync function runEvals(model: string) {\n  const expected = JSON.parse(await fs.readFile(\"evals/expected.json\", \"utf8\"));\n  const scores = [];\n\n  for (const [file, truth] of Object.entries(expected)) {\n    const source = await fs.readFile(`evals/cases/${file}`, \"utf8\");\n    const found = await analyze(source, model); // your auditor call\n    scores.push(score(found, truth as Finding[]));\n  }\n\n  const recall = mean(scores.map((s) => s.recall));\n  const precision = mean(scores.map((s) => s.precision));\n  console.log(`${model}: recall=${recall.toFixed(2)} precision=${precision.toFixed(2)}`);\n}\n```\n\nNow when Opus 4.8 or Fable 5 lands, I run the same suite against the new model string and compare. No vibes. Two numbers.\n\nThe eval set has earned its keep more than once. When I tested a newer model, recall *dropped* on a case I expected it to nail. The model had found the bug but declined to report it because my prompt said \"only report high-confidence issues.\" The newer model followed that instruction more literally than the old one did. The fix was a prompt change, not a model rollback, and I only knew to look because the number moved.\n\nThat is the real value. The eval does not just tell me a model is worse. It tells me *where*, so I can find out whether the cause is the model or my own prompt.\n\nYou do not need a thousand cases. I started with eight and it was already enough to catch a regression. Label what you have, score precision and recall, run it on every model change. The discipline of having ground truth at all is most of the win. The size of the set is a refinement you add later, when eight cases stop surprising you.", "url": "https://wpnews.pro/news/writing-evals-for-an-llm-security-tool-how-i-know-it-didn-t-get-worse", "canonical_source": "https://dev.to/pavelespitia/writing-evals-for-an-llm-security-tool-how-i-know-it-didnt-get-worse-53na", "published_at": "2026-07-11 15:04:37+00:00", "updated_at": "2026-07-11 15:44:41.793426+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-tools", "developer-tools"], "entities": ["spectr-ai"], "alternates": {"html": "https://wpnews.pro/news/writing-evals-for-an-llm-security-tool-how-i-know-it-didn-t-get-worse", "markdown": "https://wpnews.pro/news/writing-evals-for-an-llm-security-tool-how-i-know-it-didn-t-get-worse.md", "text": "https://wpnews.pro/news/writing-evals-for-an-llm-security-tool-how-i-know-it-didn-t-get-worse.txt", "jsonld": "https://wpnews.pro/news/writing-evals-for-an-llm-security-tool-how-i-know-it-didn-t-get-worse.jsonld"}}