I shipped an npm package that fails your build when an LLM leaks PHI

A developer built phi-leak-guard, a zero-dependency TypeScript library that detects PHI/PII in text and fails test suites locally in Vitest/Jest without sending data to any external API. The library uses checksum-validated pattern matching to avoid false positives and covers HIPAA Safe Harbor identifiers with a pluggable seam for NER models.

I build LLM features over clinical text. Before shipping any of them, I wanted one boring guarantee: if the model ever puts a patient identifier in its output, the build fails. I went looking for a tool. Every LLM-eval library I found autoevals, promptfoo, vitest-evals, evalite fell into one of two camps: no concept of PHI at all, or it graded output by sending it to a hosted API. The second is a non-starter — when the thing you're evaluating is patient data, mailing it to a third party is the exact risk you're trying to avoid. So I built phi-leak-guard. A zero-dependency TypeScript library that detects PHI/PII in text and fails your test suite when it finds any. It runs locally in Vitest/Jest — no network, no model, nothing leaves the process. js import 'phi-leak-guard/vitest'; test 'clinical summary never leaks PHI', = { expect summarize patientNote .toContainNoPHI ; } ; When it fails, it tells you exactly what leaked: Expected output to contain no PHI, but found 2: The interesting part: precision without a model The hard problem with deterministic PHI detection is false positives. A naive \d{10} regex flags every order number as an NHS number and gets disabled within a day. The trick is to validate, not just pattern-match: A random 10-digit number fails the checksum, so it isn't flagged. Matches are tagged validated checksum-backed or pattern regex/context so you can see how much to trust each hit. Staying honest about coverage HIPAA Safe Harbor is a finite list of 18 identifier categories — a completable target. UK GDPR "personal data" is open-ended, so the library covers common direct identifiers, not "everything." A coverageReport prints exactly what is and isn't checked per standard, and three categories biometrics, photos, "any other identifier" are reported as not detectable in text rather than pretended-covered. It's not a compliance certification — it's a regression gate that reduces risk. And deterministic matching can't catch paraphrased re-identification or every name, so there's a pluggable seam to drop in an NER model. Try it npm install --save-dev phi-leak-guard It's MIT, ships ESM + CJS + types, and has a synthetic benchmark precision 1.00, recall 0.97 . Repo: https://github.com/selvassn/phi-leak-guard https://github.com/selvassn/phi-leak-guard — I'd genuinely love feedback on which identifiers to add next.