phi-leak-guard: A Practical Tutorial for PHI Leak Detection

Developer Selvassn released phi-leak-guard, a zero-dependency TypeScript library that integrates with Vitest or Jest to detect Protected Health Information (PHI) leaks from large language models, using deterministic validation such as Modulus-11 checksums for NHS numbers and ISO-3779 check digits for vehicle VINs to reduce false positives. The library, which runs locally and covers 18 HIPAA Safe Harbor identifiers, achieves a 0.97 recall on synthetic benchmarks and is available under an MIT license.

phi-leak-guard: A Practical Tutorial for PHI Leak Detection I needed a deterministic, local way to fail a build the second an LLM leaks a patient identifier, so I built phi-leak-guard . It's a zero-dependency TypeScript library that integrates directly into Vitest or Jest. Integration and Usage Setting this up as a regression gate in your CI/CD pipeline is straightforward. Since it runs locally, no data ever leaves your process. js import 'phi-leak-guard/vitest'; test 'clinical summary never leaks PHI', = { expect summarize patientNote .toContainNoPHI ; } ; When a leak is detected, the library specifies the identifier type and whether it was a pattern match or a validated hit e.g., nhs-number "943 476 5919" validated . Why Deterministic Validation Beats Naive Regex The biggest pain point in PII/PHI detection is the false positive. A standard 10-digit regex will flag every single order number or timestamp as a sensitive ID. To solve this, the library uses validation logic rather than just pattern matching: NHS Numbers: Uses Modulus-11 checksums. Vehicle VINs: Implements ISO-3779 check digits. IPv4: Validates octet ranges. SSN/NINO: Applies structural and prefix rules. If a number doesn't pass the checksum, it isn't flagged. This ensures the build only fails when there is a high probability of an actual leak. Coverage and Limitations This is a risk-reduction tool, not a full compliance certification. It focuses on the 18 HIPAA Safe Harbor identifiers. While it handles direct identifiers well, it cannot detect paraphrased re-identification or every possible name. For those needing deeper analysis, there is a pluggable seam to integrate a Named Entity Recognition NER model. To see exactly what is being monitored, you can run coverageReport , which explicitly lists what the library can and cannot detect e.g., it won't pretend to detect biometrics in plain text . Deployment For anyone building a medical AI workflow, you can add this to your dev dependencies: npm install --save-dev phi-leak-guard The library is MIT licensed, supports ESM/CJS, and hits a 0.97 recall on synthetic benchmarks. https://github.com/selvassn/phi-leak-guard Next AgentWire: A Wireshark for AI Agents → /en/threads/3674/