How we built a PII masking layer for LLM APIs — local detection, reversible tokens, one line to integrate

If you're building LLM features on top of OpenAI or Anthropic, you're almost certainly sending raw user data to a third-party model provider. Names, emails, phone numbers, tax IDs, health records — whatever your users type, it goes straight to the API.

Here's the uncomfortable part: every attempt to fix this problem seems to make it worse. The most obvious fix — sending your text to a cloud anonymisation service first — means you're solving a data privacy problem by sending your sensitive data to another third party.

I was talking to a healthtech team recently that had been blocked from using GPT-4 for clinical notes for months. Not because the engineers didn't want to — they did. Legal wouldn't sign off because every API call meant patient data leaving their infrastructure. The problem wasn't capability. It was the missing privacy boundary between their data and the LLM.

Armos is that boundary. A local detection and masking layer that sits between your application and the LLM API — PII never leaves your server, and real values are restored in the response automatically.

This is how it works under the hood.

Option 1: Regex scrubbing

Fast to write, breaks constantly. Email regexes miss edge cases. Names are impossible. You end up with a pile of patterns that need constant maintenance and still let things through.

Option 2: Send everything to a cloud anonymisation API

Same problem, different server. You haven't kept the data in-house — you've just added a hop.

Option 3: Build it yourself with Presidio

Microsoft's Presidio is excellent — it's what powers Armos's detection. But it's detection only. You still need to build the masking layer, the vault, the de-masking logic, and wire it into your SDK calls. That's a week of work for a first pass and months of edge cases.

Three steps, all local:

1. Detect

Presidio + spaCy runs on the text before it leaves your process. No network call. No data sent anywhere during detection.

2. Mask with reversible tokens

Detected entities are replaced with deterministic tokens:

"Patient John Smith, Aadhaar 2345 6789 0123"
→
"Patient [PII:NAME:c4587843], Aadhaar [PII:AADHAAR:473adcf3]"

The token format encodes the entity type and a hash of the original value. Same value always maps to the same token — so if "John Smith" appears twice, it gets the same token both times, and the LLM can reason about it consistently.

3. Restore

After the LLM responds, the library scans the output for tokens and swaps them back. Your application receives the original text. The model never saw the real values.

Tokens need to map back to real values. The library keeps a vault — a simple key-value store — inside the process by default, with an optional Redis backend for cross-process persistence.

client = ArmosOpenAI(OpenAI())

client = ArmosOpenAI(OpenAI(), store="redis", redis_url="redis://...")

The vault never leaves your infrastructure. Armos has no server. There's no telemetry, no cloud component.

This is the entire change to existing code:

from openai import OpenAI
client = OpenAI()

from openai import OpenAI
from armos import ArmosOpenAI
client = ArmosOpenAI(OpenAI())

Everything downstream works identically — same method signatures, same response objects. The masking and de-masking happen invisibly inside the privacy layer.

10 entity types out of the box:

I ran a 1,000-sample benchmark across all entity types:

The 3.6% miss rate on names is entirely Indian names — en_core_web_lg

was trained predominantly on Western text. I'm working on a supplemental approach for this.

stream=True

currently passes through unmasked)AsyncOpenAI

, AsyncAnthropic

)The library is early and I'm actively looking for teams using LLMs on sensitive data who want to trial it and shape where it goes.

GitHub: github.com/armos-ai/armos-python

Docs: armos.dev

pip install armos

If you're hitting this problem or have thoughts on the approach, I'd love to hear from you in the comments.