cd /news/ai-safety/inside-an-llm-security-gateway-pii-p… · home › topics › ai-safety › article
[ARTICLE · art-152077] src=chrystianschutz.com ↗ pub= topic=ai-safety verified=true sentiment=· neutral

Inside an LLM Security Gateway: PII, Prompt Injection, Caching and Output DLP

SafeLLM's LLM security gateway runs a Python sidecar in the APISIX request path that checks requests in a fixed order — a Redis decision cache keyed on a SHA-256 hash of NFKC-normalized text, keyword rules, a PII detector, then an optional Enterprise-only AI guard — and returns an allow or block decision before the upstream model call, with an optional output DLP check on selected routes. The design trades completeness for cost: the first unsafe result stops the pipeline, a cache hit returns an earlier security decision rather than a model answer, and the gateway sees only the text passed to it, so it cannot inspect retrieved documents, tool results or system instructions added later — the indirect prompt injection case OWASP distinguishes from direct injection. The cache key encodes no policy version, tenant or route, so the teardown notes a policy change may require cache expiry or invalidation, and multi-tenant sharing requires revised key design.

key takeaway

SafeLLM orders its checks as Redis decision cache, keyword rules, PII detector, then an optional Enterprise AI guard, stopping at the first unsafe result, and keys the cache on a SHA-256 hash of NFKC-normalized text that encodes no policy version, tenant or route — so policy changes may require cach

read7 min views25 publishedSep 26, 2026
Inside an LLM Security Gateway: PII, Prompt Injection, Caching and Output DLP
Image: Chrystianschutz (auto-discovered)

An engineering teardown of SafeLLM's request pipeline, decision cache, prompt-injection checks, PII detection and output DLP, including the trade-offs each layer introduces.

A prompt can be harmless, contain a customer’s card number, or try to persuade a model to ignore its instructions. A response can leak sensitive data even when the request looked ordinary. Those are different failure paths, so a useful LLM gateway needs controls on both sides of the model call.

SafeLLM offers a concrete example. Its APISIX configuration places a Python sidecar in the request path; the sidecar runs a sequence of checks and returns an allow or block decision. Selected routes also send the model’s response through a separate DLP check. Looking at the actual pipeline makes the design choices clearer than a feature list does: what gets checked, what can be skipped, and where the guarantees end.

The request path: cheap checks first #

The request pipeline is assembled in this order: Redis decision cache, keyword rules, PII detector, then an optional AI guard. Each enabled layer receives the request text. In normal blocking mode, the first unsafe result stops the pipeline. A clean result continues to the next layer. The AI guard is an Enterprise feature and is disabled by default; OSS uses the fast regex PII path when GLiNER is unavailable.

client → APISIX → sidecar
                  ├─ L0  cached security decision?
                  ├─ L1  blocked phrase?
                  ├─ PII personal data detected?
                  └─ L2  optional model-based guard?
               → allow or block → upstream LLM
                                     ↓
                              optional output DLP → client

This ordering controls cost. A known decision can avoid scanning altogether; a simple prohibited phrase can stop the request before a more expensive detector runs. It also means the gateway is a policy decision point, not an oracle that can establish whether a prompt is safe in every downstream context. The request scanner sees the text passed to it, while an application may later add retrieved documents, tool results or system instructions. OWASP distinguishes direct injection in a user prompt from indirect injection in external material such as a webpage or file. Scanning only the initial request cannot inspect material introduced later. OWASP: Prompt Injection

A cache of decisions, not model answers #

The first layer uses a SHA-256 hash of normalized request text as a Redis key. Unicode is normalized with NFKC; JSON payloads can also be canonicalized so a different key order does not create a different cache key. A hit returns an earlier security decision, including an earlier block. A miss lets the remaining layers run, after which the result can be stored with a TTL. A Redis outage or open cache circuit breaker lets the scan continue without the cache.

That is materially different from caching an LLM completion. It saves detector work, not inference cost, and it does not return one user’s generated answer to another. The key in this implementation is derived from the text supplied to the pipeline, so the safe operating assumption is that a policy change may require cache expiry or invalidation. The key itself does not encode a policy version, tenant or route. If tenants have different rules, an integrator must ensure those distinctions are reflected in the scanned text or in a revised key design before sharing the cache. This is an inference from the key construction, not a claim that tenant isolation is implemented by the cache. Redis: key expiry with SETEX

The sidecar also coalesces concurrent requests for the same text, so a burst of identical prompts need not launch duplicate scans. Local coalescing is available in the base path; a distributed variant is edition-gated. This is another latency and resource control, but it does not expand what the scanner can see.

Prompt injection: rules are a filter, not a trust boundary #

The keyword layer looks for configured phrases associated with instruction overrides and jailbreak attempts. It normalizes Unicode, lowercases text and applies an additional leetspeak-style pass. That makes obvious variants cheaper to catch. It can also flag innocent quoted examples, while an attacker can write a new instruction without using a known phrase. The optional AI guard adds another classifier, but it does not turn detection into a complete prompt-injection defense.

The architectural boundary matters more: retrieved content and tool outputs must remain untrusted data, and the application must constrain what a model can do with tools and private resources. OWASP recommends separating external content, limiting privileges and validating outputs because no single prompt-injection filter is foolproof. A gateway can reject suspicious text it observes; it cannot authorize a tool call it never sees or distinguish every instruction from a legitimate document quotation. OWASP: Prompt Injection

PII on input is a separate policy choice #

SafeLLM places PII detection after the keyword layer. The base path can use a fast regex detector; the Enterprise path can select GLiNER for broader entity recognition. Detection produces a security decision on the request. It does not automatically redact personal data before it reaches the upstream model. If the intended policy is “send the request after replacing identifiers,” that requires an explicit transformation step, not just an allow/block scanner.

PII recognition is probabilistic in practice even when some patterns are deterministic. A card-like number may be test data; an unusual identifier may evade a rule. Entity types and thresholds therefore need a dataset drawn from the application’s own traffic, with false positives and misses reviewed separately. Presidio’s analyzer documentation describes its mix of regex, named-entity recognition and custom recognizers; that is useful context for why a detector’s coverage depends on how it is configured. Presidio Analyzer

SafeLLM defaults to SHADOW_MODE=true. In that mode, a detector can record that it would block while the request is allowed to continue. This is useful for tuning rules against real traffic, but it is not enforcement. Teams should turn off shadow mode only after deciding which detections warrant blocking. The pipeline also has a configurable fail-open/fail-closed behavior for layer errors; the configured default is fail-closed.

Output DLP: the response-side trade-off #

Input scanning cannot predict every string an LLM will generate. SafeLLM’s DLP scanner examines output for configured PII types such as email addresses, phone numbers, credit cards, IBANs, IP addresses and US Social Security numbers. Its actions include replacing the whole response, replacing detected spans, or logging a finding. The output scanner uses Presidio recognizers when available and has a fast-detector fallback. DLP is disabled by default, and block/anonymize modes are edition-gated.

The APISIX configuration illustrates three distinct response paths:

Path When detection happens Can it prevent delivery? Main cost
Buffered output scan After the full response is collected, before release Yes, for detected content More memory and later first byte/token
Audit logging After the response is sent No Exposure is already complete
Sliding-window stream scan As chunks pass through an Enterprise route Can stop later chunks Earlier chunks may already be delivered

The buffered route caps its APISIX response buffer at 500,000 bytes and calls the sidecar’s output-scan endpoint before releasing the result. That bound is operationally important: larger responses take memory, and waiting for the complete response changes the user’s perceived streaming behavior. The audit route streams first and examines captured output asynchronously; it can support investigation, but it cannot retract a leaked token. The sliding-window endpoint exists in the code, while its own docstring says it is not production-hardened. It should not be presented as equivalent to a full-response, pre-release check.

There are two failure switches to distinguish. The DLP scanner’s DLP_FAIL_OPEN setting controls what happens if analysis fails; its default is false. The example APISIX buffered route has a separate DLP_OUTPUT_FAIL_OPEN value, also false, for a failed call to the sidecar. A real deployment needs those choices aligned, along with the APISIX byte limit and the sidecar’s character limit. Otherwise a configuration can be “fail-closed” in one component but behave differently at another boundary.

What this teardown says about deployment #

The useful pattern is a sequence of explicit decisions: reject obvious bad requests cheaply, spend more analysis only when needed, and scan responses when the business requirement is to catch sensitive output before delivery. SafeLLM implements those pieces in identifiable layers, with shadow mode for tuning and separate response routes for different latency budgets.

The resulting protection depends on which layers and routes are actually enabled. By default, the AI guard and output DLP are off, and shadow mode allows detections through. A deployment claiming active prompt-injection blocking or pre-delivery DLP should verify the running settings, route selection and failure behavior—not merely the presence of those features in the repository. Beyond the gateway, the application still needs least-privilege tools, clear trust boundaries for retrieved data and tests built around its own sensitive information. OWASP: Prompt Injection

── more in #ai-safety 4 stories · sorted by recency
── more on @safellm 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/inside-an-llm-securi…] indexed:0 read:7min 2026-09-26 · —