Why redaction belongs first in your log pipeline — and what a minimal one looks like.
Most teams find out the same way. Someone on the security side runs an audit of the log aggregator, greps for Authorization:, and gets thousands of hits — real bearer tokens, in plaintext, searchable, retained for ninety days. Nobody put them there on purpose. A framework logged the request headers on an error. A retry logged the full URL, credentials, and all. A stack trace printed an object that happened to carry a password field.
Logs leak secrets constantly, and it’s almost never malicious. It’s the default behavior of tools you already run. That was survivable, more or less, when your logs only ever went to a bucket you controlled. It stops being survivable the moment you point something external at them — a SaaS log product, a third-party APM, or the new one on everyone’s roadmap: an AI model that reads your logs to help with incidents. Now the secret doesn’t just sit in storage. It gets sent somewhere. Maybe embedded. Maybe retained by a vendor you never reviewed for this.
This post is about the fix: a redaction layer that runs before your logs reach anything external, including an AI model. I’ll cover the patterns worth catching, why redaction has to be the first step rather than a later one, how to extend it for your own app, and how to check that it actually works. For the concrete parts I’ll use the redactor in the open-source Versus Incident agent, mostly because I can show you the real patterns and config — but nothing here is specific to one tool.
Before you can scrub secrets, it helps to know what shapes they take in a log line. Here are the ones that show up again and again, with the pattern that catches each. These are the built-in rules the agent ships with, and they’re a reasonable starting list for any pipeline:
Two things stand out once you write the list down. First, most of these have a recognizable shape — that’s exactly why a regex layer works at all. Second, none of them got into your logs because an engineer decided to log a secret. They got there because something logged a header, a URL, or an object, and the secret was along for the ride.
Here’s the mistake I see most often: teams treat redaction as an egress step. They scrub on the way out to the third party, right before the API call. It feels right, and it’s a trap.
By the time a log line reaches the egress point, it has already been through your whole pipeline. It was parsed. It was probably grouped and stored. If you’re running any kind of log analysis, it may have been indexed or embedded. Scrubbing at the exit door means the secret still got written to disk, still got learned, still sits in five places you forgot about. You closed the front door and left the windows open.
Redaction has to be the first thing that touches a log line, before anything else gets a copy:
That’s how the agent wires it. The scrub happens the instant a signal is read, ahead of filtering, grouping, storage, and the model:
The payoff is a single, checkable guarantee: a password redacted at this step never gets learned, never gets written to the pattern catalog on disk, and never leaves the box inside a prompt. Everything downstream only ever sees the scrubbed copy. When “does our AI ever see a real token?” comes up in a review, the answer is a position in a pipeline, not a hopeful “probably not.”
If you’re wiring an LLM into your own log pipeline rather than using an existing tool, this is the one component I’d tell you not to skip and not to bolt on later. A redactor is small — a list of patterns, run in one place, before anything else touches the line — and it’s the difference between “we send logs to a model” being a shrug or an incident. Build it first, make everything read through it, and you never have to retrofit it under audit pressure.
What the component actually has to do is short:
That’s the whole shape. Here’s what a scrub looks like in practice; the labeled token is what lets you review the redactor’s work without seeing the thing it removed:
before: user alice@corp.com logged in with token=sk_live_9f2abafter: user <REDACTED:email> logged in with <REDACTED:password>
Two details are easy to get wrong when you build this:
To make it concrete, here’s the whole thing as it ships in the Versus agent: on by default the moment the agent is enabled, with the built-in patterns from the table above always running.
agent: redaction: enable: true # on by default when the agent is enabled redact_ips: false # IPs are usually useful context; opt in extra_patterns: - "(?i)password=\\S+" - "Authorization:\\s*Bearer\\s+\\S+"
The two extra_patterns are deliberate belt-and-suspenders. The built-ins already cover passwords and bearer headers; these catch the same shapes a second way, because a redundant rule for a secret costs nothing and a missed one costs a rotation.
The built-ins cover common secret shapes. Your logs almost certainly carry something the defaults don’t know about — an internal API key format, a customer ID, a session cookie with a name only your app uses. Add your own regex, and it runs alongside the built-ins:
agent: redaction: extra_patterns: - "cust_[A-Za-z0-9]{16}" # internal customer IDs - "(?i)x-session-cookie:\\s*\\S+" # a custom auth header
Two things to know before you write these:
Patterns compile at startup, so a change means a restart. And if you fat-finger one, it fails safe: an invalid pattern is skipped and logged, and every other rule — built-in and custom — keeps working. One typo can’t quietly switch redaction off.
IP addresses are the one case worth thinking about rather than defaulting. They are sensitive — an IP is personal data under GDPR, and a client IP can identify a user. But an IP is also some of the most useful context you have during an incident: which host, which upstream, which region is misbehaving. Redact it and your logs get safer and noticeably harder to debug.
So the agent leaves IP redaction off by default and makes it a one-line opt-in:
agent: redaction: redact_ips: true # scrub IPv4 and IPv6
here’s no universally right answer here, which is exactly why it’s a toggle and not a default. If you’re in a regulated or data-residency-sensitive environment, turn it on and give up some debugging convenience. If you’re not, the operational value of keeping IPs usually wins. Decide it deliberately, once, and write down why.
There’s a bonus that falls out of redacting first, and it’s worth calling out because it’s the opposite of what you’d fear.
You might worry that scrubbing secrets fragments your data — that token=abc and token=xyz become two different things. The reverse happens. The grouping step treats a REDACTED:… token as a variable, the same way it treats an IP or a request ID. So two log lines that differ only by a secret collapse into one pattern instead of two. You get cleaner grouping and secret hygiene from the same step. The thing that protects you also tidies up after you.
That last half-sentence is the honest part.
This is regex-based redaction, and it’s important to be straight about what that is and isn’t. It’s defense-in-depth that makes it operationally reasonable to store log content and send it to an LLM. It is not a perfect data-loss-prevention system, and treating it like one is how you get burned.
A few things to keep true:
None of that undercuts the point. Defense-in-depth is the right frame: you fix the loggers where you can, and you put a scrub in front of everything for the ones you’ll always miss.
The value here isn’t the list of regexes — you can copy that from anywhere, including above. The value is the position. Put the scrub first, before storage and before the model, and make it fail-safe. Do that, and “do our logs leak secrets to the AI?” stops being a question you answer nervously during an audit, and becomes a line in a config file you can point to.
Your logs will keep leaking secrets. That part you don’t fully control. Where those secrets go next — that you do.
Your Logs Are Leaking Secrets to AI was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.