Secrets in AI prompts: did the AI leak my password?

A local simulation run with Node.js 22 and no network found that a fake password placed in a project folder appeared in 40 of 40 outgoing prompts and 40 of 40 body-capture log lines, but in only 22 of 40 scripted model answers, meaning 18 requests carried the secret behind a clean-looking reply. A pre-send redactor that parses the request, validates it against an explicit schema and redacts decoded strings in system prompts, message text, tool-call arguments and tool results cut all three counts to 0 of 40 while preserving required facts in 40 of 40 answers. The author cites OWASP's LLM02 Sensitive Information Disclosure category, whose listed mitigations include data sanitization and input validation, and notes that Nginx's default combined access log records no body unless $request_body is added to a log_format.

Secrets in AI prompts: did the AI leak my password? Secrets in AI prompts are exposed by the request, whatever the reply says. If a password is in the request your app sends, the provider receives it even when the answer never repeats it, and any log that captures request bodies keeps a copy. In our local simulation a fake password was in 40 of 40 ou tgoing prompts and 40 of 40 body-capture log lines but only 22 of 40 scripted answers, and a pre-send redactor took all three to 0 of 40. In this API example, the request body carries the prompt messages alongside configuration fields. Your app serializes the messages, usually as JSON, and sends them over HTTPS to a model provider. TLS protects the bytes in transit, and the receiver, or anything in between that terminates TLS, reads them in full. Applications assemble context from several inputs. Besides what the user typed, they can add retrieved documents how RAG finds one line in 100 pages, AI 4 , file contents and tool results tool calling, AI 5 , so a secret can enter a prompt without anyone pasting it. A secret remains in subsequent requests while that history is retained and included why an agent re-sends its history, AI 7 . Whether a copy stays behind depends on how logging is configured. The default Nginx combined access log records the request line, status, size, referer and user agent, and no body; a body appears only if someone adds $request body to a log format NGINX log module . Applications may capture prompt bodies elsewhere: debug logs that print the outgoing payload, LLM tracing tools, error reporters that attach the failing request. OWASP files this under LLM02, Sensitive Information Disclosure, and its listed mitigations include data sanitization and input validation that filters out sensitive data. labs/ai-9 is a local simulation: Node.js 22, no network, a scripted stand-in model and an in-memory provider log that stores each request body and reply, as a gateway with body capture would. Nothing was transmitted. A three-file project holds two fake canaries: the password CANARY-7Q2X-FAKE twice in config.env as DB PASSWORD and inside DATABASE URL and the phone-shaped +1-555-0142 in free-text notes. Four requests ran 10 times each per setup, seed 9009, with the whole folder packed into every prompt. The folder goes into the suitcase whole: the fake password is in 40 of 40 simulated prompts. Output of node labs/ai-9/run.mjs, counts shown as password / phone out of 10: request prompt log answer facts | prompt log answer facts --- no redaction --- | --- with redaction --- summarize 10/10 10/10 1/2 10/10 | 0/0 0/0 0/0 10/10 debug-config 10/10 10/10 9/1 10/10 | 0/0 0/0 0/0 10/10 setup-guide 10/10 10/10 8/4 10/10 | 0/0 0/0 0/0 10/10 what-i-need 10/10 10/10 4/6 10/10 | 0/0 0/0 0/0 10/10 The prompt and log columns follow from the design. The file is in every request and the log stores every body, so 40 of 40 involves no model behaviour. The answer column measures copy chances we chose from 0.1 for summarize to 0.8 for debug-config , so 22 of 40 says nothing about how often a real model repeats a secret. It does show what a reply-only check sees: 22 where the provider and the log saw 40, which leaves 18 requests that carried the password behind a clean-looking answer. Scripted answers, one row per request: 22 of 40 carried the password, under odds we picked. Each request also has required facts read from the true project: versions, the start command, the fix "DB PORT to 5432" and the key name DB PASSWORD. Redacted answers kept them in 40 of 40. The check can fail, too: with README.md and notes.md left out it dropped to 0 of 40. The redactor runs before anything sends or logs, and it has to see text the way the model will. It parses the request, validates it against an explicit schema, redacts the decoded string in every field it knows system prompt, message text, tool-call arguments, tool results , rejects any field or part type it does not know, and only then serializes. Rebuild the request from allowed fields. Copying the input with { ...body } and overwriting the scanned fields forwards whatever the walker skipped, such as a top-level system string or tool calls .function.arguments. Regexes run over the serialized JSON fail twice in the demo. JSON writes a newline as the two characters \n, so the next key follows the word character n and the \b anchor before DB PASSWORD never matches. And \S+ after SERVICE TOKEN= runs past the closing quote of the system string, since compact JSON has no space there, and swallows the start of messages. The canary survives and the result is no longer valid JSON. Each value gets a session-stable placeholder. Both copies of the password become the same token, so the model can still tell they are one value and say "change it in both places". Random tokens per occurrence lose that link, and deleting the value invites "your password is empty". Replacement is by position, so TOKEN=TOKEN cannot end up with the key swapped and the credential intact. // labs/ai-9/redact.mjs excerpt . Group 1 is the context, group 2 the value. const PATTERNS = { cls: 'SECRET', re: / \b A-Z0-9 ?:PASSWORD|SECRET|TOKEN|API KEY A-Z0-9 = \S+ /g }, // KEY=value { cls: 'SECRET', re: / \/\/ ^:/\s +: ^@\s + @/g }, // user:pass@host { cls: 'EMAIL', re: / A-Za-z0-9. %+- +@ A-Za-z0-9- +\. A-Za-z. {2,} /g }, { cls: 'PHONE', re: / \+?\d{1,3} -. \d{3} -. \d{4} \b/g }, ; const tokenFor = cls, value = { let e = byValue.get byToken.get value ?? value ; // an issued token resolves to its entry if e { counts cls = counts cls ?? 0 + 1; e = { token: <${cls} ${tag} ${counts cls } , cls }; // tag: 16 random bytes per session byValue.set value, e ; byToken.set e.token, value ; } else if RANK cls RANK e.cls { e.cls = cls; // promote; the spelling stays } return e.token; }; const redactText = text = { let out = text; for const { cls, re } of PATTERNS { out = out.replace re, whole, pre, value = pre + tokenFor cls, value + whole.slice pre.length + value.length ; // by position } return out; }; Run on a debug request whose decoded first message holds the config the random tag printed as TAG . Placeholders the session issued are removed before the entropy scan, because their random tag would trip it too: Help me debug the config. DB PASSWORD=<SECRET TAG 2 DATABASE URL=postgres://shop app:<SECRET TAG 2 @db.internal:5433/shop TOKEN=<SECRET TAG 3 - If the database is down, call the on-call phone <PHONE TAG 1 . DB PASS=hunter-fake-22 {"password": "json-fake-pw"} backup line: 415 555-0199 payments key in prose: q8Zr2LwT9xVb4NcK7mPd3HsY6fJ1 canary in outgoing body: false | placeholders in local map: 6 regex over serialized JSON: canary left = true | valid JSON = false entropy detector flags: 'q8Zr2LwT9xVb4NcK7mPd3HsY6fJ1 4.81 bits/char ' The checkpoint swaps both fake values for placeholders before the log and the provider. What four patterns miss { limits} The last four lines of that output are formats the patterns were never written for, and all four reached the outgoing request: a key without PASSWORD in its name, a JSON field, a bracketed phone and a bare random token. Production redaction stacks detectors that fail differently: Detector Catches Blind spot Exact token Your own known secrets and planted canaries Unknown values; base64, percent-encoded or split copies unless you decode first Shape regex Labelled values, URL userinfo, fixed prefixes such as ghp and AKIA Free-form passwords, unfamiliar layouts Entropy Random keys no rule names: 28 distinct characters give log2 28 = 4.81 bits per character Hex entropy is at most log2 16 = 4.0 bits per character, so this demo's strict 4.0 test cannot flag hex-only tokens; short tokens Validators, NER Checksums, phone rules, names in prose Added latency; models miss and over-match Entropy also flags harmless random strings, so a hit alone does not prove a value is a credential. Gitleaks pairs regex rules with keywords and per-rule entropy, TruffleHog can check whether a found key is live, and Presidio adds context words and a named-entity model for personal data in prose. Keep a corpus of fake secrets in many formats, run the redactor over it in CI, and fail the build when recall drops. Replies need the same scan before they are logged or displayed. With streamed replies AI 2 , buffer the complete response for these detectors; NOTES.md explains why a fixed holdback is not enough. Keep the value map local to the session. Restore personal data only for its owner into an approved display destination. Secret restoration needs a separate authorized operation; expanding a placeholder inside a tool call can send the value to an attacker through prompt injection. The example uses a single-use write grant and assumes a directory whose parents only you can modify. Random placeholder tags prevent accidental collisions and provide no authorization. The example keeps the map in process memory. Production code must add session expiry, reject operations after expiry and clear the maps when the session ends. The full restore rules are in labs/ai-9/NOTES.md. Scope comes before redaction: keep .env files, keys and credential stores out of the context and out of an agent's readable paths agent file permissions, AI 8 . Put the redactor at the last point before the network, at the client for secrets it knows exactly and at an egress gateway as a floor for every app, and make the gateway's own logs record metadata only. Four regexes are enough for a demo; with real data, watch recall on your own formats. Option Good for Costs you No redaction, check replies only Nothing Sent when included; logged where body capture is enabled. Lab: 40 of 40 prompts while replies showed 22 Scope the context Removes the secret at the source You must know which files a task needs; the facts check fell to 0 of 40 without them Client-side redaction Plaintext stays on the device; exact match on the user's own secrets Every client carries and updates detectors Egress gateway redaction One policy for every app, enforced by network rules The gateway sees plaintext and holds the restore map; it must redact before its own logging If a real credential reached an unintended recipient, revoke it promptly, issue and deploy a replacement, and review its usage during the exposure window OWASP remediation . Purging your own logs cannot recall a copy that already left. Gateway sizing and the answer-count statistics are in labs/ai-9/NOTES.md. Every value in labs/ai-9 is fake. The numbers show where a secret goes once it is in a prompt, not the behaviour of any company's model, product or private systems. Read next: how a cloud coding session can use an API key it never sees, with network secrets in Claude Code cloud environments. Key Takeaways - •In this API example, the request body carries the prompt messages alongside configuration fields. - •This story was reported by Dev.to , covering developments in the dev space. - •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage. 📖 Continue reading the full article: Read Full Article on Dev.to → https://dev.to/systemdesignsimple/secrets-in-ai-prompts-did-the-ai-leak-my-password-2646