{"slug": "redacting-personal-data-from-logs-and-llm-prompts-in-python-emails-cnics-cards", "title": "Redacting personal data from logs and LLM prompts in Python: emails, CNICs, cards, IBANs and phone numbers", "summary": "A developer published a tested Python redaction module that replaces emails, CNICs, IBANs, card numbers and phone numbers in logs and LLM prompts with typed placeholders like [EMAIL] and [CNIC] before the text is written or sent. Testing showed a broad phone-number pattern wrongly redacted an order number, so the fix requires international numbers to start with '+' and local numbers to match Pakistan's 03XX format, while card detection relies on a Luhn check that cuts but does not eliminate false positives. The developer recommends applying redaction in the logger rather than at every call site and ordering specific patterns before generic digit patterns.", "body_md": "Personal data spreads through **logs and LLM prompts** without anyone deciding to put it there: an error message includes the request body, a support summary pastes the customer's email, a debugging line prints a document number. Redact it **before** it's written or sent: match each kind of identifier and replace it with a **typed placeholder** such as `[EMAIL]` or `[CNIC]`, so the text stays readable and you can still see what kind of data was there. Two lessons from testing my own redactor: a **broad pattern produces false positives** (my phone-number pattern redacted an **order number**), and **card numbers need a Luhn check** to tell them apart from other long numbers. Here's the tested version and its limits.\n\n``` python title=\"redact.py\"\n\nimport re\n\ndef luhn_ok(digits: str) -> bool:\n\n    total = 0\n\n    for i, ch in enumerate(reversed(digits)):\n\n        d = int(ch)\n\n        if i % 2:\n\n            d = d * 2 - 9 if d > 4 else d * 2\n\n        total += d\n\n    return total % 10 == 0\n\nPATTERNS = [\n\n    (\"EMAIL\", re.compile(r\"[\\w.+-]+@[\\w-]+(?:.[\\w-]+)+\")),\n\n    (\"CNIC\", re.compile(r\"\\b\\d{5}-\\d{7}-\\d\\b\")),\n\n    (\"IBAN\", re.compile(r\"\\b[A-Z]{2}\\d{2}(?: ?[A-Z0-9]{4}){3,7}(?: ?[A-Z0-9]{1,4})?\\b\")),\n\n    (\"CARD\", re.compile(r\"\\b(?:\\d[ -]?){12,18}\\d\\b\")),\n\n    (\"PHONE\", re.compile(r\"+\\d{1,3}(?:[ -]?\\d){7,12}\\b|\\b03\\d{2}-?\\d{7}\\b\")),  # +international, or Pakistani 03XX\n\n]\n\ndef redact(text: str) -> str:\n\n    \"\"\"Replace personal data with typed placeholders. Card-like numbers are only redacted if they pass Luhn.\"\"\"\n\n    for label, pattern in PATTERNS:\n\n        def sub(m, label=label):\n\n            if label == \"CARD\" and not luhn_ok(re.sub(r\"\\D\", \"\", m.group())):\n\n                return m.group()\n\n            return f\"[{label}]\"\n\n        text = pattern.sub(sub, text)\n\n    return text\n\n```\n**Order matters.** The specific patterns (email, CNIC, IBAN) run before the generic digit patterns. Otherwise a CNIC such as `35202-1234567-1` could be partly consumed by the card or phone pattern, leaving fragments behind.\n\n## What the tests check\n\n| Input                                                    | Output                                  |\n| -------------------------------------------------------- | --------------------------------------- |\n| `Customer ali@example.com uploaded CNIC 35202-1234567-1` | `Customer [EMAIL] uploaded CNIC [CNIC]` |\n| `Card 4111 1111 1111 1111 declined`                      | `Card [CARD] declined`                  |\n| `Order 1234567812345678 shipped`                         | unchanged: fails Luhn, not a card       |\n| `Payout to GB82 WEST 1234 5698 7654 32 failed`           | `Payout to [IBAN] failed`               |\n| `Call +92 300 1234567 after 5pm`                         | `Call [PHONE] after 5pm`                |\n| `or 0300-1234567`                                        | `or [PHONE]`                            |\n| `Latency p95 was 1234 ms on 2026-11-20`                  | unchanged                               |\n| `Request 7f9c2a10-55e1-4c3b-9d1a-0b6f3e2c8a77 ok`        | unchanged: request IDs stay usable      |\n\nMy first phone pattern accepted any run of 10 to 16 digits with optional `+`, spaces and dashes. It redacted the **order number** as `[PHONE]`, which would have made the logs useless for exactly the debugging they exist for. The fix is to be specific: international numbers must start with `+`, and local numbers must match a known national format (here Pakistan's mobile `03XX-XXXXXXX`). Bare digit runs are left alone. That's a deliberate trade-off: a local number typed without a recognisable format gets through, and you add formats for the countries you serve. The Luhn rule works the same way for cards: one random number in ten passes Luhn, so it reduces false positives without eliminating them. The algorithm is explained in [Luhn and IBAN check digits](https://aliahmed.si/blog/luhn-iban-check-digits/).\n\n## Where should redaction happen?\n\n* **In the logger**, not at every call site. A logging filter or formatter that runs `redact()` on every message catches the lines people forget about. Redact structured fields by **name** too (`email`, `cnic`, `document_number`), which is more reliable than matching values.\n* **Before LLM calls.** Summarising a support ticket doesn't need the customer's card number. Redacting before the prompt also keeps the data out of the provider's logs, and makes prompt-injection leaks less damaging; see [canary tokens for detecting prompt leaks](https://aliahmed.si/blog/canary-tokens-system-prompt-leaks/).\n* **Not in the system of record.** The KYC case itself needs the real document number. Redaction is for copies, logs and prompts, not for the data the business actually has to process.\n\n## What regex can't catch\n\n* **Names and addresses.** \"Ali Ahmed, House 12, Street 4\" has no fixed shape. That needs a named-entity recognition model, or better, not logging free text at all.\n* **Digits in other scripts.** Python's `\\d` matches Urdu and Arabic digits (`۳۵۲۰۲`), but the same patterns in many other regex engines and log tools match only ASCII digits. Convert digits to ASCII before redacting, as in [validating a Pakistani CNIC number](https://aliahmed.si/blog/pakistan-cnic-number-validation/).\n* **Images and documents.** A photo of an ID card in a log bucket is personal data too.\n\nTreat the redactor as a safety net that **reduces** exposure, and keep the rule that personal data shouldn't be logged in the first place.\n\n## Summary\n\n* Redact logs and prompts with **typed placeholders**, so text stays readable.\n* Run specific patterns first; confirm card numbers with **Luhn**.\n* Make broad patterns (phone numbers) **specific**, or they eat IDs you need for debugging.\n* Redact in the logger and before LLM calls, not in the system of record.\n* Regex misses names, addresses and images: log less, as well as redacting.\n\nRelated: [KYC vs AML vs KYB](https://aliahmed.si/blog/kyc-vs-aml-vs-kyb/) and [request IDs in Node.js logs](https://aliahmed.si/blog/request-id-logging-asynclocalstorage/).\n\n---\n\nWritten by Ali Ahmed · [Original article](https://aliahmed.si/blog/redact-pii-logs-llm-prompts-python/)\n```\n\n", "url": "https://wpnews.pro/news/redacting-personal-data-from-logs-and-llm-prompts-in-python-emails-cnics-cards", "canonical_source": "https://dev.to/quantumbreakz/redacting-personal-data-from-logs-and-llm-prompts-in-python-emails-cnics-cards-ibans-and-phone-49p5", "published_at": "2026-11-20 14:36:33+00:00", "updated_at": "2026-11-20 14:45:22.941090+00:00", "lang": "en", "topics": ["ai-safety", "developer-tools", "large-language-models", "ai-agents"], "entities": ["Python"], "also_reported_by": [], "takeaway": "Run specific patterns (email, CNIC, IBAN) before generic digit patterns and gate card matches on a Luhn check, since a broad 10-16 digit phone pattern redacted an order number and broke log debugging.", "recommendation": {"advice": "Redact personal data in the logger with typed placeholders, ordering specific patterns before generic digit patterns and requiring a Luhn check for cards to avoid redacting order numbers.", "playbook": "agent-security", "playbook_url": "https://wpnews.pro/playbooks/agent-security"}, "alternates": {"html": "https://wpnews.pro/news/redacting-personal-data-from-logs-and-llm-prompts-in-python-emails-cnics-cards", "markdown": "https://wpnews.pro/news/redacting-personal-data-from-logs-and-llm-prompts-in-python-emails-cnics-cards.md", "text": "https://wpnews.pro/news/redacting-personal-data-from-logs-and-llm-prompts-in-python-emails-cnics-cards.txt", "jsonld": "https://wpnews.pro/news/redacting-personal-data-from-logs-and-llm-prompts-in-python-emails-cnics-cards.jsonld"}}