{"slug": "a-local-scrubber-for-text-you-re-about-to-send-to-an-llm", "title": "A local scrubber for text you're about to send to an LLM", "summary": "A new open-source tool called Redact, available on GitHub, strips sensitive data such as names, emails, phone numbers, SSNs, card numbers, addresses, API keys, and database passwords from text before it is sent to an LLM, running entirely locally with no LLM involved. The tool uses regexes, checksums, a small NER model, and a merge layer to detect and redact or pseudonymize values, with the original values kept in a separate file that never leaves the machine. It supports blanking values as <PERSON> or swapping in stable placeholders like PERSON_001, and includes 12 test fixtures covering logs, résumés, chat, source code, medical, financial, and adversarial cases.", "body_md": "**A local scrubber for text you're about to send to an LLM.**\n\nBefore you paste a log file, a résumé, or a support thread into an LLM, this\ntool strips out the sensitive parts first — names, emails, phone numbers,\nSSNs, card numbers, addresses, API keys, database passwords — and hands back a\nclean copy. It runs entirely on your machine with no LLM involved: a stack of\nregexes and checksums catches the structured stuff, a small NER model handles\nthe fuzzier things like people and companies, and a merge layer reconciles\nthem when they disagree. You can either blank each value out as `<PERSON>`\n\nor\nswap in stable placeholders like `PERSON_001`\n\nso the text still reads\ncoherently, with the original values kept in a separate file that never leaves\nyour machine. Most of the work went into *not* over-redacting — teaching it\nthat \"Django\" in a skills list is a framework, not a person.\n\nFor how the detection actually works, see [PIPELINE.md](/Sushmey/Redact/blob/main/PIPELINE.md).\n\n```\npython -m venv redact_venv\nsource redact_venv/bin/activate\npip install -r requirements.txt\npython -m spacy download en_core_web_sm\n```\n\nThe GLiNER model (`gliner_multi_pii-v1`\n\n, ~1.1 GB) downloads from Hugging Face\non first run and is cached.\n\n```\nsource env.sh          # thread-safety env vars — see the comments in the file\n\npython -m redact notes.txt                 # -> notes.redacted.txt\npython -m redact notes.txt -o clean.txt     # choose the output path\npython -m redact notes.txt --pseudonymize   # -> notes.redacted.txt + notes.redacted.txt.map.json\n```\n\n`--pseudonymize`\n\nalso writes `notes.redacted.txt.map.json`\n\n(label → original\nvalue). That file is sensitive — keep it local, never send it anywhere.\n\nAs a library:\n\n``` python\nfrom redact import redact_text\n\nsanitized, counts, mapping = redact_text(text, language=\"en\", pseudonymize=False)\n# counts  -> {\"EMAIL_ADDRESS\": 3, \"PERSON\": 5, ...}\n# mapping -> {} unless pseudonymize=True\npython run_tests.py\n```\n\n12 fixtures in `tests/`\n\n(logs, résumés, chat, source code, medical, financial,\nand adversarial \"same shape, different meaning\" cases). Each checks both\ndirections: sensitive values must be gone, ordinary values must survive\nunchanged.", "url": "https://wpnews.pro/news/a-local-scrubber-for-text-you-re-about-to-send-to-an-llm", "canonical_source": "https://github.com/Sushmey/Redact/tree/main", "published_at": "2026-08-29 17:29:10+00:00", "updated_at": "2026-08-29 17:48:45.199652+00:00", "lang": "en", "topics": ["ai-tools", "ai-safety"], "entities": ["Redact", "GitHub", "GLiNER", "Hugging Face", "Sushmey"], "alternates": {"html": "https://wpnews.pro/news/a-local-scrubber-for-text-you-re-about-to-send-to-an-llm", "markdown": "https://wpnews.pro/news/a-local-scrubber-for-text-you-re-about-to-send-to-an-llm.md", "text": "https://wpnews.pro/news/a-local-scrubber-for-text-you-re-about-to-send-to-an-llm.txt", "jsonld": "https://wpnews.pro/news/a-local-scrubber-for-text-you-re-about-to-send-to-an-llm.jsonld"}}