{"slug": "show-hn-aiproxy-redact-pii-locally-before-it-reaches-claude-openai", "title": "Show HN: Aiproxy – Redact PII locally before it reaches Claude/OpenAI", "summary": "A developer released aiproxy, a free, open-source anonymizing reverse proxy that scrubs people and company names and other sensitive content with a local LLM before forwarding requests to Anthropic's Claude or OpenAI, then restores the originals in the reply. The tool keeps originals and tokens in a local encrypted store, ships four processors (secrets_regex, anonymizer local-LLM NER, image_redactor, logger), handles text, images and PDFs, and offers a /dryrun endpoint to preview redactions; it requires Python 3.11+, Ollama, 6 GB free RAM for CPU-only NER and about 4 GB of disk for the 3B NER model. The author warns that detection is not guaranteed, since a local LLM can miss an entity and a real value can still reach the upstream provider, and advises running in fail_closed mode and treating the mapping store as a secret.", "body_md": "aiproxy is free and keeps your data on your machine. If it saves you from\nleaking PII to an LLM provider, **[buy me a coffee ☕](https://buymeacoffee.com/cucoo)** —\nit funds the time that keeps the project maintained.\n\nAnonymizing reverse proxy for LLM APIs. Point Claude Code (or any AI tool)\nat aiproxy; it scrubs people/company names and other sensitive content with\na **local** LLM before forwarding to Anthropic, then restores the originals\nin the reply. Pluggable filter/processor interface. Handles text, images,\nPDFs.\n\n```\nAI tool ──▶ aiproxy ──(redacted)──▶ Anthropic\n   ▲            │                       │\n   └─(restored)─┴───────(tokens)────────┘\n```\n\n- Reversible anonymization: originals ↔ tokens kept in a local encrypted store.\n- One `Processor` interface — drop in your own filters (or reviewed[plugins](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/WRITING_A_PROCESSOR.md) ).\n- Ships: `secrets_regex` ,`anonymizer` (local-LLM NER),`image_redactor` ,`logger` .\n- `/dryrun` endpoint to preview redactions before trusting it.\n- Scales to a shared org service: per-user identity + isolation, self-service\nterm lists (`/terms` ), SQLite**or** Postgres store, config hot-reload\n(`SIGHUP` ), secrets-manager key sourcing,`/readyz` . See[`docs/ORG_DEPLOYMENT.md`](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/ORG_DEPLOYMENT.md) .\n\nFull install + usage: [`docs/MANUAL.md`](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/MANUAL.md).\n\n**Use at your own risk — no warranty.** aiproxy is provided *as is*,\nwithout warranty of any kind (see the **No Liability** clause in\n[`LICENSE`](https://github.com/cucooOnGithub/aiproxy/blob/master/LICENSE)). Detection is **not guaranteed**: a local LLM can miss\nan entity, so a real value can still reach the upstream provider. You are\nresponsible for validating redaction for your own data and for any\nobligations you have to the people whose data you process. Run it in\n`fail_closed` mode, verify with `/dryrun`, and treat the mapping store as a\nsecret. See the [threat model](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/MANUAL.md#threat-model).\n\n⚠️ \n\nThe pipeline redacts on the way out and restores on the way in; the upstream API only ever sees tokens. Both directions share the encrypted store and the local NER model.\n\n``` php\nflowchart LR\n  C[\"AI client\"] --> I[\"Ingress · identity<br/>resolve user_id<br/>session = user/raw\"]\n  I --> P\n  subgraph P[\"Pipeline\"]\n    direction TB\n    L[\"logger\"] --> S[\"secrets_regex\"] --> A[\"anonymizer · NER → ⟦TOKEN⟧\"] --> IR[\"image_redactor\"]\n  end\n  P -- \"redacted + tokens\" --> U[\"Upstream LLM<br/>(sees tokens only)\"]\n  U -- \"tokens\" --> P\n  P -- \"originals restored\" --> C\n  A <-- \"detect\" --> N[\"Local NER (Ollama)\"]\n  P <-- \"put / token_for · cache · terms\" --> ST[(\"Store · encrypted<br/>sqlite | postgres\")]\n  K[\"Key source<br/>env | file | command\"] -. \"at-rest key\" .-> ST\n```\n\nScaled to a shared org service it runs stateless frontends behind a load\nbalancer over a shared Postgres store and NER service — diagrams and design in\n[`docs/MANUAL.md#1b-architecture`](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/MANUAL.md#1b-architecture) and\n[`docs/ORG_DEPLOYMENT.md`](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/ORG_DEPLOYMENT.md).\n\nFirst get the code (needs `git` and Python ≥ 3.11):\n\n```\ngit clone https://github.com/cucooOnGithub/aiproxy.git aiproxy\ncd aiproxy\n```\n\nThen pick a deployment shape. Full detail for each is in the manual:\n[single workstation](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/MANUAL.md#2-install--single-workstation) ·\n[organization](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/MANUAL.md#2b-install--organization).\n\nOne person, one machine; local SQLite store and local NER model.\n\n**Requirements**\n\n|  | Minimum | Comfortable | \n|---|---|---|\n| OS | macOS, Linux, or Windows 10+ | same | \n| Python | 3.11+ | 3.11+ | \n| RAM free | 6 GB (CPU-only NER) | 8 GB+ | \n| GPU | none — runs on CPU | any GPU with ≥ 4 GB VRAM (much faster NER + pre-warm) | \n| Disk | ~4 GB (deps + the 3B NER model) | SSD | \n| Also installed | Ollama (local LLM runtime) and the Tesseract OCR binary — the installer handles both |  | \n\nThe default NER model is `qwen2.5:3b-instruct` (~2 GB). OCR runs on CPU.\n\n**One-command install** — sets up the venv + deps, Ollama (and starts it),\nTesseract, the NER model, a store key, and `config.yaml`:\n\n```\n./install.sh                                           # macOS / Linux\npowershell -ExecutionPolicy Bypass -File .\\install.ps1 # Windows\n```\n\nThen verify and run:\n\n```\npython -m aiproxy doctor --config config.yaml   # check everything is set up\npython -m aiproxy serve  --config config.yaml\nexport ANTHROPIC_BASE_URL=http://127.0.0.1:8080  # match listen.port\n```\n\nUp and running? If aiproxy is now keeping your data private, consider\n[**buying me a coffee ☕**](https://buymeacoffee.com/cucoo) to support it.\n\n## Manual quick start (skip the installer)\n\n```\npython -m venv .venv && source .venv/bin/activate\npip install -e \".[media]\"                    # core; needs the tesseract binary\nollama pull qwen2.5:3b-instruct\ncp config.example.yaml config.yaml\nexport AIPROXY_STORE_KEY=\"$(python -m aiproxy genkey)\"   # if store.encrypt: true\npython -m aiproxy serve --config config.yaml\n```\n\n## Build a standalone binary (optional)\n\n``` php\n./build.sh                                        # -> dist/aiproxy  (macOS/Linux)\npowershell -ExecutionPolicy Bypass -File .\\build.ps1   # -> dist\\aiproxy.exe (Windows)\n./dist/aiproxy serve --config config.yaml\n```\n\nPyInstaller can't cross-compile — build on each target OS. Tesseract and\nOllama are **not** bundled. Details in\n[docs/MANUAL.md](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/MANUAL.md#build-a-standalone-binary-optional).\n\nMany users behind one service: central config owned by IT/security, per-user\nisolation, SQLite **or** Postgres, stateless frontends behind a load balancer.\nFull rollout and config in\n[docs/ORG_DEPLOYMENT.md](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/ORG_DEPLOYMENT.md) and\n[the manual](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/MANUAL.md#2b-install--organization).\n\n**Improving the local LLM at org scale.** On a workstation the NER model is\nkept small so it loads fast per request; a shared deployment removes that\nconstraint and can run a far more accurate model:\n\n- **Promote NER to a dedicated warm GPU host** shared by every replica — the\nmodel is loaded once and stays resident, so bigger models become viable.\n- **Use a larger model or an ensemble** for higher recall:`llm.models: [qwen2.5:7b-instruct, llama3.1:8b]` (union of detections;\nlatency is the sum, so only practical when warm and shared). Budget\n~8–16 GB VRAM for a 7–8B model, more for an ensemble.\n- **Keep it resident** with`llm.keep_alive` so no request pays a cold load.\n- **Tune for your data** : expand`llm.entity_types` , adjust the NER`prompt` ,\nand grow the`anonymizer.name_gazetteer` from your directory (HR/AD) for\ndeterministic recall the model alone would miss.\n\nDevelopers: [`docs/DEVELOPING.md`](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/DEVELOPING.md) explains the\narchitecture, the `Document`/` Processor`/` Store` abstractions, how to make\ncommon changes, the SemVer/tag release flow, and the PR workflow and checklist.\nShipping a custom redaction filter without touching core?\n[`docs/WRITING_A_PROCESSOR.md`](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/WRITING_A_PROCESSOR.md).\n\nNot a coder? You can still help the project a lot by\n[**buying me a coffee ☕**](https://buymeacoffee.com/cucoo) or sharing it.\n\nWorking, 97 passing tests. Not yet smoke-tested against live providers except the local echo round-trip below.\n\n- **Providers** : Anthropic (`/v1/messages` ) and OpenAI\n(`/v1/chat/completions` ), streaming and non-streaming (SSE tokens restored\neven when split across chunks). Transparent auth passthrough — the proxy\nholds no key; it forwards the caller's`x-api-key` /`Authorization` bearer, so a Claude subscription works unchanged.\n- **Anonymizer** : local-LLM NER via Ollama, reversible tokens, encrypted\nSQLite store. Configurable model, prompt, and entity types. Ensemble\n(`MultiNER` , union of several models) or a single model;`keep_alive` ,\ncache TTL, and startup pre-warm for VRAM-tight boxes. Deterministic\nbackstops (recall doesn't depend on the LLM):`allowlist` ;`denylist` (irreversible`[REDACTED]` );`always_tokenize` (known strings masked**reversibly** , hidden upstream but restored in Claude's output — e.g. a\nclient name that must still appear in generated documents); a name gazetteer\n(`anonymizer.name_gazetteer` ) that joins known name-parts across spaces,\ndots, underscores and hyphens, catching`john.rivera` and`J. Rivera` even inside a path; and a**known-value sweep** that re-masks any value\ntokenized once in a session across every later block, so a value the NER\ncatches inconsistently never leaks later.\n- **Archive/cache** : optional`archive.dir` keeps redacted documents on disk\n(named by a hash of the original, deduped) and doubles as a document cache\nthat skips re-running the LLM/OCR on a re-presented file.\n- **Documents** : images (OCR→mask), PDFs (true text redaction + embedded\nimages + scanned-page rasterize), and OOXML Office`.docx` /`.pptx` /`.xlsx` (body text tokenized reversibly +`*/media/` images masked).\n- **Scope** : only genuine user messages and attached documents are redacted.\nClaude's own operational content — the Claude Code system prompt, CLAUDE.md,\nskills, slash-command text, system reminders, local command output — reaches\nthe model verbatim. Tunable via`scope:` (`redact_system` ,`skip_tags` ).\n- **Caching** : in-memory NER cache (TTL) + persistent per-document\nredaction cache (re-presented docs skip re-work).\n- **Verified live** : local echo upstream — real name/email/company went\nupstream as tokens and came back restored to the client.\n- **Not done** : Office text split across runs only masked within one node;\nOffice over the Anthropic API needs a client that sends the raw OOXML\nbytes.\n\n```\naiproxy/proxy/        server, provider adapters (Anthropic/OpenAI), SSE streaming\naiproxy/pipeline/     Processor interface + runner + registry\naiproxy/processors/   anonymizer, secrets_regex, image_redactor, logger\naiproxy/store/        sqlite store (mappings + redaction cache) + at-rest encryption\naiproxy/llm/          Ollama NER client, MultiNER ensemble, build_ner\naiproxy/config/       YAML loader (+ AIPROXY_UPSTREAM_* env overrides)\ndocs/MANUAL.md        install + usage + extending\n```\n\nHide confidential data (customer PII, client/NDA names, secrets, IP) from\nhosted LLM providers while still using AI tools. Business and compliance\nrationale (GDPR/minimization, pseudonymization, honest limits): see\n[docs/BUSINESS_JUSTIFICATION.md](https://github.com/cucooOnGithub/aiproxy/blob/master/docs/BUSINESS_JUSTIFICATION.md).\n\naiproxy is built and maintained in the open, for free, by one person. There's no company behind it and no telemetry — your data never leaves your machine, so the project earns nothing from using it. If it's useful to you or your team, please consider chipping in:\n\nEven a single coffee is real encouragement and directly funds new filters, better name recall, and keeping the docs and releases current. If your company relies on it, a few coffees (or sharing the project) goes a long way. Thank you.\n\naiproxy is licensed under the **PolyForm Noncommercial License 1.0.0**\n(`LICENSE`): free to use, modify, and distribute for any **noncommercial**\npurpose. Copyright and all IP are retained by the authors, who reserve all\ncommercial rights — a separate commercial license is available on request.\nThird-party components keep their own permissive licenses\n(`THIRD_PARTY_LICENSES.md`); local model weights are licensed by their\npublishers and are the operator's responsibility (`NOTICE`).", "url": "https://wpnews.pro/news/show-hn-aiproxy-redact-pii-locally-before-it-reaches-claude-openai", "canonical_source": "https://github.com/cucooOnGithub/aiproxy", "published_at": "2026-11-11 12:15:32+00:00", "updated_at": "2026-11-11 12:50:54.736360+00:00", "lang": "en", "topics": ["ai-safety", "ai-tools", "large-language-models", "ai-agents"], "entities": ["aiproxy", "Anthropic", "Claude", "OpenAI", "Ollama", "Claude Code", "cucooOnGithub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-aiproxy-redact-pii-locally-before-it-reaches-claude-openai", "markdown": "https://wpnews.pro/news/show-hn-aiproxy-redact-pii-locally-before-it-reaches-claude-openai.md", "text": "https://wpnews.pro/news/show-hn-aiproxy-redact-pii-locally-before-it-reaches-claude-openai.txt", "jsonld": "https://wpnews.pro/news/show-hn-aiproxy-redact-pii-locally-before-it-reaches-claude-openai.jsonld"}}