{"slug": "show-hn-secret-guard-a-local-llm-gateway-keys-never-leave-your-machine", "title": "Show HN: Secret Guard – a local LLM gateway; keys never leave your machine", "summary": "Developer luochen1990 released Secret Guard, a local LLM gateway that replaces secrets in outbound agent requests with equal-length, same-charset mock values and restores the real credentials when responses return, so LLM vendors never see actual passwords, private keys, tokens or cookies. The tool requires only pointing an agent harness's API base URL at the local gateway, and supports per-secret strategies of redact/restore (default), \"reject\" (returns 503 when the secret appears) and \"placeholder\" (swaps in a never-restored [REDACTED:{id}] token that doubles as a canary). Version 0.1.0 ships as static musl binaries for Linux x86_64 and ARM64 with SHA256 checksums, plus a Nix flake, and a WebUI records each forward showing exactly what the LLM saw.", "body_md": "**A lightweight local LLM gateway: outbound requests have their Secrets replaced with\nequal-length realistic Mocks; responses are restored back to real values on the way in —\nthe LLM never touches a real Secret, while your Agent tools keep working.**\n\n[Docs](https://secret-guard.lambda.lc/en/) · [Download](https://secret-guard.lambda.lc/en/download/) · [Quick Start](https://secret-guard.lambda.lc/en/tutorial-quick-start/) · [Protocol Support](#protocol-support) · [简体中文](https://github.com/luochen1990/secret-guard/blob/master/README.zh-CN.md)\n\nThe local Agent Harness (claude-code / opencode / hermes-agent and friends) does your work by stuffing material wholesale into the prompt — code, configuration, terminal output, along with the passwords, private keys, tokens, and cookies in them, all delivered to the LLM vendor's servers. Once that data leaves your machine, it is out of your control: it may end up in LLM vendor logs, be used for model training, or leak in a breach.\n\nAnd \"never put Secrets in the context\" is simply not viable in Agent workflows —\ncredentials are exactly what an Agent tool needs to act on real systems on your behalf.\nSecret Guard resolves this tension: **your Agent tools keep working with your Secrets,\nwhile the LLM never touches a real value.**\n\nThe only integration cost is one line: point your Agent tool's API base URL at the local gateway, and traffic flows through it to the LLM vendor:\n\n```\nOutbound:  Agent tool ──real Secret──►  secret-guard  ──realistic Mock──►  LLM vendor\nReturn:    local tools ◄──real Secret──  secret-guard  ◄──Mock reply────  LLM vendor\n                        (Redact / Restore both happen locally, transparent in both directions)\n```\n\n- **Outbound Redact** : Secrets in requests bound for the LLM are replaced with\nequal-length, same-charset realistic Mocks (default strategy) — the LLM keeps working\nwith them, none the wiser.\n- **Return Restore** : when LLM responses come back (tool-call arguments included), Mocks\nare restored to real values — a curl command carrying a Secret lands on your machine\nwith authentication passing as usual.\n- **Per-secret strategy** : redact/restore above is the default; a secret can instead be\nconfigured`strategy = \"reject\"` to refuse forwarding (503) the moment it appears in a\nrequest — for credentials that must never leave the machine — or`strategy = \"placeholder\"` to swap it for an explicit`[REDACTED:{id}]` token that is never\nrestored. Every placeholder secret doubles as a canary: if its real value ever shows\nup in an LLM response, you get an alert (WARN log + WebUI timeline badge, no secret\nvalues in the alert) while the response itself passes through untouched.\n- **Built-in protection for Secret Guard's own credentials** : the sg API tokens it\nissues (and the provider API keys you register with it) are protected automatically —\nzero configuration, effective the instant a token is created. If a prompt injection\ntalks your Agent into pasting its own gateway token into the conversation, the LLM\nvendor only ever sees an equal-length Mock.\n\nThe same request, before and after it leaves the machine (captured from a real run; only the token value is fictional):\n\n```\nAgent Harness sends:  GITHUB_TOKEN=ghp_0123456789abcdefghijklmnopqrstuvwxyz\nLLM vendor receives:  GITHUB_TOKEN=2_9rr_61dr1lmv2wf15aotyyt262l3t9_2pt4uih\n```\n\nA quick tour of the WebUI — session timeline, the LLM's-eye request view with the Mock highlighted, and the restored real value on the response side:\n\nEvery forward can be inspected in the WebUI's Records page (session timeline + request detail, with Secret locations highlighted as the Mock — i.e. exactly what the LLM saw):\n\n```\n# Binary archives (Linux x86_64 / ARM64, fully static musl, with SHA256 checksums):\n#   https://github.com/luochen1990/secret-guard/releases\n#   archive name pattern: secret-guard-<version>-<triple>.tar.gz\n#   (e.g. secret-guard-0.1.0-x86_64-linux-musl.tar.gz)\n\n# Nix (flake):\nnix run github:luochen1990/secret-guard -- --help\n# or wire up the overlay / nixosModule into your flake and use pkgs.secret-guard\n\n# Cargo (builds from source, requires rustc >= 1.96):\ncargo install --git https://github.com/luochen1990/secret-guard\n# or after cloning the repository: cargo install --path .\n```\n\nNixOS users should prefer the `services.secret-guard.*` structured-options deployment\n(config validated at eval time, credentials injected via files) — see\n[docs/deployment-nixos.md](https://github.com/luochen1990/secret-guard/blob/master/docs/deployment-nixos.md).\n\n**Optional authentication (OIDC login) is a cargo feature**: the default build ships\nwithout the OIDC login chain (smaller dependency graph). Binary archives and\n`cargo install` therefore default to single-user mode; setting `[auth] enabled = true`\nthere fails at startup with a \"rebuild with --features oidc\" hint. To get OIDC:\nbuild from source with `cargo install --path . --features oidc`, or use the NixOS\nmodule (its default package enables the feature). The local API-key path is always\ncompiled in. See [docs/configuration.md](https://github.com/luochen1990/secret-guard/blob/master/docs/configuration.md) `[auth]`.\n\nCreate `secret-guard.toml` somewhere (one upstream provider + one secret to protect is\nall it takes):\n\n```\n[[providers]]\nid = \"openai-main\"\nkind = \"direct\"                         # direct upstream (use \"router\" for routing endpoints, \"pool\" for plan pools)\napi_key = \"sk-your-upstream-key\"        # entry-level credential shared by all endpoints — must come BEFORE [[providers.endpoints]] (TOML sub-table switch)\n[[providers.endpoints]]\nprotocol = \"openai\"                     # at most one [[providers.endpoints]] per protocol;\nbase_url = \"https://api.openai.com\"     # unmatched ingress protocols fall back to the first endpoint\n\n[[secrets.entries]]\nid = \"my-github-token\"\nvalue = \"ghp_0123456789abcdefghijklmnopqrstuvwxyz\"   # the secret to protect (never sent upstream)\n```\n\nMind the nesting: secrets live under `[[secrets.entries]]` while providers are flat\n`[[providers]]` (the two styles differ). Full field reference:\n[docs/configuration.md](https://github.com/luochen1990/secret-guard/blob/master/docs/configuration.md).\n\n```\nsecret-guard run --port 18787\ncurl http://127.0.0.1:18787/o/openai-main/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"model\":\"gpt-4o-mini\",\"messages\":[{\"role\":\"user\",\"content\":\"My token is ghp_0123456789abcdefghijklmnopqrstuvwxyz, please summarize.\"}]}'\n```\n\nThe LLM vendor receives a realistic Mock — `redactions=1` in the forward-completion log is\nthe replacement count; open `http://127.0.0.1:18787/` in a browser to review the request\nin the Records page.\n\nExamples use port `18787` throughout; the default listen address is `127.0.0.1:8787`\n(changeable via `[server] port` or the `SG_PORT` environment variable).\n\nFor daily use, just point your Agent tool's `base_url` at the gateway — no curl needed:\n\n``` python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://127.0.0.1:18787/o/openai-main/v1\", api_key=\"ignored\")\n```\n\nURLs take the form `/{proto_short}/{provider_id}/*rest`, encoding both the **ingress\nprotocol** and the **target provider**:\n\n| proto_short | Protocol | SDK example ( `base_url` ) | \n|---|---|---|\n| `o` | OpenAI | `http://127.0.0.1:18787/o/<provider-id>/v1` | \n| `a` | Anthropic | `http://127.0.0.1:18787/a/<provider-id>` | \n| `g` | Gemini | `http://127.0.0.1:18787/g/<provider-id>` | \n| `l` | Ollama | `http://127.0.0.1:18787/l/<provider-id>` | \n| `r` | Responses (OpenAI Responses API) | `http://127.0.0.1:18787/r/<provider-id>/v1` | \n\nSDK notes: the OpenAI SDK needs the `/v1` prefix included yourself; the Anthropic SDK\nappends `/v1/messages` on its own, so `http://.../a/<provider-id>` is enough.\n\nThe real API key lives **only** in your local `secret-guard.toml` — the harness itself holds a\ndummy value. secret-guard injects the real key when forwarding upstream, so even a fully\ncompromised harness (or a breach of its cloud storage) never exposes your key.\n\n**opencode** — custom provider via the OpenAI-compatible route, in `opencode.json`:\n\n```\n{\n  \"$schema\": \"https://opencode.ai/config.json\",\n  \"provider\": {\n    \"secret-guard\": {\n      \"npm\": \"@ai-sdk/openai-compatible\",\n      \"name\": \"secret-guard (local)\",\n      \"options\": {\n        \"baseURL\": \"http://127.0.0.1:18787/o/<provider-id>/v1\",\n        \"apiKey\": \"dummy-key-harness-never-sees-the-real-one\"\n      },\n      \"models\": {\n        \"<model-id>\": { \"name\": \"<any display name>\" }\n      }\n    }\n  }\n}\n```\n\nModel IDs come from the upstream list — the gateway passes `GET /v1/models` straight\nthrough, so `curl http://127.0.0.1:18787/o/<provider-id>/v1/models` shows what you can use.\n\n**claude-code** — two env vars, zero config:\n\n```\nexport ANTHROPIC_BASE_URL=http://127.0.0.1:18787/a/<provider-id>\nexport ANTHROPIC_AUTH_TOKEN=dummy-key-harness-never-sees-the-real-one\n```\n\nCross-protocol translation is built in (e.g. an OpenAI-form `o` route to an Anthropic\nupstream, streaming included), so pick the route that matches your harness — not your\nupstream.\n\n- `secret-guard.toml` — declarative config, hand-written, read-only in the process.\n- `secret-guard.state.toml` — dynamic state the WebUI persists on edit; delete it to\nreset. A dynamic source can override the static one for the same entity.\n\n**Sensitive data warning**: `state.toml` contains **plaintext** secrets (WebUI-created\nsecret values, explicitly written provider api_keys) at the same sensitivity level as\n`secret-guard.toml` — keep it in `.gitignore`; an accidental commit leaks Secrets into\nversion history.\n\n⚠️ \n\nEvery completed forward logs a one-line INFO summary\n(`forward ... status=... elapsed_ms=... redactions=N provider=...`), so you can confirm\ntraffic passes through secret-guard from the command line; upstream failures (502/504)\ncarry readable causes in the error body. Lower the verbosity with\n`RUST_LOG=secret_guard=warn` (default `info`) when it gets noisy.\n\nHanding production traffic to an intermediary process requires proof it won't get in the way. The following properties are pinned in the repository by property-based tests and end-to-end integration tests, fully verified on every release:\n\n- **Transparent relay** : without Redact, byte-exact pass-through; with Redact, nothing\nchanges semantically except the real↔mock swap (only serialization noise like field\norder) — behavior is indistinguishable from a direct connection.\n- **Strictly reversible** : every Mock maps one-to-one to a real value, and Restore is the\nexact inverse of Redact — tool calls carrying Secrets are never corrupted.\n- **Deterministic Mocks** : a Secret's Mock is stable across turns, preserving the LLM\nvendor's prefix-cache hits — token cost is unaffected in normal sessions (the\nmock-escaped-and-replayed exception is documented).\n- **No collisions in context** : a Mock never coincides with other content in the request,\nso responses are never wrongly replaced.\n- **Multi-provider routing** : multiple providers (OpenAI / Anthropic / Gemini / Ollama /\nResponses) with per-request model-wildcard routing to different upstreams.\n- **Plan-pool failover (Pool)** : chain multiple coding-plan subscriptions (each with its\nown credential) behind one endpoint; when one plan hits its usage-window limit, traffic\nfails over to the next one automatically and switches back once the window resets —\nzero manual intervention.\n- **On-demand verbose logging** : by default the gateway stays memory-lean (the session\ntimeline keeps working); flip one toggle in the WebUI and subsequent requests capture\nfull request/response bodies for troubleshooting.\n\nA source-level comparison with LiteLLM / LLM Guard / PasteGuard — seven\nsecret-protection dimensions, including where the alternatives are stronger:\n[docs/comparison.md](https://github.com/luochen1990/secret-guard/blob/master/docs/comparison.md).\n\n| Protocol | Forwarding | Secret Redact | Cross-protocol translation | \n|---|---|---|---|\n| OpenAI (Chat Completions) | ✅ | ✅ | ✅ ⇄ Anthropic (incl. streaming) / ⇄ Responses (incl. streaming) | \n| Anthropic (claude) | ✅ | ✅ | ✅ ⇄ OpenAI (incl. streaming) / ⇄ Responses (incl. streaming) | \n| OpenAI Responses | ✅ | ✅ (incl. streaming) | ✅ ⇄ Chat Completions / Anthropic (incl. streaming) | \n| Gemini / Ollama | ✅ byte pass-through | 🚧 [Roadmap](#roadmap--contributing) | 🚧 [Roadmap](#roadmap--contributing) | \n\n- Gemini / Ollama are transparent byte pass-through (forwarding itself fully works);\nthe codec does not cover them yet, so Redact does not apply — with secrets configured,\nrequests on those protocols are **rejected by default** (503,`[redact] on_unsupported_protocol = \"fail_closed\"` ; the error message carries a hint).\nThey can still be created via hand-written config / the REST API (fine for secret-free\npure forwarding); the WebUI creation form does not offer them, but editing an existing\nentry or a Detect probe recommendation supports them via \"(experimental)\" options.\n- **Fail-safe degradation** : on Redact's abnormal paths (unparsable upstream responses /\nnon-2xx), Mocks are passed through by default — real values never land in failure\nbodies;`on_fallback_restore = \"restore\"` opts into restoration. Full semantics of the\nthree degradation switches:[docs/configuration.md](https://github.com/luochen1990/secret-guard/blob/master/docs/configuration.md) .\n- Protocol behavior is pinned by property-based tests and simulated-upstream integration\ntests; real-Provider field validation for Anthropic / Gemini / Ollama has not happened\nyet (see [Roadmap](#roadmap--contributing) ) — when first adopting those protocols,\nverify one real request in the WebUI Records page before going full speed.\n\nThe following directions are in progress — issues and PRs welcome at the main repository\n[github.com/luochen1990/secret-guard](https://github.com/luochen1990/secret-guard);\nthe full development guide lives in [AGENTS.md](https://github.com/luochen1990/secret-guard/blob/master/AGENTS.md):\n\n- **Gemini / Ollama codec** : bring Redact and cross-protocol support to these families.\nA new protocol only needs a Reader + Writer trait implementation (~200 lines) with no\ndispatch changes — see`src/codec/AGENTS.md` for the architecture.\n- **Responses streaming fidelity** : close the residual known losses of Responses\nstreaming translation (hosted tools / refusal parts dropped with WARN, reasoning\ndelta event-type normalization, multi content part folding) — streaming itself,\nincluding the Redact restore path, now works for every codec-covered pair; tracked\nin[docs/known-limitations.md](https://github.com/luochen1990/secret-guard/blob/master/docs/known-limitations.md) .\n- **Unknown-secret detection (NER / regex / entropy) as an alerting layer** : flag\nsuspected undeclared secrets in the WebUI without auto-redacting (zero false-positive\nreplacements stay guaranteed). See[#257](https://git.lambda.lc/lc-studio/secret-guard/issues/257) .\n- **Real-Provider field validation** : integration-test profiles against real upstreams\nfor the protocol matrix.\n\nYou might be looking for:\n\n- Your first Redact-protected request in three minutes → [Quick Start (site)](https://secret-guard.lambda.lc/en/tutorial-quick-start/)\n- Full configuration field reference → [docs/configuration.md](https://github.com/luochen1990/secret-guard/blob/master/docs/configuration.md)\n- WebUI page-by-page guide → [WebUI Guide (site)](https://secret-guard.lambda.lc/en/manual-webui/)\n- NixOS deployment and credential injection → [docs/deployment-nixos.md](https://github.com/luochen1990/secret-guard/blob/master/docs/deployment-nixos.md)\n- Architecture and dataflow contracts → [docs/design/](https://github.com/luochen1990/secret-guard/blob/master/docs/design) (contracts.md)\n- Source-level comparison with alternatives → [docs/comparison.md](https://github.com/luochen1990/secret-guard/blob/master/docs/comparison.md)\n- Development workflow / test strategy / module contracts → [AGENTS.md](https://github.com/luochen1990/secret-guard/blob/master/AGENTS.md)\n\nMIT — see [LICENSE](https://github.com/luochen1990/secret-guard/blob/master/LICENSE).", "url": "https://wpnews.pro/news/show-hn-secret-guard-a-local-llm-gateway-keys-never-leave-your-machine", "canonical_source": "https://github.com/luochen1990/secret-guard", "published_at": "2026-10-18 15:24:48+00:00", "updated_at": "2026-10-18 15:48:07.472661+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-tools", "large-language-models", "developer-tools"], "entities": ["Secret Guard", "luochen1990", "GitHub", "claude-code", "opencode", "hermes-agent", "Nix"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-secret-guard-a-local-llm-gateway-keys-never-leave-your-machine", "markdown": "https://wpnews.pro/news/show-hn-secret-guard-a-local-llm-gateway-keys-never-leave-your-machine.md", "text": "https://wpnews.pro/news/show-hn-secret-guard-a-local-llm-gateway-keys-never-leave-your-machine.txt", "jsonld": "https://wpnews.pro/news/show-hn-secret-guard-a-local-llm-gateway-keys-never-leave-your-machine.jsonld"}}