The full design of Declaw's credential vault: how a secret stays out of the sandbox, gets attached at the egress proxy on the way out, and why a fully compromised, prompt-injected agent still leaks nothing but a placeholder.
An AI agent is only useful once you give it secrets — an LLM API key, a database password, a token for an internal service. The usual way to hand one over is an environment variable in the sandbox where the agent runs. That's also the problem: the sandbox is exactly where untrusted, model-driven code executes. A prompt-injected agent — or a compromised dependency — can read its own environment, dump /proc
, walk the process table, and your key walks out with it. You've handed the keys to the one component you've already decided not to trust, and sandboxing doesn't change that: a good sandbox bounds the blast radius of the code, but the secret is sitting inside that radius with it.
We covered the practical side — store a secret, scope it, call an API from an agent that never holds the key — in How to Give an AI Agent Code Execution Without Handing Over Your Credentials. This post is the other half: the whole design, end to end — store, injection, isolation — and why a fully compromised agent leaks nothing but a placeholder.
So we stopped trying to find a better hiding spot inside the box, and made the secret never enter the box at all.
The shape #
Three moving parts:
- The secret value lives server-side in a vault (we use OpenBao). It's write-only: you store it once, and no API ever returns it again — not to you, not to the agent.
- The sandbox is created with a reference — just the secret's name. Inside the VM, the environment variable holds a placeholder string,
declaw:vault-managed
. - Every sandbox's outbound traffic already routes through an egress proxy. When the agent makes a request to a host you've scoped the secret to, the proxy attaches the real credential — after the request has left the VM.
The agent uses the credential against the destination you intend, and never holds it. A prompt-injected agent that dumps its whole environment leaks a placeholder.
What makes this a real boundary and not a demo is where the proxy runs. The agent runs in a Firecracker microVM. The proxy runs in that sandbox's host-side network namespace — outside the guest, across the virtualization boundary — and it is the VM's only route to the network. The secret lives in the proxy's memory for exactly one request and is never written into the guest: not a file, not an env var, not a log line.
How the VM trusts the proxy's certs #
The first question any engineer asks: if the proxy terminates TLS to inject into an HTTPS request, why doesn't the agent's client reject the substituted certificate?
Each sandbox gets its own ephemeral CA — a per-sandbox ECDSA P-256 key whose private key never leaves the proxy's memory and is never written to disk. For each upstream host the agent contacts, the proxy mints a short-lived leaf signed by that CA.
For the guest to accept those leaves, the CA's public cert is installed into the guest's trust stores at boot — appended to the system bundle and exported through the variables every major runtime honors:
SSL_CERT_FILE → /etc/ssl/certs/ca-certificates.crt
REQUESTS_CA_BUNDLE → /etc/ssl/certs/ca-certificates.crt
CURL_CA_BUNDLE → /etc/ssl/certs/ca-certificates.crt
NODE_EXTRA_CA_CERTS → /usr/local/share/ca-certificates/declaw-sandbox.crt
So OpenSSL, Python requests, curl, and Node all chain-validate the proxy's leaf with no per-app configuration. The CA is per-sandbox, so one sandbox's CA is worthless against another's traffic.
One limitation falls out of the same mechanism: a client that pins a specific upstream certificate — rather than trusting the system store — will reject the proxy's minted leaf, so those endpoints can't be brokered. It's rare in server-to-server API calls, but it's real.
Injection fires on domain match, not on the env var. The placeholder is a courtesy so the agent's SDK has a value to read; the proxy injects into any request to a scoped host, whether or not the agent ever touched the variable — and it uses Set, so it also overrides any header the agent tried to supply itself.
Scoping, and why it's anchored #
A secret is attached only to hosts that match one of its scopes. A scope's domain pattern is an exact host, a *.
-wildcard, or a ~
-prefixed regex. Matching runs on RE2 — linear-time, no catastrophic backtracking — and every scope's pattern is anchored to ^(?:…)$
, so a credential scoped to api.example.com
can't be coaxed onto api.example.com.evil.com
. Anchoring is enforced rather than advisory: the pattern is wrapped (so even a top-level alternation a|b
anchors correctly as ^(?:a|b)$
), and it applies to hand-written custom scopes, not just the built-in presets.
Storing the secret #
Values go into OpenBao under a KV-v2 mount, one namespace per team — tenant isolation at the store, not just in a query filter. Our own Postgres holds only metadata and the per-domain injection rules; there is no value column anywhere in it. If the metadata write fails after the vault write, we roll the vault write back, so a half-created secret never leaves an orphan.
You don't even have to store the secret with us. A secret can instead be a connector descriptor — a small JSON pointer into a store you already run:
{"provider": "aws_sm", "secret_id": "prod/openai", "region": "us-east-1"}
On a cache miss the worker reads the live value from your store, uses it for that one request, and never persists it. Eleven stores are wired: AWS Secrets Manager and SSM Parameter Store, GCP Secret Manager, Azure Key Vault, HashiCorp Vault / OpenBao, Infisical, Doppler, Kubernetes Secrets, 1Password Connect, CyberArk Conjur, and Akeyless. The connectors reuse the same credential-minting machinery as the brokered injection types below, so a connector can authenticate to your store with workload identity instead of a stored key.
For the common case there's a preset catalog (~39 providers): you pick the provider and paste the key, and the destination host, injection format, and provider quirks are filled in for you — the version header one API requires, the Token vs Key scheme word another wants, the fixed basic-auth username a third pins. Presets expand server-side into ordinary scopes, so they're sugar over the same machinery, not a separate code path.
Attaching the credential #
The static types are the boring-but-essential ones: Authorization: Bearer <v>
, an arbitrary header, HTTP basic, or a query parameter — plus four affordances (a value prefix, a fixed basic-auth username, extra non-secret headers, static query params) so a single scope can express a provider's real contract without a new mechanism. All of it is applied in the proxy's request director, after the security scan and PII redaction have run, so the value never reaches the scanner or the audit trail.
The interesting types broker a short-lived credential at egress instead of forwarding a stored one:
AWS SigV4. The agent's SDK signs with throwaway credentials; the proxy strips that signature and re-signs the request. The stored material can be static keys, an AssumeRole (short-lived creds, refreshed before expiry), or — the keyless path — AssumeRoleWithWebIdentity, where the call to STS is sent unsigned because the OIDC web-identity token is the authentication. No long-lived AWS keys stored anywhere.OIDC. The proxy mints a short-lived bearer worker-side, caches it per (team, secret), and re-mints before expiry. It supports client-credentials, RFC 8693 token-exchange (a projected service-account or CI OIDC token as the subject — e.g. federating into GCP Workload Identity), private_key_jwt, and jwt-bearer, discovering the token endpoint from the issuer's.well-known/openid-configuration
. The client secret and the minted token both stay proxy-side; only the resultingAuthorization: Bearer
reaches upstream.HMAC. Request signing with a configurable digest (sha1/256/512), encoding (hex/base64), signing-string template over{body}{method}{path}{timestamp}
, prefix, and header layout — the GitHub, Stripe, and Slack signature shapes all fall out of the same config.
The JOSE signing behind private_key_jwt (RS256/PS256/ES256) is implemented on the Go standard library. Which brings me to the part I like most.
Passwordless databases #
The agent can talk to Postgres, MySQL, MongoDB, Redis, or an SMTP relay with no password at all. It opens a normal connection; the proxy performs the authentication handshake upstream on its behalf, then steps aside and pipes bytes. The agent gets a working, authenticated session and never holds the password.
That meant implementing the auth side of each wire protocol, on the standard library:
Postgres— read the client's startup message, decline its SSLRequest withN
to keep the near leg plain, authenticate upstream with cleartext / MD5 / SCRAM-SHA-256 (full SCRAM, server signature verified), then synthesize AuthenticationOk to the client and replay the server's post-auth messages through ReadyForQuery.MySQL— clear the CLIENT_SSL capability bit in the server greeting (MySQL has no "decline TLS" reply), discard the agent's auth response, and send our own: mysql_native_password, caching_sha2_password fast-auth, and the full-auth path on a cache miss — request the server's RSA public key, XOR the password with the scramble, and RSA-OAEP encrypt it.MongoDB— SCRAM-SHA-256 over OP_MSG, with just enough hand-rolled BSON to carry the SASL conversation. No driver.** Redis**— a RESP AUTH (with ACL username if present) ahead of the agent's traffic.** SMTP**— EHLO, upgrade the upstream leg with STARTTLS, AUTH PLAIN, then return the capability list to the client with AUTH and STARTTLS stripped so it proceeds on the already-authenticated session.
For a database that mandates TLS on the wire, each broker can opt into upstream TLS — SMTP via STARTTLS, Redis and MongoDB at connect time, and Postgres and MySQL through their in-protocol negotiation. The agent↔proxy leg stays cleartext inside the sandbox's network namespace, where the proxy is the only gateway.
The SCRAM engine is shared between Postgres and Mongo and built on crypto/hmac
crypto/sha256
— SCRAM's Hi() is just a single-block PBKDF2, so we didn't need a library for it. The whole brokering layer carries no golang.org/x/crypto
import; same for the JWS signer.
Multiple secrets, one request #
Scopes accumulate. Every scope whose domain matches the request injects, so a gateway that needs both its own key and the upstream provider's key — both scoped to the same host — gets both on a single request. Scopes apply in delivery order; if two write the same header, last writer wins. That's deliberate: it's what lets a provider-quirk preset layer a required non-secret header over the credential with no special case.
When things fail #
The failure modes are explicit, and they fail safe:
HTTP injection fails open-to-401. If a fetch, mint, or signature errors, the proxy skips that scope and forwards the request without the credential — the upstream rejects it with a 401. It never substitutes a fallback or leaks a partial value.Socket brokering fails hard-closed. If a database handshake errors, the connection is dropped rather than handed to the agent half-authenticated.Revocation is a TTL. Fetched values are cached worker-side for a configurable window (default 60s). Rotate the stored value — or rotate it in your own store, for a connector — and running sandboxes pick it up on their next request. Set the window to zero and every request re-fetches: an instant kill switch. (Re-scoping a secret — a new destination or injection format — is an in-place update too, no delete-and-recreate.)
What you can see #
Every injection emits a masked audit event — which env var, which injection type, the target header, the destination host, and a count — and never the value. You get a precise record of what credential was attached to which destination and when, with the secret appearing in no log.
The one-line version #
A fully compromised, prompt-injected agent that tries to steal "its" API key ends up exfiltrating the string declaw:vault-managed
. The credential was never in the box to take — it was attached on the way out, to the destination you sanctioned, by a process the agent can't read across the VM boundary.
If you'd rather verify that than take my word for it, one of the scenarios in Declaw Arena drops you into a root shell in a sandbox with a vault-backed key and dares you to steal it. You get root. You find declaw:vault-managed
.
That's the whole idea: let an agent act with a credential without ever trusting it to hold one.