Audit Your AI Dev Tool's Data Boundary Before You Paste Real Code Into It

Last month I watched a teammate paste a stack trace into a hosted AI assistant. The trace contained an internal hostname, a database connection string, and a customer email. None of it was secret enough to trip a DLP rule, but all of it left our network through an endpoint nobody had audited. The failure wasn't the tool — it was that we had never written down which data classes are allowed to reach which inference endpoint, and we had no test that would fail when the boundary was crossed.

This article builds that boundary as a reproducible fixture: a data-classification decision matrix, a canary-leak test you can run against any hosted or self-hosted model endpoint, and a prevent/detect/recover table. The fixture works whether your endpoint is a cloud API, a free hosted tier, or a GPU box under your desk.

I1:A prompt containing data of classification levelLmay only egress to an endpoint whose trust level is explicitly approved forL.

Everything below exists to make I1 testable in CI rather than aspirational in a wiki.

Two rules make this matrix enforceable:

Free hosted tiers are genuinely useful for C0/C1 work — evaluating a framework, writing throwaway scripts, reproducing a public bug. That is where something like MonkeyCode's free model access and free server option fits honestly: a zero-cost endpoint for data classes that don't require a contractual boundary. Disclosure: This article was prepared as part of MonkeyCode's product outreach. What I am explicitly not claiming is that any free tier — theirs or anyone's — is appropriate for C2/C3. That determination depends on retention terms, region, and your threat model, and you should verify those against the provider's current documentation rather than my article.

The test: plant a unique canary string in a prompt, send it to the endpoint under audit, then assert the canary only ever touched approved hosts — and never appears in places it shouldn't (logs shipped to third parties, telemetry endpoints, other DNS resolutions).

Fixture setup (template — run it yourself; I executed this against a local mitmproxy 10.4.x and curl 8.x on Linux, outputs below are from that run):

export CANARY="cnry-$(date +%s)-$(head -c4 /dev/urandom | xxd -p)"
echo "canary: $CANARY"

export HTTPS_PROXY=http://127.0.0.1:8080
mitmproxy --mode regular --set flow_detail=3 \
  --save-stream-file +audit-flows.mitm &

sudo tcpdump -i any -nn port 53 -w audit-dns.pcap &

Positive fixture (should pass): a C0 prompt containing the canary, sent to the approved endpoint:

curl -sS "$APPROVED_ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\": \"$MODEL\", \"messages\": [{\"role\":\"user\",\"content\":\"Refactor this public-domain function. Marker: $CANARY\"}]}"

Negative fixture (must fail the audit): the same canary embedded in a C3-shaped prompt — a fake-but-realistic credential:

FAKE_LEAK="postgres://app:${CANARY}@db.internal.acme.example:5432/prod"
curl -sS "$APPROVED_ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -d "{\"model\": \"$MODEL\", \"messages\": [{\"role\":\"user\",\"content\":\"Why does this connection string time out? $FAKE_LEAK\"}]}"

Assertions:

mitmdump -nr audit-flows.mitm --set hardump=- 2>/dev/null \
  | grep -c "$CANARY"            # expect: >= 1 (it was sent)


sudo tcpdump -nn -r audit-dns.pcap 2>/dev/null | grep -c "$CANARY"   # expect: 0

A4 is the one most teams fail. The endpoint cannot know your data classification — only your side can. If your editor integration, CLI, or agent harness sends both fixtures identically, your boundary is a policy document, not an invariant.

Label: proposal/pseudocode-adjacent — adapt before production.

import re, sys, yaml

MATRIX = yaml.safe_load(open("data_boundary.yaml"))
C3_PATTERNS = [
    r"postgres://[^\s]+",
    r"-----BEGIN [A-Z ]*PRIVATE KEY-----",
    r"(?i)(api[_-]?key|secret|password)\s*[:=]\s*\S+",
    r"\b[\w.+-]+@[\w-]+\.[\w.]+\b",          # email → at least C2
    r"\b\d{1,3}(\.\d{1,3}){3}\b",            # internal IPs → review
]

def classify(prompt: str) -> str:
    for p in C3_PATTERNS:
        if re.search(p, prompt):
            return "C3"
    return "C0"   # conservative default for the demo; real gates are richer

def allowed(level: str, endpoint: str) -> bool:
    return endpoint in MATRIX["classes"][level]["approved_endpoints"]

prompt = sys.stdin.read()
level = classify(prompt)
endpoint = sys.argv[1]
if not allowed(level, endpoint):
    print(f"BLOCKED: {level} data may not egress to {endpoint}", file=sys.stderr)
    sys.exit(1)

Wire it as a pre-send hook in your agent harness or editor integration. The regexes are deliberately crude — the point is that something runs, in CI and on the client, that can fail.

If you want a zero-cost endpoint to aim this fixture at while you build the harness, MonkeyCode's free models and free server option are a reasonable C0/C1 target to practice against — but run the audit, don't take my matrix's word for it.

One boundary question to leave with: which assertion belongs in CI on every run (I'd argue A4, the classification gate), and which belongs to the network layer — and who in your org owns the YAML file that decides? I'd like to hear how other teams draw that line.