When a Self-Hosted Model Gateway Tripped Trust & Safety

A developer writing on the blog hoelee.com documented how a self-hosted OpenAI-compatible model gateway caused six independent workflow seats to be disabled simultaneously after a vendor trust-and-safety block, because the gateway flattened upstream 403 policy decisions into 401 invalid_api_key errors. The fix separated error classes so policy blocks return a distinct upstream_policy_block response, and subscription work was moved to isolated single-task invocations of the vendor's official CLI so OAuth tokens never leave the vendor's binary.

Originally published on my blog — read it there with the full context, code and updates https://blog.hoelee.com/posts/when-a-self-hosted-model-router-tripped-trust-and-safety/ . A single self-hosted OpenAI-compatible gateway served every model tier: long-context reading, refactors, cheap background jobs. One key, one model picker, one dashboard where spend was visible in one place. Six independent workflow seats — repo indexing, CI triage, interactive refactors — sat behind it, each authenticating with its own subscription OAuth credential. Then all six came back disabled inside one short detection window. None of them had been doing anything unusual. What tripped was a scheduled credential check, and what it found was a block already applied upstream. The interesting part was not the block. It was the ten minutes I spent rotating credentials that were never wrong. For the first ten minutes, every consumer of the gateway's error reported a credential failure — orchestrator, shell probe, CLI — because that is what the body said: 401 invalid api key . The upstream had actually returned a policy decision. Three probes separated the layers, and they are worth keeping for any "the model stopped answering" ticket: confirm the gateway answers its own health endpoint; confirm a model from an unrelated provider succeeds on the same key; then loop the subscription models and compare the error codes. One model failing means a broken model. Every model on one provider failing identically, while other providers succeed on the same key, means the provider relationship is closed — and no key rotation moves that. The real defect is the error contract, not the outage: Before: policy decisions disguised as auth failures if upstream.status == 403: return jsonify error="invalid api key" , 401 After: distinct classes, distinct client behaviour if "Terms of Service" in upstream.text or "disabled in this account" in upstream.text: return jsonify error="upstream policy block", upstream status=403, alert="credential-policy" , 502 return jsonify error="invalid api key" , 401 A 401 that means "your key is wrong" and a 403 that means "the vendor closed this seat" must never collapse into the same response. Flattening them costs the responder the first ten minutes of every incident. An API key is a bearer credential: whoever holds it can use it. An OAuth token is bound to a client identity — client id, consent record, scopes. When a different process presents that token, the request keeps the claims and loses the behaviour: headers, TLS fingerprint, request cadence. Two properties of the initial gateway topology made the pattern obvious to any detection system: Simultaneous failures across independent workflows are not six human reviews. They are one deterministic cluster rule firing. The vendor's terms had said it in advance: scripting the official CLI is acceptable, pooling seats or automating rotation across them is not. The lesson is not "use fewer seats". It is that a subscription is a seat, not an API. The moment its token leaves the vendor's client, it becomes a shared credential. So the gateway keeps serving models paid for by API key, and subscription work moved to task dispatch: the orchestrator runs isolated, single-task invocations of the official CLI bound to a dedicated workflow seat. No pooled credentials, no hijacked persistent session, no rotation. The token never leaves the vendor's binary. php orchestrator - task dispatcher - official CLI owns the token - vendor | +- cwd, permission scope, timeout, tree-kill, no retries Four dispatcher properties mattered more than any prompt: -y otherwise blocks the parent forever. proc = subprocess.Popen argv, cwd=repo, stdin=subprocess.DEVNULL, no interactive hangs stdout=subprocess.PIPE, stderr=subprocess.PIPE, start new session=True, own process group try: out, err = proc.communicate timeout=task timeout except subprocess.TimeoutExpired: os.killpg proc.pid, signal.SIGKILL kill the whole tree raise Context stopped travelling in prompts as well: the delegate reads repository instruction files and queries a self-hosted memory service over MCP, so a brief stays six lines instead of a transcript dump. Delegate writes are tagged status: draft and final archiving stays with the orchestrator. Nine turns in ten are routine orchestration — tool loops, retrieval, cron, log triage, short edits. Those run on a curated open-weight plan: $10/month carrying a $100 monthly allowance on the fast model, with per-model pools beside it. At $0.15 per million input tokens and $0.60 per million output, a 4k-in/500-out turn costs about $0.0009 — under a tenth of a cent. The remaining tenth is heavy engineering: multi-file refactors, legacy archaeology, anything with a real build-and-test contract. That dispatches to an authorised CLI seat, where the vendor's client owns the credential and the allowance belongs to the subscription rather than to a pooled bucket. Pay for frequency with API-key models, pay for depth with authorised seats, and never let the two share a token. If a credential is bound to a client, where should a platform team draw the execution boundary when metered APIs and client-bound CLI tools have to coexist: standardise on pure APIs and pay the premium, or run a dispatcher that governs official binaries? I have moved to the second, and I would like to hear from anyone who has run the first at team scale and made it survive an audit. Two related posts from the same infrastructure work: I write about self-hosted infrastructure, DevOps and the failures behind them at blog.hoelee.com https://blog.hoelee.com/ .