Every team wants AI. Then three questions arrive: What does it cost? Where are the keys? Can we stop it?
I'm a platform engineer, and I built SanVi to answer all three on Kubernetes.
Apps call one OpenAI-compatible endpoint (LiteLLM engine) with their own virtual key. Each key has its own models, budget and rate limit. No key → 401. Over the rate limit → 429. Over budget → blocked. Every call is logged with team, model, tokens and cost in Postgres.
The provider key lives only in Vault and syncs in through External Secrets. I rotated it: about 70 seconds, no redeploy.
Leaked key, runaway agent loop, spend spike: kill_switch.sh block <team> returns 401 in seconds, and unblock restores it. Nothing is deleted.
Teams, models and budgets live in access/teams.yaml. Changes go by pull request; CI checks policy (a greedy $50 budget was blocked); Argo CD applies on merge.
The Right-Size Advisor runs the same eval suite on each model. On my workload Haiku matched Sonnet: 95% cheaper, 40% faster. Any change that drops below 80% fails CI.
The deployment agent turns one sentence into a validated Dockerfile and Helm chart, but it can only open a pull request.
A compliance script checks 12 controls (no secrets in Git, least-privilege access, audit trail, kill switch, quality gate...) mapped to ISO 27001, SOC 2 and EU AI Act themes, on every change.
Repo, Quickstart and Helm chart: https://github.com/sandhilk/sanvi-gateway It's a personal project, so blunt feedback is welcome. What would you need before running this in your company?