{"slug": "nexus-open-source-ai-gateway-for-enterprise-llm-traffic", "title": "Nexus β open-source AI gateway for enterprise LLM traffic", "summary": "Nexus, an open-source AI gateway, now intercepts enterprise large language model traffic at the SDK, network, and OS layers to enforce compliance, audit, and control policies across all LLM applications. The gateway normalizes requests from any provider to a canonical OpenAI format, supports 20 provider adapters, and offers features including multi-axis quotas, semantic caching, PII detection, and role-based access control. Enterprises can deploy the three independent componentsβAI Gateway, Compliance Proxy, and Desktop Agentβto secure and govern LLM traffic without requiring application code changes.", "body_md": "Make AI safe to use across the enterprise.\n\nNexus Gateway intercepts enterprise LLM traffic at three layers and runs all of it through one compliance engine, one audit pipeline, and one control plane.\n\n| Mode | Where it intercepts | Code |\n|---|---|---|\nπ AI Gateway |\nSDK layer β virtual keys on `/v1/chat/*` , `/v1/responses` , `/v1/embeddings` , `/v1/messages` |\n`packages/ai-gateway/` |\nπ Compliance Proxy |\nNetwork layer β transparent TLS bump (`CONNECT` + MITM) |\n`packages/compliance-proxy/` |\nπ» Desktop Agent |\nOS layer β macOS / Linux / Windows builds all in development, awaiting QA | `packages/agent/platform/{darwin,linux,windows}/` |\n\nThe three pipes are independent: AI Gateway, Compliance Proxy, and Agent each run the **full hooks pipeline on their own traffic** (`packages/shared/policy/hooks/`\n\n, plus the per-service compliance pipeline β e.g. `packages/agent/internal/compliance/pipeline.go`\n\n). The Agent always egresses directly to the upstream provider β it does **not** care whether enterprise network policy then routes that traffic through the Compliance Proxy.\n\nWhen it does β Agent stamps an Ed25519-signed `X-Nexus-Attestation`\n\nheader on the outbound request (E60, `packages/agent/internal/identity/attestation/`\n\n). The Compliance Proxy peeks this header *before* the TLS bump (`packages/shared/transport/tlsbump/forward_handler.go:119`\n\n); if the signature verifies, the CONNECT becomes pure passthrough β no MITM, no hooks, no audit on that flow, since the Agent already ran them.\n\nApplications speak the OpenAI SDK. Nexus normalises every request to a canonical OpenAI shape, then translates wire format on the way to the actual provider. Shipped adapter codecs today (`packages/ai-gateway/internal/providers/specs/`\n\n):\n\n**First-class codecs (11):**`openai`\n\n,`anthropic`\n\n,`gemini`\n\n,`vertex`\n\n,`azure`\n\n,`bedrock`\n\n,`cohere`\n\n,`minimax`\n\n,`glm`\n\n,`replicate`\n\n,`voyage`\n\n.**OpenAI-compatible passthrough (9):**`deepseek`\n\n,`moonshot`\n\n,`mistral`\n\n,`groq`\n\n,`fireworks`\n\n,`together`\n\n,`perplexity`\n\n,`xai`\n\n,`huggingface`\n\nβ all under`packages/ai-gateway/internal/providers/specs/compat/`\n\n.\n\nReasoning tokens, function calls, vision inputs, structured outputs are carried through the translation. Adding a new provider is a documented procedure under `.claude/skills/add-provider-adapter/`\n\n.\n\n**Exact-match response cache**β Valkey-backed, Redis-wire-compatible.** Provider-native cache accounting**β surfaces Anthropic`cached_tokens`\n\nand Gemini`cachedContentTokenCount`\n\nin billing when the provider reports them.**Semantic vector cache** via the`valkey-search`\n\nmodule β`packages/ai-gateway/internal/cache/semantic/`\n\n(lookup, writer, client, circuit breaker, singleflight, poison guard, index lifecycle).**In-flight singleflight**β concurrent identical prompts fold into one upstream call.\n\n**Multi-axis quotas**β per organization, per virtual key, per provider, per model. Each axis has its own budget and sliding-window enforcement.**Token-based or USD-based budgets.****Hard limits and soft limits**β soft fires an alert; hard rejects with 429.** Real-time accounting**β counters update on every traffic event, no batch lag.** Routing strategies**in`packages/ai-gateway/internal/routing/strategies/`\n\n:`single`\n\n,`fallback`\n\n,`loadbalance`\n\n,`conditional`\n\n,`absplit`\n\n,`policy`\n\n,`smart`\n\n.\n\nPII detection Β· data classification Β· keyword filtering Β· content safety Β· rate limiting Β· IP allowlists Β· request-size validation Β· webhook forwarders Β· per-stage audit (request hooks and response hooks recorded independently) Β· body capture (256 KiB inline + spillstore for the rest, see `packages/shared/storage/spillstore/`\n\n) Β· SIEM forwarder (`packages/compliance-proxy/internal/siem/`\n\n) Β· three-tier kill switch Β· emergency passthrough (`bypassHooks`\n\n/ `bypassCache`\n\n/ `bypassNormalize`\n\n).\n\nChat Β· Embeddings Β· Structured outputs Β· Function / tool calling Β· Vision input Β· Reasoning tokens. Multimodal (epic E62) in development.\n\n**IAM**β RBAC + ABAC with an NRN resource model (`packages/shared/identity/iam/`\n\n).**Virtual keys** with per-key model scope.**OIDC federation** with JIT user provisioning (`packages/control-plane/internal/identity/authserver/login/oidc.go`\n\n, JIT flag in`scim_store.go`\n\n).**Organization / project hierarchy** with per-org quota.**Credential vault**β AES-256-GCM (`packages/control-plane/internal/platform/crypto/aes_gcm.go`\n\n,`packages/ai-gateway/internal/credentials/decrypt/decrypt.go`\n\n) with key rotation.**Agent fleet management**β Hub CA, Thing-based config sync, drift detection.\n\nFive Go services + one React control console. The diagram below shows **only the traffic plane** β the three independent intercept pipes and where each one egresses. Control plane (Hub-centric) and storage are summarized in the component table immediately after.\n\n```\nflowchart TB\n SDK[\"SDK app
(OpenAI SDK)\"]\n HTTPS[\"HTTPS app
(network-proxied)\"]\n Endpoint[\"Developer endpoint
(Cursor / Claude Code / β¦)\"]\n\n AIGW[\"AI Gateway :3050
routing Β· cache Β· quota
+ hooks pipeline\"]\n CPProxy[\"Compliance Proxy :3128
MITM TLS
+ hooks pipeline\"]\n Agent[\"Desktop Agent Β· local
OS-level intercept
+ hooks pipeline\"]\n\n Provider[\"LLM Provider
(OpenAI / Anthropic / Gemini / β¦)\"]\n\n SDK ==>|\"/v1 + VK\"| AIGW\n HTTPS ==>|HTTPS via proxy| CPProxy\n Endpoint ==>|OS-level capture| Agent\n\n AIGW ==> Provider\n CPProxy ==> Provider\n Agent ==> Provider\n\n Agent -. \"X-Nexus-Attestation verified
β passthrough\" .-> CPProxy\n```\n\nThe lateral dotted arrow is the **attestation handoff**: the Agent always egresses directly, but when enterprise network policy happens to route Agent traffic through the Compliance Proxy, the Agent's Ed25519-signed `X-Nexus-Attestation`\n\nheader (E60, `packages/agent/internal/identity/attestation/`\n\n) is verified at TLS-bump time (`packages/shared/transport/tlsbump/forward_handler.go:119`\n\n); on success the CONNECT becomes pure passthrough β no MITM, no hooks, no audit on that flow, since the Agent already ran them on its end.\n\n**Control plane (out-of-band).** All four Go services register with **Nexus Hub** as Things via `packages/shared/transport/thingclient/`\n\n(WebSocket primary, HTTP fallback) and pull configuration from the Hub's device shadow on boot and on change-signal β the Hub never pushes full state. The Control Plane admin API (`:3001`\n\n) and the React UI (`:3000`\n\n) sit alongside, talking to the Hub the same way.\n\n| Component | Port | Code |\n|---|---|---|\nNexus Hub |\n3060 | `packages/nexus-hub/` β Thing Registry, Device Shadow, config sync, jobs, agent CA, SIEM bridge |\nControl Plane |\n3001 | `packages/control-plane/` (Echo) β admin API / BFF, IAM, SSO, analytics |\nAI Gateway |\n3050 | `packages/ai-gateway/` β `/v1` AI traffic, provider adapters, routing, quota |\nCompliance Proxy |\n3128 | `packages/compliance-proxy/` β CONNECT, MITM, compliance pipeline |\nAgent |\nlocal | `packages/agent/` β macOS uses pf packet filter (`packages/agent/internal/platform/darwin/pfintercept/` ); Linux uses `iptables` ; Windows uses `WinDivert` . The legacy NETransparentProxyProvider path (`packages/agent/platform/darwin/NexusAgent/NexusAgentExtension/` ) is still in the repo behind `interceptMode=ne` , but new builds default to pf. All three platforms are development-complete, not yet QA-signed-off. |\nControl Plane UI |\n3000 | `packages/control-plane-ui/` β React + Vite + TypeScript |\n\n**Storage stack**\n\n**PostgreSQL 16**β durable storage. Prisma schema in`tools/db-migrate/`\n\nis the source of truth for dev-time migrations; runtime code reads via hand-written SQL +`pgx`\n\n(no`sqlc`\n\n).**Valkey 8**β Redis-wire-compatible, pinned to`valkey/valkey-bundle:8-trixie`\n\nin`docker-compose.yml`\n\nfor BSD-license parity; the`valkey-search`\n\nmodule ships in the bundle image and backs the semantic vector cache. Pure cache only β no pub/sub.**NATS JetStream**β event streaming and Hub coordination via`packages/shared/transport/mq/`\n\n.\n\n| Tool | Version | Notes |\n|---|---|---|\n| Node.js | 20+ |\nnpm workspaces require npm 10+ |\n| Go | 1.25+ |\nAll Go modules share `go.work` at the repo root |\n| Docker | any recent | Hosts PostgreSQL, Valkey, NATS via `docker-compose.yml` |\n\n```\n./scripts/dev-start.sh\n```\n\nThe script:\n\n- Verifies prerequisites (Node 20+, Go 1.25+, Docker, OpenSSL).\n- Auto-creates\n**repo-root** from`.env`\n\n`.env.example`\n\nwith safe dev defaults for`CHANGE_ME_*`\n\nsecrets (`INTERNAL_SERVICE_TOKEN`\n\n,`ADMIN_KEY_HMAC_SECRET`\n\n,`CREDENTIAL_ENCRYPTION_KEY`\n\n=`openssl rand -hex 32`\n\n, β¦). All four Go services read this via`packages/shared/core/bootenv/`\n\nat boot. - Starts PostgreSQL + Valkey + NATS via\n`docker-compose.yml`\n\n. - Runs\n`npm install`\n\n. - Auto-creates\nand propagates`tools/db-migrate/.env`\n\n`CREDENTIAL_ENCRYPTION_KEY`\n\ninto it so`prisma db seed`\n\ncan re-encrypt the seed credentials. - Applies the Prisma schema (\n`db push`\n\n) and seed under`tools/db-migrate/`\n\n. - Auto-generates the\n**Compliance Proxy dev CA** at`packages/compliance-proxy/dev-certs/{ca.crt,ca.key}`\n\nso the TLS-bump cert issuer can boot. - Prints the per-service\n`go run β¦ -config .dev.yaml`\n\ncommands. - Finally starts the Control Plane UI dev server.\n\nFlags:\n\n`--force-reset`\n\nβ DESTRUCTIVE: wipe local Postgres / Valkey / NATS volumes + the entire`nexus_gateway`\n\ndatabase before re-applying the schema.`--no-dev`\n\nβ bootstrap only; print the per-service commands and exit instead of starting the UI dev server.\n\nOpen one terminal per Go service after the bootstrap finishes:\n\n```\ncd packages/nexus-hub && go run ./cmd/nexus-hub/ -config nexus-hub.dev.yaml # port 3060\ncd packages/control-plane && go run ./cmd/control-plane/ -config control-plane.dev.yaml # port 3001\ncd packages/ai-gateway && go run ./cmd/ai-gateway/ -config ai-gateway.dev.yaml # port 3050\ncd packages/compliance-proxy && go run ./cmd/compliance-proxy/ -config compliance-proxy.dev.yaml # port 3128\nnpm run dev:control-plane-ui # port 3000\n```\n\nThe `-config .dev.yaml`\n\nflag is **required** β each binary defaults to `.config.yaml`\n\n, which is the prod-shape template and is intentionally missing dev-only fields like `hub.id`\n\n. Without the flag the service fails fast at boot.\n\nEach Go service tees logs to `packages//logs/.log`\n\nin dev mode (configured in the service's `*.dev.yaml`\n\n). Override the path with `LOG_FILE=/path/to/file`\n\n.\n\nBrowse to [http://localhost:3000](http://localhost:3000) and sign in as the seeded super-admin:\n\n```\nadmin@nexus.ai / admin123\n```\n\nAdditional seeded roles (`alice@nexus.ai`\n\n, `carol@nexus.ai`\n\n, `bob@nexus.ai`\n\n, `diana@nexus.ai`\n\n) are defined in `tools/db-migrate/seed/seed.ts`\n\n.\n\nAfter the stack is up, walk through [ examples/01-hello-world/](/AlphaBitCore/nexus-gateway/blob/main/examples/01-hello-world) β a 3-minute curl-through-the-gateway demo that ends with you reading the resulting\n\n`traffic_event`\n\nPostgres row.The Control Plane uses OAuth + PKCE bearer tokens. Helpers wrap the flow:\n\n```\ncp tests/.env.local.example tests/.env.local # gitignored; edit if you need to override defaults\nsource tests/lib/loadenv.sh local # picks up tests/.env.local + tests/.env.local.example defaults\nsource tests/lib/auth.sh\n\ncp_login # idempotent; caches token at /tmp/nexus_test_token_local\ncp_curl /api/admin/analytics/cost?groupBy=device\ncp_curl -X POST /api/admin/routing-rules -d @rule.json\n```\n\nFor direct DB inspection in dev:\n\n```\ndocker exec $(docker ps --filter \"name=postgres\" -q | head -1) \\\n psql -U postgres -d nexus_gateway -c \"SELECT ...\"\n```\n\nYou came for an AI gateway. You also get the disciplined AI pair-programming setup that built it. [ CLAUDE.md](/AlphaBitCore/nexus-gateway/blob/main/CLAUDE.md),\n\n[,](/AlphaBitCore/nexus-gateway/blob/main/.cursor/rules)\n\n`.cursor/rules/`\n\n[, and the](/AlphaBitCore/nexus-gateway/blob/main/.claude/skills)\n\n`.claude/skills/`\n\n[lint suite form a fork-adoptable methodology:](/AlphaBitCore/nexus-gateway/blob/main/scripts)\n\n`scripts/check-*`\n\n**Binding rules** in`CLAUDE.md`\n\nplus**35** entries (`.cursor/rules/`\n\n`ls .cursor/rules/`\n\n).**26 invocable skills** under`.claude/skills/`\n\nβ`/prod-deploy`\n\n,`/smoke-gateway`\n\n,`/spec-writing`\n\n,`/add-provider-adapter`\n\n, hardened runbooks for repeatable procedures.**23**β every binding rule has a mechanical gate; pre-commit + CI dual layer.`scripts/check-*`\n\nlint scripts**95% per-package coverage gate** enforced by`scripts/check-go-coverage.sh`\n\n+`scripts/.coverage-allowlist`\n\n.**2-round completion self-audit** before claiming \"done\" (see`CLAUDE.md`\n\nβ Mandatory rules β Workflow discipline β Self-audit).\n\n```\npackages/\n nexus-hub/ Go β Thing Registry, Shadow, config sync, jobs, SIEM bridge, agent CA\n control-plane/ Go + Echo β admin API / BFF, IAM, SSO, analytics\n ai-gateway/ Go β /v1 AI traffic, provider adapters, routing, quota\n compliance-proxy/ Go β transparent TLS proxy, CONNECT, compliance pipeline\n agent/ Go β desktop traffic interception (macOS / Linux / Windows;\n all builds in development, awaiting QA)\n shared/ Go β cross-service business logic (hooks, traffic, configtypes,\n mq, thingclient, cache, β¦)\n control-plane-ui/ React + Vite + TypeScript β admin dashboard\n ui-shared/ Shared design tokens, chart colors, i18n bundles\n\ntools/db-migrate/ Prisma schema + migrations + seed (dev-time only)\n\nscripts/ dev-start.sh + check-* lint scripts\ntests/ Test harnesses, .env.local.example, auth.sh helper, smoke scripts\nexamples/ Self-contained demos (01-hello-world, β¦)\n\ndocker-compose.yml Local PostgreSQL + Valkey + NATS\ngo.work Go workspace (one module per package + tools)\nMakefile build / test targets per service\n```\n\n**Go services**β Go 1.25+ with`go.work`\n\n; Echo on Control Plane / Nexus Hub / AI Gateway (`labstack/echo/v4 v4.15.2`\n\n); structured logging via`log/slog`\n\n; metrics via Prometheus`promauto`\n\n; Redis-wire client`redis/go-redis/v9 v9.19.0`\n\n; WebSocket via`coder/websocket v1.8.14`\n\n.**Control Plane UI**β React + Vite + TypeScript (strict mode); React Query via the`useApi`\n\nhook; layered design tokens in`packages/ui-shared/src/styles/`\n\n(`global.css`\n\nraw β`light.css`\n\n/`dark.css`\n\nsemantic, flipped by`data-theme`\n\n); i18n with`react-i18next`\n\n(`en`\n\n/`zh`\n\n/`es`\n\nunder`packages/control-plane-ui/public/locales/`\n\nand`src/i18n/locales/`\n\n); tests via Vitest.**Database**β PostgreSQL 16. Prisma is the dev-time source of truth (`tools/db-migrate/`\n\n); runtime queries use hand-written SQL +`pgx`\n\n.**Cache**β Valkey 8 (Redis-wire-compatible, BSD-licensed`valkey/valkey-bundle:8-trixie`\n\nimage). Pure cache only β no pub/sub anywhere.**MQ**β NATS JetStream behind the`packages/shared/transport/mq/`\n\ninterface.**Monorepo**β npm workspaces (`packages/control-plane-ui`\n\n,`packages/agent/ui/frontend`\n\n,`tools/db-migrate`\n\n) +`go.work`\n\nfor Go.\n\nEvery Go module under `packages/`\n\nreferences its sibling workspace packages by `require github.com/AlphaBitCore/nexus-gateway/packages/ v0.0.0--`\n\n. Those pseudo-version `require`\n\ns are only there to make each module syntactically valid on its own β real resolution comes from `go.work`\n\nat the repo root.\n\nThis has one consequence: if `go.work`\n\nis missing from the build context, Go falls back to the literal pseudo-version in `require`\n\nand tries to fetch the module from GitHub instead of using the local source tree. The build \"succeeds\" against an old remote snapshot, masking local changes.\n\nRules for every build environment:\n\n**Fresh clone**β`git clone`\n\nalready includes the committed`go.work`\n\nand`go.work.sum`\n\n. Run`go build`\n\nfrom inside the repo.**Docker**β copy`go.work`\n\n+`go.work.sum`\n\n**and every**, not just the service's own folder. Minimum viable layout:`packages/`\n\ndirectory the service transitively depends on\n\n```\nWORKDIR /build\nCOPY go.work go.work.sum ./\nCOPY packages/shared packages/shared\nCOPY packages/ packages/\nWORKDIR /build/packages/\nRUN go build -o /out/ ./cmd//\n```\n\n**CI**β use full`actions/checkout`\n\n(default fetch-depth, no sparse-checkout).**Sanity probe**β`GOWORK=off go build ./cmd//`\n\nfrom inside a workspace package should refuse to build or pull a remote snapshot.\n\nIf a contributor reports \"Go keeps downloading our own modules from GitHub\", the answer is always: their build context is missing `go.work`\n\n(or they have `GOWORK=off`\n\nset).\n\n| Command | Purpose |\n|---|---|\n`./scripts/dev-start.sh` |\nOne-shot bootstrap (Docker + DB + seed + UI) |\n`npm run dev:control-plane-ui` |\nStart the UI dev server only |\n`make build-all` |\nBuild the Go services + UI. Go binaries land in `dist/bin//` . |\n`make test-all` |\nRun `go test -race -count=1` for every Go module + UI Vitest |\n`make clean` |\nRemove `dist/bin/` and `packages/control-plane-ui/dist/` . Platform agent packages under `dist/{macos,linux,windows}/` are preserved β clean those via the per-platform targets (`agent-clean-macos` , `agent-clean-windows` ). |\n`npm run check:all` |\nRun every pre-commit lint (i18n parity, design tokens, terminology, migration timestamps, useApi keys, sidebar icons, β¦). CI runs the same set. |\n`npm run db:migrate` |\nCreate a new Prisma migration in `tools/db-migrate/` |\n\nTo build, sign, notarize, or package the macOS Agent (`.app`\n\n/ `.pkg`\n\n), always invoke the `build-agent`\n\nClaude Code skill β not the raw `wails`\n\n/ `codesign`\n\n/ `notarytool`\n\ncommands. See `CLAUDE.md`\n\nβ \"macOS Agent builds MUST go through `Skill('build-agent')`\n\n\" binding rule for why.\n\nβ binding charter. Plan + Todo gate, English-only artifacts, IAM impact review, macOS NE fail-open, pre-edit reading, completion-time self-audit, real-implementation-only, development-phase greenfield policy.`CLAUDE.md`\n\nβ workflow summary, pre-commit checks, high-blast-radius surfaces, review pointers.`CONTRIBUTING.md`\n\n**Project Maintainer**β the original idea behind Nexus Gateway came from him, and he stayed hands-on throughout: code, tests, design reviews, architectural decisions.**The wider team**β engineers, code reviewers, QA, design folks, and the people running prod. The architecture decisions, design reviews, code-review catches, and prod incidents that shaped this codebase all came from team collaboration.β Anthropic's CLI assistant did the lion's share of the implementation work, side-by-side with the human maintainers.[Claude Code](https://claude.com/claude-code)\n\n*AI is already here. Keep learning, keep adapting.*", "url": "https://wpnews.pro/news/nexus-open-source-ai-gateway-for-enterprise-llm-traffic", "canonical_source": "https://github.com/AlphaBitCore/nexus-gateway", "published_at": "2026-05-26 21:50:44+00:00", "updated_at": "2026-05-26 22:11:29.219248+00:00", "lang": "en", "topics": ["ai-safety", "ai-infrastructure", "ai-products", "ai-tools", "ai-agents"], "entities": ["Nexus Gateway", "Nexus", "AI Gateway", "Compliance Proxy", "Desktop Agent", "Ed25519", "X-Nexus-Attestation", "MITM"], "alternates": {"html": "https://wpnews.pro/news/nexus-open-source-ai-gateway-for-enterprise-llm-traffic", "markdown": "https://wpnews.pro/news/nexus-open-source-ai-gateway-for-enterprise-llm-traffic.md", "text": "https://wpnews.pro/news/nexus-open-source-ai-gateway-for-enterprise-llm-traffic.txt", "jsonld": "https://wpnews.pro/news/nexus-open-source-ai-gateway-for-enterprise-llm-traffic.jsonld"}}