Self-hosted LiteLLM gateway for Codex on AWS ECS with Bedrock

A self-hosted LiteLLM gateway for OpenAI's Codex on AWS ECS with Bedrock provides per-team virtual keys, spend caps, and CloudWatch logging, with a reference CDK stack in the guidance-codex repo under the feat/enterprise-gateway-readiness branch. The gateway routes model calls to Bedrock models like Claude 3.5 Sonnet and GPT-4o, costs an estimated $150–300/month, and is overkill for small teams or those not mixing providers.

Self-hosted LiteLLM gateway for Codex on AWS ECS with Bedrock What this actually gets you Codex keeps doing what it does best — reading your repo, proposing edits, running tests inside its sandbox. The only difference: every model call hits /v1/responses on your LiteLLM endpoint instead of OpenAI directly. LiteLLM validates the request, checks the caller's virtual key against your policy, then assumes its ECS task role to invoke the Bedrock model Claude /en/tags/claude/ 3.5 Sonnet, GPT-4o, whatever you've approved . The response streams back through the same path. You get: - Per-team or per-project virtual keys with hard spend caps - Request/response logging to CloudWatch for compliance - WAF rules in front of the ALB for IP allowlists or geo-blocking - RDS PostgreSQL backing LiteLLM's usage and budget tables - Secrets Manager + KMS for key storage — no plaintext creds in task definitions Deploy the stack The reference implementation lives in the guidance-codex repo under the feat/enterprise-gateway-readiness branch. Clone it and follow the LiteLLM quickstart — it provisions everything via CDK. git clone -b feat/enterprise-gateway-readiness \ https://github.com/openai-on-aws/guidance-codex.git cd guidance-codex Bootstrap CDK if you haven't in this account/region cdk bootstrap Deploy the gateway stack ALB → Fargate → RDS → Bedrock cdk deploy LiteLLMGatewayStack The stack outputs the gateway URL and a sample virtual key. Save both. Configure Codex to use your gateway On each developer machine, set two environment variables before launching Codex: export OPENAI BASE URL=https://<gateway-alb-dns /v1 export OPENAI API KEY=sk-litellm-<your-virtual-key Then run codex normally. The CLI will POST to your /v1/responses endpoint, and you'll see the traffic in CloudWatch Logs under the Fargate task's log group. Validate the loop works end-to-end 1. Streaming — ask Codex to "write a quicksort in Python with comments." You should see tokens appear incrementally in the terminal. 2. Function calling — have it "list all .py files in this repo and summarize each." Codex will emit a list files tool call, run it locally, then send the result back through the gateway for the next turn. 3. Budget enforcement — exceed the virtual key's daily limit; the gateway returns a 429 with a clear error body. When this is overkill Small team, high trust — IAM Identity Center with direct Bedrock access is simpler. No gateway to maintain. Don't want to run infra — Portkey's managed gateway handles auth, budgets, and observability with a SaaS control plane. You trade operational burden for a monthly bill. Only need OpenAI models — LiteLLM adds latency one extra hop . If you're not routing to Bedrock or mixing providers, a plain API key works fine. Worth the effort? For orgs that need audit trails, hard spend controls, and zero long-lived credentials on laptops — yes. The CDK stack is production-grade multi-AZ Fargate, RDS with deletion protection, WAF managed rules . Expect ~$150–300/month for the gateway infra at modest traffic, plus Bedrock model costs. The repo's quickstart gets you to a working endpoint in under an hour. Next Claude Fable 5.1 just cracked a 370-year-old royalist cipher → /en/threads/8733/