# Self-hosted LiteLLM gateway for Codex on AWS ECS with Bedrock

> Source: <https://promptcube3.com/en/threads/8763/>
> Published: 2026-09-03 22:50:12+00:00

# Self-hosted LiteLLM gateway for Codex on AWS ECS with Bedrock

## What this actually gets you

Codex keeps doing what it does best — reading your repo, proposing edits, running tests inside its sandbox. The only difference: every model call hits `/v1/responses`

on your LiteLLM endpoint instead of OpenAI directly. LiteLLM validates the request, checks the caller's virtual key against your policy, then assumes its ECS task role to invoke the Bedrock model ([Claude](/en/tags/claude/) 3.5 Sonnet, GPT-4o, whatever you've approved). The response streams back through the same path.

You get:

- Per-team or per-project virtual keys with hard spend caps
- Request/response logging to CloudWatch for compliance
- WAF rules in front of the ALB for IP allowlists or geo-blocking
- RDS PostgreSQL backing LiteLLM's usage and budget tables
- Secrets Manager + KMS for key storage — no plaintext creds in task definitions

## Deploy the stack

The reference implementation lives in the `guidance-codex`

repo under the `feat/enterprise-gateway-readiness`

branch. Clone it and follow the LiteLLM quickstart — it provisions everything via CDK.

```
git clone -b feat/enterprise-gateway-readiness \
  https://github.com/openai-on-aws/guidance-codex.git
cd guidance-codex
# Bootstrap CDK if you haven't in this account/region
cdk bootstrap
# Deploy the gateway stack (ALB → Fargate → RDS → Bedrock)
cdk deploy LiteLLMGatewayStack
```

The stack outputs the gateway URL and a sample virtual key. Save both.

## Configure Codex to use your gateway

On each developer machine, set two environment variables before launching Codex:

```
export OPENAI_BASE_URL=https://<gateway-alb-dns>/v1
export OPENAI_API_KEY=sk-litellm-<your-virtual-key>
```

Then run `codex`

normally. The CLI will POST to your `/v1/responses`

endpoint, and you'll see the traffic in CloudWatch Logs under the Fargate task's log group.

## Validate the loop works end-to-end

1. **Streaming** — ask Codex to "write a quicksort in Python with comments." You should see tokens appear incrementally in the terminal.

2. **Function calling** — have it "list all `.py`

files in this repo and summarize each." Codex will emit a `list_files`

tool call, run it locally, then send the result back through the gateway for the next turn.

3. **Budget enforcement** — exceed the virtual key's daily limit; the gateway returns a 429 with a clear error body.

## When this is overkill

**Small team, high trust**— IAM Identity Center with direct Bedrock access is simpler. No gateway to maintain.** Don't want to run infra**— Portkey's managed gateway handles auth, budgets, and observability with a SaaS control plane. You trade operational burden for a monthly bill.**Only need OpenAI models**— LiteLLM adds latency (one extra hop). If you're not routing to Bedrock or mixing providers, a plain API key works fine.

## Worth the effort?

For orgs that need audit trails, hard spend controls, and zero long-lived credentials on laptops — yes. The CDK stack is production-grade (multi-AZ Fargate, RDS with deletion protection, WAF managed rules). Expect ~$150–300/month for the gateway infra at modest traffic, plus Bedrock model costs. The repo's quickstart gets you to a working endpoint in under an hour.

[Next Claude Fable 5.1 just cracked a 370-year-old royalist cipher →](/en/threads/8733/)