A self-hosted LiteLLM proxy with a Postgres database gives you one row per agent request: time, model, input and output tokens, cost, the key that sent it, a session ID and your own tags. Setup takes about 15 minutes with Docker. I ran the steps below on 19 October 2026 with LiteLLM 1.106.0 and Claude Code 2.1.303, against a mock model so no provider was billed; the Codex and Cursor steps come from the docs and I have not run them.
A proxy records one row per request; a provider dashboard records buckets #
An LLM proxy sits between your agents and the provider and writes a row for every call. Anthropic's usage report comes in buckets of one minute to one day, grouped at best by API key, workspace or model, its cost report in daily buckets, and the Admin API behind them needs an organization account. Neither says which session, task or tool spent the tokens.
LiteLLM's LiteLLM_SpendLogs table has columns for the request ID, model, prompt and completion tokens, spend in dollars, start and end time, the hashed key, session_id, request_tags, status and optionally the request and response bodies. Cache read and write counts sit in the row's metadata.usage_object, not in columns of their own.
Jared Siegel's post of 18 October, which reached the Hacker News front page, makes the case with two screenshots: his proxy log, with session, key alias, cost and time to first token per row, against a ChatGPT usage chart where 71.8% of usage is labelled "Unknown". He does not name his proxy.
What you need #
- Docker with Compose, or Python 3.10 or later with
uv. - A provider API key: Anthropic Console or OpenAI. A Claude subscription works only with the caveats under "Claude Code still bills the subscription" below.
- Claude Code, and optionally OpenAI Codex CLI.
- For Cursor , a public HTTPS address for the proxy: Cursor calls it from its own servers.
1. Start LiteLLM and Postgres #
Start the proxy and its database with LiteLLM's quickstart Compose file, pinned to a release and bound to your own machine:
curl -sSLO https://github.com/BerriAI/litellm/raw/main/docker/docker-compose.quickstart.yml
printf 'LITELLM_MASTER_KEY=sk-%s\nLITELLM_SALT_KEY=sk-%s\nLITELLM_BIND=127.0.0.1:\n' \
"$(openssl rand -hex 32)" "$(openssl rand -hex 32)" > .env
sed -i '' 's#docker.litellm.ai/berriai/litellm:latest#ghcr.io/berriai/litellm:v1.106.0#' docker-compose.quickstart.yml
docker compose -f docker-compose.quickstart.yml up -d
curl -s http://127.0.0.1:4000/health/liveliness
Expect "I'm alive!" within about 15 seconds. The sed line is the macOS form; on Linux drop the ''. The Compose file binds port 4000 on every interface unless LITELLM_BIND is set, and it ships with the latest tag; LiteLLM's own deploy docs say to pin a version. Keep .env: a new salt key makes stored credentials unreadable. The admin UI is at http://127.0.0.1:4000/ui, user admin, password the master key.
2. Add the models your agents ask for #
Register each model ID the agent will send, with your provider key. The quickstart stores models in the database, so add them through the API (or the UI):
export MK=$(grep ^LITELLM_MASTER_KEY .env | cut -d= -f2)
curl -s -X POST http://127.0.0.1:4000/model/new \
-H "Authorization: Bearer $MK" -H 'Content-Type: application/json' \
-d '{"model_name":"claude-sonnet-5-5",
"litellm_params":{"model":"anthropic/claude-sonnet-5-5","api_key":"'"$ANTHROPIC_API_KEY"'"}}'
Repeat for claude-opus-5-5 and claude-haiku-5-5: /model can switch to either, and some subagents, such as the built-in claude-code-guide, run on Haiku. The response echoes the model with its parameters encrypted. LiteLLM's subscription tutorial uses a single anthropic/* wildcard route in a config.yaml instead.
3. Create one virtual key per tool #
Give every agent its own key, so each row says which one spent it, with a budget as a ceiling:
curl -s -X POST http://127.0.0.1:4000/key/generate \
-H "Authorization: Bearer $MK" -H 'Content-Type: application/json' \
-d '{"key_alias":"claude-code","max_budget":20,"budget_duration":"30d"}'
The response contains "key": "sk-...", "key_alias": "claude-code" and "max_budget": 20.0. Make another for Codex. The provider key never leaves the proxy, which is the point for agents running in a sandbox.
4. Send one test request #
Check the key and the route with a direct Messages call before involving an agent:
export VK=sk-... # the claude-code key
curl -sS -X POST http://127.0.0.1:4000/v1/messages \
-H "Authorization: Bearer $VK" -H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-H "x-litellm-tags: task:proxy-test" \
-d '{"model":"claude-sonnet-5-5","max_tokens":20,"messages":[{"role":"user","content":"Capital of France?"}]}'
Expect a JSON message starting {"content":[{"text":..., and response headers that include x-litellm-version: 1.106.0 and x-litellm-key-spend. A 401 means the key or header is wrong; Invalid model name passed in means step 2 is missing that model.
5. Point Claude Code at the proxy #
Claude Code reads the proxy address and key from environment variables (LLM gateway docs):
unset ANTHROPIC_API_KEY
export ANTHROPIC_BASE_URL=http://127.0.0.1:4000
export ANTHROPIC_AUTH_TOKEN=$VK
export ANTHROPIC_CUSTOM_HEADERS="x-litellm-tags: project:billing-api"
claude
Run /status: the base URL line should show the proxy and the auth line should name ANTHROPIC_AUTH_TOKEN. To keep it, put the same variables under env in ~/.claude/settings.json, never in a committed project file. My headless run through the proxy finished and reported its cost; the session's tokens counted toward the key's spend.
Claude Code sends an x-claude-code-session-id header on every request, and subagents add x-claude-code-agent-id (gateway protocol). LiteLLM 1.106.0 copies the session header into the session_id column: my test request with that header logged it there. CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 adds a request class (main, subagent, compaction and others) and a prompt ID.
6. Point Codex at the proxy #
Codex uses the Responses API, which LiteLLM serves at /v1/responses. Add a provider to ~/.codex/config.toml, with the top-level keys before the first table (LiteLLM Codex guide):
model = "gpt-6.1-sol"
model_provider = "litellm"
[model_providers.litellm]
name = "LiteLLM"
base_url = "http://127.0.0.1:4000/v1"
env_key = "LITELLM_API_KEY"
wire_api = "responses"
Register gpt-6.1-sol in step 2 with an openai/ prefix and your OpenAI key, export the Codex virtual key as LITELLM_API_KEY and run codex. A project-level .codex/config.toml ignores model_provider, so this has to live in your home directory.
Cursor is the odd one out. Its API key docs say every request is "routed through Cursor's servers", and Tab completion always uses Cursor's own models, so a proxy sees only chat-model traffic, and only at a public address: Cursor's Override OpenAI Base URL takes the proxy URL plus /cursor, and builds without that setting can use the Azure OpenAI fields with the bare proxy URL (LiteLLM Cursor guide).
Check that rows arrive #
Query the paginated spend log endpoint with the master key:
curl -s "http://127.0.0.1:4000/spend/logs/v2?start_date=2026-10-19%2000:00:00&end_date=2026-10-19%2023:59:59&page=1&page_size=100" \
-H "Authorization: Bearer $MK"
My test row came back as "model": "anthropic/claude-sonnet-5-5", "prompt_tokens": 2095, "completion_tokens": 503, "spend": 0.00922, the session ID and the tags, plus a User-Agent tag LiteLLM adds by itself. The cost is list price: 2,095 tokens at $2 per million plus 503 at $10. Rows are written in batches, so allow a few seconds.
One gap from my test: the mock model's streamed replies to Claude Code counted toward the key's spend but wrote no row to the spend log, while the direct request did. I could not repeat it against a paid provider, so check the Logs page after your first real session. Rows also go straight into Postgres if you prefer SQL: select "startTime", model_group, spend, session_id, request_tags from "LiteLLM_SpendLogs".
A proxy, OpenTelemetry and a trace platform answer different questions #
A proxy answers "which key, client and session spent what, across every provider", and can stop a key at its budget. Claude Code's built-in OpenTelemetry answers the same for Claude Code alone, with no extra hop: CLAUDE_CODE_ENABLE_TELEMETRY=1 exports claude_code.token.usage and claude_code.cost.usage, and a claude_code.api_request event per call (monitoring).
Trace platforms answer "what happened inside the session". Langfuse and MLflow hook into Claude Code and record turns, tool calls and tokens: Langfuse reads the session transcript after each response, and MLflow exports a trace when the session ends. Helicone and the Portkey Gateway are proxies too, but Helicone has been in maintenance mode since Mintlify acquired it, and Portkey's open-source gateway keeps local logs in its own console while its README marks logging, tracing and observability integrations as features of the hosted app. Among the AI tools developers run beside their agents, none of these is on a public Stackness stack without bots as of 19 October 2026 (data sources).
What goes wrong #
Claude Code still bills the subscription
Setting only ANTHROPIC_BASE_URL keeps your claude.ai login: requests pass through the proxy but are billed to the subscription (LLM gateway). Set ANTHROPIC_AUTH_TOKEN, run /logout if startup warns about two credential sources, then check /status. LiteLLM can forward your own subscription token instead, recording list-price cost for rows the subscription pays for. Anthropic's terms say developers may not "collect, store, or intermediate" claude.ai credentials, so keep that route to your own login on your own machine.
Invalid model name passed in
The agent asked for a model ID the proxy does not know, usually one picked in /model or one a subagent runs on. Add the exact ID in step 2, use a wildcard route, or pin ANTHROPIC_MODEL.
Cache reads drop to zero after switching to the proxy
A proxy that rewrites the system prompt or drops cache_control makes every turn bill as uncached input, and Anthropic's gateway protocol says to forward both unchanged. Compare the cache line in /usage before and after the switch.
Too many MCP tools in context
Claude Code turns MCP tool search off when ANTHROPIC_BASE_URL is not a first-party host, because most proxies do not forward tool_reference blocks (MCP docs). Set ENABLE_TOOL_SEARCH=true if your proxy forwards them; LiteLLM's Claude Code quickstart recommends it.
Cursor says "Access to private networks is forbidden"
The base URL points at localhost or a private address. Expose the proxy at a public HTTPS address with its own authentication.
Budgets are ignored or keys fail with "No connected db"
The proxy has no database. Virtual keys, budgets and spend logs all need DATABASE_URL, which the quickstart Compose file sets.
The proxy's total does not match the invoice
LiteLLM prices rows from its own list-price table. Use it to attribute spend; reconcile against the provider's invoice. LiteLLM had two bad releases this year: PyPI versions 1.82.7 and 1.82.8 shipped a credential stealer for about 40 minutes on 24 March, per its security post; the official Docker image was not affected. Pin versions and keep the port on 127.0.0.1.
Skip the proxy when one tool and one machine are all you have #
A proxy is worth its upkeep when several agents, people or providers share keys. A solo developer on Claude Code alone gets per-model tokens from /usage (/cost is now an alias), session reports from ccusage, and per-request rows from OpenTelemetry, all with no extra hop. LiteLLM measured 2 ms median overhead across four 4-vCPU instances against a mock OpenAI endpoint, and 12 ms on two; it publishes no number for a single machine like a laptop. The subagent cost post shows where per-key budgets help.
Checklist #
- LiteLLM pinned to a release, bound to 127.0.0.1,
.envkept - Every model ID the agents send registered, Haiku included
- One virtual key per tool, each with a budget
- Direct
/v1/messagestest returns a message andx-litellm-key-spend - Claude Code:
ANTHROPIC_BASE_URLandANTHROPIC_AUTH_TOKENset,/statusconfirms - Codex: provider in
~/.codex/config.toml,LITELLM_API_KEYexported - Tags set with
ANTHROPIC_CUSTOM_HEADERS - Rows visible in
/spend/logs/v2or the Logs page after a real session - Cache reads still high in
/usageafter the switch
New posts by email #
About one email a week with what we published. No account needed.
Which of these do you use? #
Nobody on Stackness runs LiteLLM yet. Tick what you use and be the first.