{"slug": "reduce-token-cost-with-metrum-ai-router-routing-intelligence-you-can-train", "title": "Reduce token cost with Metrum AI Router – routing intelligence you can train", "summary": "Metrum AI released an open-source AI Router that routes agent workloads to the cheapest eligible model meeting a defined quality-and-cost contract, citing a committed Harbor run in which Codex on `big-coder` consumed 48,470 Harbor input tokens in a single deterministic job. The router admits token budgets and quotas before any cache-miss upstream call, expires routing evidence via the group `contract` field `max_eval_age_days`, and has its Learned Routing Policy abstain when uncertainty is high. It targets platform teams managing caller keys, model groups, and monthly token budgets across many engineers, such as an illustrative 5,000 engineers sharing routed groups, and requires Go as declared in `go.mod` plus Python 3 for local bootstrap.", "body_md": "Agent workloads often make dozens of model calls per task. Paying frontier\nprice for each call is the default path, and token spend shows up after the\nmonthly budget is gone. In one committed Harbor run, Codex on `big-coder` used\n48,470 Harbor input tokens in a single deterministic job\n([docs/harbor-case-study.md](/metrum-ai/router/blob/main/docs/harbor-case-study.md)).\n\nA model group is a quality and cost contract you define. Routing picks the\ncheapest eligible candidate with evidence of meeting that contract. Evidence\nexpires through the group `contract` field `max_eval_age_days`, so stale\nvalidation drops targets from eligibility. Learned Routing Policy abstains when\nuncertainty is high. Token budgets and quotas are admitted before any\ncache-miss upstream call.\nSee [Model Group Contracts](#model-group-contracts) and\n[docs/MODEL_GROUP_CONTRACTS.md](/metrum-ai/router/blob/main/docs/MODEL_GROUP_CONTRACTS.md).\n\n``` php\nflowchart LR\n  Caller[Caller] --> Admit[Auth and budget admission]\n  Admit --> Eligible[Request-shape eligibility]\n  Eligible --> LRP[Learned Routing Policy]\n  LRP --> Upstream[Selected upstream]\n  Upstream --> Feedback[Status usage cost latency]\n  Collect[Offline collect label train eval] --> Bundle[Validated policy bundle]\n  Bundle --> LRP\n```\n\n**Who this is for**\n\n- Platform teams that own caller keys, model groups, and monthly token budgets across many engineers (illustrative: 5,000 engineers sharing routed groups).\n- Teams that mix private OpenAI-compatible targets with hosted providers under one group contract.\n- Operators willing to spend an afternoon on local bootstrap, then a longer shadow window before enforce.\n\n**Who this is not for**\n\n- Teams that want a hosted service operated by someone else.\n- Teams with a single provider and no private models.\n- Teams that need the project to guarantee provider uptime, model quality, or\ncompliance outcomes\n([architecture-limitations](/metrum-ai/router/blob/main/docs-site/docs/reference/architecture-limitations.md) ).\n\nCommunity participation is governed by [CONTRIBUTING.md](/metrum-ai/router/blob/main/CONTRIBUTING.md), the\n[Code of Conduct](/metrum-ai/router/blob/main/CODE_OF_CONDUCT.md), and [GOVERNANCE.md](/metrum-ai/router/blob/main/GOVERNANCE.md).\nQuestions and bugs follow [SUPPORT.md](/metrum-ai/router/blob/main/SUPPORT.md); suspected vulnerabilities\nmust use the private reporting path in [SECURITY.md](/metrum-ai/router/blob/main/SECURITY.md).\nTrademark use is governed by [TRADEMARKS.md](/metrum-ai/router/blob/main/TRADEMARKS.md).\n\nHosted docs: [overview](https://llm-api.apps.metrum.ai/docs/overview) and `/docs/`\non a running router. Technical brief: [docs/solution-brief.md](/metrum-ai/router/blob/main/docs/solution-brief.md).\nDoc ownership map: [docs/DOCS_MAINTENANCE.md](/metrum-ai/router/blob/main/docs/DOCS_MAINTENANCE.md).\n\nHosted product docs: [overview](https://llm-api.apps.metrum.ai/docs/overview)\n(also served from a running router at `/docs/`).\n\nPrerequisites are Go as declared in `go.mod` and Python 3 for the local\nbootstrap. An OpenAI API key is enough for one Chat completion. Building and\nstarting the router require no private repository access.\n\n```\ngit clone https://github.com/metrum-ai/router.git\ncd router\npython3 scripts/local_dev_bootstrap.py --out-dir tmp/local-dev\n# Set OPENAI_API_KEY in tmp/local-dev/env.json.\ngo run ./cmd/metrum-ai-router --config tmp/local-dev/config.yaml\n```\n\nThe bootstrap issues a local runtime `license.json` (SKU `oss-self-managed`)\nand a caller token file `tmp/local-dev/router.token`. It does not print secrets.\nConfirm `/readyz`, then `GET /v1/models` and one Chat request as in\n[Local Quickstart](/metrum-ai/router/blob/main/docs-site/docs/getting-started/local-quickstart.md).\n\n`config.example.yaml` remains the full catalog reference. Do not copy it for a\nfirst local trial.\n\nFor packaged installs, use the public [installation\nguide](/metrum-ai/router/blob/main/docs-site/docs/installation/index.md). The documented deployment modes\nare Linux binary, Docker Compose, and Kubernetes. See [Architecture, Platforms,\nAnd Limitations](/metrum-ai/router/blob/main/docs-site/docs/reference/architecture-limitations.md) for the\noperator-owned boundaries and explicit non-goals.\n\nStatic weights go stale when models and prices change. Generic learned routers\noften lack a quality floor, abstention, and operator-owned labels. Without those\ncontrols, cheap targets under-shoot acceptance tests and spend stays locked to\nthe expensive anchor for weeks. Learned Routing Policy (LRP) trains offline on\nyour outcome data. It selects only among router-eligible targets, abstains under\nuncertainty, and ships behind `strategy: external` with an explicit\n`external_policy.mode` promotion path you own. The operator runbook is\n[docs/LEARNED_ROUTING_POLICY.md](/metrum-ai/router/blob/main/docs/LEARNED_ROUTING_POLICY.md). The service\nREADME is [services/learned-routing-policy/README.md](/metrum-ai/router/blob/main/services/learned-routing-policy/README.md).\n\nSynthetic holdout comparison (anchor vs routed) from\n[docs/evidence/learned-routing-policy/public-training.json](/metrum-ai/router/blob/main/docs/evidence/learned-routing-policy/public-training.json)\n(seed 42, 800 requests, n=113 holdout). Abstention rate is not in that JSON.\n\n**Synthetic holdout: anchor vs routed (n=113)**\n\n| Policy | Cost USD | Quality mean | Floor violations | Abstention rate | n | \n|---|---|---|---|---|---|\n| always_anchor | 0.151829 | 1.0 | 0.0 | not measured | 113 | \n| lrp | 0.1507895 | 1.0 | 0.0 | not measured | 113 | \n| always_cheapest | 0.0151829 | 0.48672566371681414 | 0.5132743362831859 | not measured | 113 | \n| bt_only | 0.151829 | 1.0 | 0.0 | not measured | 113 | \n| oracle | 0.1507895 | 1.0 | 0.0 | not measured | 113 | \n\n`promotable` is false. Failed gates `cost_vs_anchor` and `real_data_and_embedding`\nshow promotion gates working as designed on synthetic data. A real run needs\nreal embeddings and real outcomes before any live enforce decision.\nSee [learned-routing-case-study](/metrum-ai/router/blob/main/docs-site/docs/evaluation/learned-routing-case-study.md)\nand [public-training.json](/metrum-ai/router/blob/main/docs/evidence/learned-routing-policy/public-training.json).\n\n1. Define acceptance tests and a quality floor for the model group.\n2. Collect approved traffic with `lrp collect` .\n3. Label outcomes with `lrp fanout` and`lrp judge` .\n4. Train and gate with `lrp featurize` ,`lrp train` ,`lrp eval` , and`lrp validate` .\n5. Run `lrp serve` with`external_policy.mode: shadow` for at least the\ndocumented 24-hour staging shadow window; treat 7-day staging enforce as a\nseparate later gate\n([docs/LEARNED_ROUTING_POLICY.md](/metrum-ai/router/blob/main/docs/LEARNED_ROUTING_POLICY.md) ,[lrp-train-and-serve](/metrum-ai/router/blob/main/docs-site/docs/routing/lrp-train-and-serve.md) ).\n6. Read shadow vs served comparison from decision telemetry and usage reports.\n7. Flip staging to `external_policy.mode: enforce` ; rollback is one config\nchange to`mode: baseline` .\n\nWithout per-target quality and length models, selection guesses from prices or\nstatic weights and under-shoots the floor. LRP trains per-target quality and\noutput-token models offline with LightGBM. Quality scores are\nisotonic-calibrated. Train and serve share the same feature definitions and\nembedding artifacts.\nSee [docs/LEARNED_ROUTING_POLICY.md](/metrum-ai/router/blob/main/docs/LEARNED_ROUTING_POLICY.md).\n\nCheapest-first without a floor sends 0.513 of the synthetic holdout under the\nquality floor (`always_cheapest` at n=113). Among targets the router already\nmarked eligible, LRP picks the cheapest predicted to meet the operator quality\nfloor. If none meet the floor, it picks the highest predicted quality. Unknown\nprices are not treated as free. Optional per-project floors, upstream latency\ngates, and cache-aware cost estimates apply at selection. Cache savings are\nused only when trustworthy cache metadata and a catalog cached-input price are\npresent.\nSee [docs/LRP_SELECTION_CONSTRAINTS.md](/metrum-ai/router/blob/main/docs/LRP_SELECTION_CONSTRAINTS.md).\n\nA confident wrong prediction sends a hard request to a weak model.\nAbstention and ensemble uncertainty cut that path. `lrp train --ensemble-size 5`\nfits a bootstrap ensemble. With `uncertainty_abstention: true`, high calibrated\nquality standard deviation abstains to `abstention_anchor` (or first fallback)\nand labels `lrp:uncertain`. Thompson exploration\n(`exploration_strategy: thompson`) is restricted to `exploration_projects`. PSI\nand embedding-centroid drift can recommend shadow; the router\n`external_policy.mode` remains the activation authority. Bradley-Terry cold\nstart injects baseline predictions only for targets already eligible.\nSee [docs/LRP_UNCERTAINTY.md](/metrum-ai/router/blob/main/docs/LRP_UNCERTAINTY.md).\n\nMixing host-side execution with LLM scores collapses pass/fail meaning and\npollutes training. Deterministic verifiers run only inside the isolated judge\nworker. LLM judging and human audit are separate outcome classes with different\nmeanings.\nSee [docs/LRP_VERIFIERS.md](/metrum-ai/router/blob/main/docs/LRP_VERIFIERS.md) and\n[docs/LRP_HUMAN_JUDGE.md](/metrum-ai/router/blob/main/docs/LRP_HUMAN_JUDGE.md).\n\nFlipping learned influence without a reversible mode burns a weekend of\nincident rollback. Router `external_policy.mode` values\n([internal/router/external_strategy.go](/metrum-ai/router/blob/main/internal/router/external_strategy.go)):\n\n- `baseline` : no policy call; first eligible configured target.\n- `shadow` : policy is called and recorded; first eligible configured target is served.\n- `enforce` : the policy recommendation is served (default when`mode` is omitted).\n\nRollback of learned influence is `mode: baseline` or restoring prior group\nconfig. Operators may require Ed25519-signed bundles before load.\nSee [docs/LRP_SIGNED_BUNDLES.md](/metrum-ai/router/blob/main/docs/LRP_SIGNED_BUNDLES.md).\n\nPost-completion `external_policy.feedback` posts request ID, status, usage,\ncost, latency, TTFB, and selected target. It does not retrain quality models.\nSee [internal/router/external_policy_feedback.go](/metrum-ai/router/blob/main/internal/router/external_policy_feedback.go)\nand [docs/EXTERNAL_POLICY_CONTEXT.md](/metrum-ai/router/blob/main/docs/EXTERNAL_POLICY_CONTEXT.md).\n\nField names match [config.example.yaml](/metrum-ai/router/blob/main/config.example.yaml) and\n[internal/router/config.go](/metrum-ai/router/blob/main/internal/router/config.go). Catalog-only LRP groups\nstay commented until exact-shape validation and protected evaluation pass.\n\n```\nmodels:\n  workload-staging:\n    strategy: external\n    external_policy:\n      url: http://127.0.0.1:18093/route\n      allow_hosts: [127.0.0.1]\n      mode: shadow\n      timeout_ms: 500\n      max_response_bytes: 65536\n      include_request: true\n      on_error: fail_closed\n      headers:\n        X-LRP-Auth: ${LRP_POLICY_AUTH_HEADER}\n      feedback:\n        enabled: true\n    targets:\n      - { provider: example_chat, model_ref: example-model, weight: 1 }\n```\n\n`include_request: true` belongs only in trusted infrastructure. Enable\nmodel-group `pii_filter` before sending request content. Loopback URL matches\ncurrent router egress rules for the LRP sidecar.\n\n```\nuv sync --project services/learned-routing-policy --locked\nmake lrp-test\nmake lrp-synthetic-demo\n```\n\nSynthetic results do not authorize live promotion.\n\nEvidence means checked-in artifacts with dates, gates, and reproducible commands. Synthetic results are published so gates and wiring can be inspected before anyone spends on real embeddings. Figures below do not authorize live promotion.\n\n| Claim | Evidence | Source | Date | \n|---|---|---|---|\n| Synthetic holdout LRP quality mean 1.0, floor violation 0, cost USD 0.1507895 (n=113) | Single synthetic run, seed 42, 800 requests | [docs/evidence/learned-routing-policy/public-training.json](/metrum-ai/router/blob/main/docs/evidence/learned-routing-policy/public-training.json) | 2026-09-09 | \n| `promotable` is false; gates`cost_vs_anchor` and`real_data_and_embedding` failed | Same snapshot | [public-training.json](/metrum-ai/router/blob/main/docs/evidence/learned-routing-policy/public-training.json) | 2026-09-09 | \n| Synthetic embeddings, mock outcomes; real LightGBM and router wiring | Operator and public case study | [docs/LEARNED_ROUTING_POLICY.md](/metrum-ai/router/blob/main/docs/LEARNED_ROUTING_POLICY.md) ,[learned-routing-case-study](/metrum-ai/router/blob/main/docs-site/docs/evaluation/learned-routing-case-study.md) | 2026-09-09 | \n| Native shadow: LRP recommended `strong` ; router served configured-first`cheap` | Single synthetic inference snapshot | [docs-site/docs/routing/lrp-train-and-serve.md](/metrum-ai/router/blob/main/docs-site/docs/routing/lrp-train-and-serve.md) | 2026-09-09 | \n| Harbor Codex reward 1, Claude Code reward 0 on `big-coder` | One deterministic weighted-group run; not LRP | [docs/harbor-case-study.md](/metrum-ai/router/blob/main/docs/harbor-case-study.md) | 2026-06-29 | \n\nCounting spend after completion lets concurrent large-cap requests overshoot a\nmonthly cap before anyone sees the invoice. Token-budget admission reserves\nestimated input tokens, tool/schema payload size, and the requested output cap\nbefore a cache-miss upstream call. TPM, daily token, monthly token, and lifetime\nkey budgets include in-flight reservations. Completed requests reconcile to\nreported usage; failed or canceled requests release the reservation; cache hits\ndo not consume persisted token quota.\n([API Key Flow](#api-key-flow), `internal/router/service.go`, `internal/router/quota.go`)\n\nIllustrative: 5,000 engineers share one caller project with a $1,500/month\ntoken-budget cap. When remaining monthly tokens map to $12 of headroom and the\nnext request reserves an estimated $18 of input-plus-output cap, admission\nfails with `429` / `quota-exhausted` (or `403` / `key-exhausted` when the\nlifetime key is done) before any provider call. Usage reports show the\nrejection without an upstream attempt. Arithmetic: $1,500/month ÷ 5,000\nengineers ≈ $0.30/engineer/month of shared headroom if the cap is fully used.\n\n```\ncallers:\n  - id: example-standard-dev\n    rate: { rpm: 120, tpm: 200000, concurrent: 8 }\n    quota:\n      day: { requests: 5000, tokens: 20000000 }\n      month: { tokens: 400000000 }\n    key: { lifetime_tokens: 2000000000, soft_pct: 90, on_exhaust: disable }\n```\n\nForwarding first and waiting for a provider `400` spends a billed call on an\nineligible shape. Illustrative: one 8,000-token tool request rejected locally\nat $2.50 per million input tokens avoids about $0.02 of upstream spend\n(8000 / 1e6 × 2.50). The router filters dialect, per-skin tool support,\nmodalities, structured outputs, reasoning controls, max-token honoring, and\npayload size before strategy selection. If none remain, it returns\n`502 no-eligible-target` with no upstream attempt.\n(`internal/router/service.go`, `internal/router/request_shape_eligibility.go`)\n\nStatic routing weights do not expire when an eval ages out. After\n`max_eval_age_days: 30`, a target whose `validated_at` is older than 30 days\ndrops from eligibility even if its weight is still positive. An optional\nmodel-group `contract` applies `require_tags`, `min_eval_quality_score`,\n`min_eval_pass_rate`, `max_eval_age_days`, and `allowed_validation_status`.\n([Model Group Contracts](#model-group-contracts), `internal/router/contract.go`)\n\n```\nmodels:\n  support-chat:\n    strategy: weighted\n    contract:\n      quality_floor:\n        require_tags: [validated]\n        min_eval_quality_score: 0.90\n        min_eval_pass_rate: 0.95\n        max_eval_age_days: 30\n        allowed_validation_status: [passed]\n```\n\nProvider list prices change; historical reports that reprice old rows rewrite\npast months. Illustrative: a 20% mid-month list-price cut would rescale 15 prior\ndays of USD if rows were not frozen at request time. Each usage row stores\ninput/output price per million, pricing source and date, computed USD, plus\nrouting/policy/pricing fingerprints. Savings baselines are source-dated\noperator comparisons.\n([Usage Reports](#usage-reports), `internal/router/usage_db.go`)\n\nSeparate gateways for private GPUs and hosted APIs force callers to pick a\nmodel name per request. vLLM, SGLang, and any OpenAI-compatible service register\nas catalog targets with the same activation rules. One group can weight a\nprivate target with a hosted fallback after exact-shape validation, for example\n80/20 in the snippet below.\n([docs/SELF_HOSTED_UPSTREAMS.md](/metrum-ai/router/blob/main/docs/SELF_HOSTED_UPSTREAMS.md))\n\n```\nmodels:\n  mixed-hardware:\n    strategy: weighted\n    targets:\n      - { provider: private_vllm, model_ref: small-local, weight: 80 }\n      - { provider: hosted_chat, model_ref: fallback, weight: 20 }\n```\n\nA single account rate limit turns one 429 into a fleet outage for that model.\nIllustrative: three provider accounts pooled 1:1:1 absorb three times the\nper-account RPM before the group is empty. A group can pool the same model\nacross provider accounts or endpoints. Optional provider/model/target shaping\ncan start bounded adaptive cooldowns after classified 429 or quota exhaustion\nwhen those knobs are enabled. Fallback runs on retryable classes. Ordinary\nnon-retryable 4xx is not replayed to another provider.\n([API Key Flow](#api-key-flow), `internal/router/upstream_shape.go`)\n\nShipping provider keys or raw prompts into a routing script creates a second\nsecret surface. Illustrative: one leaked key forces rotation across every\ncaller that shared it, often a multi-day outage window. Scripts and external\npolicies receive safe identifiers only. Provider keys are injected server-side.\nOptional `pii_filter` runs before cache key, routing input, and upstream call\n(`redact_only`, `redact_and_restore`, `fail_on_match`). Content capture is\nopt-in, redacted, and AES-256-GCM encrypted.\n([TypeScript Routing](#typescript-routing), [PII Filtering](#pii-filtering),\n`internal/router/content_capture.go`)\n\nA routing dispute without a joinable `request_id` becomes a week of log spelunking.\nWhen usage persistence, diagnostics, and optional decision telemetry are\nenabled, attempts, traces, traffic-shape events, request shapes, translation\nshapes, sanitized upstream errors, and terminal errors join by `request_id`.\n`/admin/reports/api/request-evidence` returns a completeness-scored bundle.\nDecision telemetry stores scalar buckets only. Pre-selection failures may have\nno routing-decision row.\n([Usage Reports](#usage-reports), `internal/router/decision_telemetry.go`)\n\nHosted gateways keep prompts and keys on someone else's control plane. This\ncore is Apache-2.0. No license key required by default. Optional signed-license\nverification is local. Prompts and responses are not retained by default\n(zero-day retention unless you enable capture). The documented policy is not to\ntrain on traffic. Linux binary, Compose, and Kubernetes are the documented\nruntimes. Operators can run on-premises or air-gapped infrastructure.\n([Editions](#editions),\n[deployment-paths](/metrum-ai/router/blob/main/docs-site/docs/licensing/deployment-paths.md),\n[architecture-limitations](/metrum-ai/router/blob/main/docs-site/docs/reference/architecture-limitations.md))\n\nPointing Claude Code or Codex at a generic OpenAI proxy still fails tool and\ncatalog startup checks. Keep the two smoke commands in\n[CLI Smoke Tests](#cli-smoke-tests); Harbor already showed Codex and Claude Code\nagainst the same `big-coder` group in one deterministic pair of jobs\n([docs/harbor-case-study.md](/metrum-ai/router/blob/main/docs/harbor-case-study.md)). Claude Code uses\n`ANTHROPIC_BASE_URL` and `ANTHROPIC_AUTH_TOKEN`. Codex uses\n`/v1/codex/models.json` as a caller-filtered Responses catalog. Tool-bearing\nrequests bypass the response cache. Containerized tool variants:\n[coding-agent-clients](/metrum-ai/router/blob/main/docs-site/docs/getting-started/coding-agent-clients.md#containerized-tool-smokes).\n\nFrom [Explicit Limitations And Non-Goals](/metrum-ai/router/blob/main/docs-site/docs/reference/architecture-limitations.md#explicit-limitations-and-non-goals):\n\n- Model-group names and provider availability are deployment-defined; the project does not guarantee access to any provider or model.\n- Catalog metadata is not capability proof. Tools, images, API bridges, and large request shapes require direct upstream and router-level validation.\n- The in-process response cache is per process and is cleared by restart.\n- After the first native SSE event, the HTTP response is committed. A later\nfailure cannot change the caller's `200` , append a reliable error envelope,\nor fall back to another target; clients must detect a missing terminal event.\n- SQLite is not a shared multi-writer database and must not back horizontally scaled router replicas.\n- The project does not provide provider uptime, model-quality, legal, compliance, or support-service guarantees.\n\nRemaining bullets live on that page.\n\n**Does LRP see my prompts?**\nBy default the external-policy payload uses derived scalars such as token and\ntool counts, not prompt text. `external_policy.include_request: true` sends\nrequest content only to a trusted sidecar; enable `pii_filter` first.\n([External Routing Policy Service](#external-routing-policy-service))\n\n**What happens when the LRP sidecar is down?**\nDefault `on_error: fail_closed` returns `502 routing-policy-error` and no\nupstream call. Optional `on_error: fallback` serves the first eligible\nconfigured target instead.\n([docs/LEARNED_ROUTING_POLICY.md](/metrum-ai/router/blob/main/docs/LEARNED_ROUTING_POLICY.md))\n\n**Can I run without LRP?**\nYes. Use `static`, `weighted`, `failover`, `dynamic_score`, or TypeScript\n`script` strategies. LRP is optional behind `strategy: external`.\n\n**How is this different from LiteLLM?**\nLiteLLM Server manages a unified interface to 100+ LLMs in OpenAI\nChatCompletions/Completions format, plus cost tracking, auth, spend, budgets,\nand load balancing ([https://docs.litellm.ai/docs/proxy/quick_start](https://docs.litellm.ai/docs/proxy/quick_start)).\n\n**Does it learn online?**\nNo. `external_policy.feedback` posts status, usage, cost, and latency for\noffline pipelines. It does not retrain quality models\n([internal/router/external_policy_feedback.go](/metrum-ai/router/blob/main/internal/router/external_policy_feedback.go)).\n\n**What do I need to run on GPU?**\nMeasured CPU BGE embedding stage latency is published for the LRP case study.\nThere is no universal GPU requirement for the router or LRP sidecar. Private\nGPUs remain optional as routing targets\n([learned-routing-case-study](/metrum-ai/router/blob/main/docs-site/docs/evaluation/learned-routing-case-study.md),\n[docs/SELF_HOSTED_UPSTREAMS.md](/metrum-ai/router/blob/main/docs/SELF_HOSTED_UPSTREAMS.md)).\n\n**What does Enterprise edition add?**\nEnterprise is a separate distribution with production validation, named\nsupport, signed releases, and related commercial entitlements\n([Editions](#editions)).\n\n**How do I roll back learned routing?**\nSet `external_policy.mode: baseline` (or restore prior group config). That\nstops policy influence without redeploying binaries\n([internal/router/external_strategy.go](/metrum-ai/router/blob/main/internal/router/external_strategy.go)).\n\n**Where does data live?**\nState, usage, and optional content capture stay on operator-controlled storage.\nUpstream provider calls and approved judging can still transfer prompts and\nresponses outside your network boundary when you configure those paths.\n\nOne caller-facing model group can select different upstream models for different\nrequest shapes. The committed offline proof seeds observations, disables\naffinity/cache, posts `testdata/proof/trivial.json` and\n`testdata/proof/complex.json` through `dynamic_score`, then projects\n`/admin/reports/api/request-evidence` fields:\n\n```\nmake proof-routing\n```\n\nExpected output (generated by the mock harness, not hand-written):\n\n```\n[\n  {\n    \"modelGroup\": \"proof-routing\",\n    \"requestedModel\": \"proof-routing\",\n    \"selectedCandidateIndex\": 0,\n    \"selectedModel\": \"cheap-summarizer\",\n    \"selectedProvider\": \"mock\",\n    \"strategy\": \"dynamic_score\",\n    \"termNames\": [\"summarize_cheap\"]\n  },\n  {\n    \"modelGroup\": \"proof-routing\",\n    \"requestedModel\": \"proof-routing\",\n    \"selectedCandidateIndex\": 1,\n    \"selectedModel\": \"validated-coder\",\n    \"selectedProvider\": \"mock\",\n    \"strategy\": \"dynamic_score\",\n    \"termNames\": [\"code_validated\"]\n  }\n]\n```\n\nIllustrative curl against a **pre-warmed** router with decision telemetry and\nadmin drilldown enabled (cold-start weights and default affinity can pin both\nrequests to one target; use `make proof-routing` as the reproducible gate):\n\n```\nfor f in testdata/proof/trivial.json testdata/proof/complex.json; do\n  RID=$(curl -sS \"$ROUTER/v1/chat/completions\" \\\n        -H \"Authorization: Bearer $ROUTER_TOKEN\" \\\n        -H 'Content-Type: application/json' -d @\"$f\" \\\n        -D - -o /dev/null | awk -F': ' '/[Xx]-Request-Id/{print $2}' | tr -d '\\r')\n  curl -sS \"$ROUTER/admin/reports/api/request-evidence?request_id=$RID\" \\\n       -u \"$ADMIN_USER:$ADMIN_PASS\" | jq '{\n         requestedModel: .request.requestedModel,\n         modelGroup: .request.modelGroup,\n         selectedProvider: .request.provider,\n         selectedModel: .request.model,\n         strategy: .decisionTelemetry.routingDecisions[0].strategy,\n         selectedCandidateIndex: .decisionTelemetry.routingDecisions[0].selectedCandidateIndex,\n         termNames: [.decisionTelemetry.dynamicScoreTerms[] | select(.selected==true) | .termName]\n       }'\ndone\n```\n\nFormer README headings remain reachable below or from this index.\n\nThe response cache is in-process and not persistent. Configure it with:\n\n```\nserver:\n  cache:\n    enabled: true\n    max_bytes: 134217728\n    default_ttl: 15m\n```\n\n`default_ttl` is the maximum duration for an entry. `max_bytes` is the total LRU byte budget. Entries are evicted when expired or when the cache exceeds `max_bytes`.\n\nThe cache key is based on normalized request semantics, caller/project scope, and selected target: caller id, project, model group, system/input/messages, max tokens, temperature, stop sequences, selected sampling fields (`top_p`, `seed`, `frequency_penalty`, `presence_penalty`, `logit_bias`, reasoning/thinking, previous response id), provider, and target model. It does not use the raw request body, caller request IDs, router request IDs, caller tokens, or provider response IDs. Unknown behavior-changing request fields bypass the cache.\n\nCached payloads are sanitized before storage. The router caches text, model, stop reason, usage, and warnings, but not upstream `id`, raw provider payloads, or provider-specific metadata. Every caller-facing response gets a fresh router-owned `resp_...` ID, including cache hits.\n\nCache hits are logged with `cache=hit` and cached usage for telemetry. They do not call providers and do not increment persisted quota/lifetime token counters. Each request also records a cache snapshot with enabled state, item count, occupied bytes, max bytes, and occupancy percentage so usage reports can show cache hit rate and occupancy over time.\n\nThe deployment artifact is a binary package. Operators should not need this source tree on the deployment host.\n\n```\nmake build        # build Docusaurus docs, then local router/tool binaries with embedded docs\nmake build-go-only # local router/tool binaries without rebuilding docs\nmake package      # linux amd64 and linux arm64 tarballs\nmake package-all  # same as package\nmake package-docker # linux amd64 and linux arm64 Docker packages\nmake package-docker-all # same as package-docker\n```\n\nRelease package targets require a clean git tree and reject `-dirty` versions. Use `ALLOW_DIRTY_PACKAGE=1` only for local development artifacts that will not be shipped. Package tar creation runs with `COPYFILE_DISABLE=1`, and package validation rejects macOS AppleDouble metadata, unexpected package files, missing package-safe docs, internal/private markers, raw secrets, local state, and wrong binary architecture.\n\nEach tarball contains:\n\n```\nbin/metrum-ai-router\nbin/metrum-ai-router-token-gen\nbin/metrum-ai-router-usage-report\nbin/metrum-ai-router-migrate\nbin/metrum-ai-routerctl\nbin/metrum-ai-router-fleetctl\nbin/metrum-ai-router-fleet-sign\nbin/metrum-ai-router-license\nbin/metrum-ai-router-customer-lifecycle\nconfig/config.example.yaml\nconfig/env.example.json\nconfig/enterprise-license-skus.json\nconfig/scripts/router.ts\ndocs/PACKAGE_README.md\ndocs/BINARY_INSTALL.md\ndocs/DOCKER_COMPOSE_INSTALL.md\ndocs/KUBERNETES_INSTALL.md\ndocs/PACKAGE_VALIDATION.md\ndocs/solution-brief.md\ndocs/LICENSE.md\nLICENSE\nNOTICE\nTHIRD_PARTY_NOTICES.md\nMODEL_LICENSES.md\ncaddy/Caddyfile\n```\n\n`metrum-ai-router-fleetctl plan|deploy|status|delete|customer` is the\nbinary-package-only Fleet lifecycle contract. `metrum-ai-router-fleet-sign`\nissues protected intent/admission/delete documents and ships only in binary\npackages (never in customer Docker images).\n`metrum-ai-router-license` issues signed runtime-policy `license.json`\nfiles and ships only in binary packages (never in runtime Docker images).\n`plan`, `deploy`, and `delete` consume one mode-`0600`,\nprofile-key-signed, reference-only deployment intent; it contains the protected\nprofile, runtime bundle, and license references without their resolved values.\n`customer create|status|smoke|grant-caller|get-config|list-callers|revoke-caller|update-quota|quota-status|update-config|delete` orchestrates\ndisposable SQLite Fleet instances from the packaged binary alone (no Python/repo).\n`metrum-ai-routerctl` provides customer-local safe config, caller-token-file,\nlicense, model, and aggregate-usage operations and is included in Docker images;\nFleet binaries are not. The default deployment is SQLite state with one Router\ncontainer and one replica; it neither provisions nor binds RDS. Dedicated RDS\nrequires an explicit approved `database_profile` manifest branch and a\nseparately signed, scoped external admission that Fleet never creates. After\ndisposable-E2E evidence exists, one qualified maintainer may self-review before\na production-like non-production rehearsal.\nPackages ship canonical `metrum-ai-router*` binaries only; older CLI names are\nsource-only exit-2 notices under `cmd/` and are not packaged. Fleet and\nmulti-environment customer CLI guidance lives in\n[docs/MULTI_ENVIRONMENT_DEPLOYMENT_CLI.md](/metrum-ai/router/blob/main/docs/MULTI_ENVIRONMENT_DEPLOYMENT_CLI.md)\nand [docs/CUSTOMER_INSTANCE_OPERATIONS_RUNBOOK.md](/metrum-ai/router/blob/main/docs/CUSTOMER_INSTANCE_OPERATIONS_RUNBOOK.md).\n\nPackaged Markdown is copied only from `scripts/package_docs_allowlist.txt`. That allowlist is limited to package-safe offline bootstrap docs such as `docs/PACKAGE_README.md`, quick install notes, validation notes, and the package-safe solution brief. Full customer/admin docs are built from `docs-site/` and served by the router under `/docs/`. Internal source-checkout runbooks such as `docs/DOCS_MAINTENANCE.md`, production runbooks, private host procedures, source-maintenance notes, and security-review working notes are intentionally not packaged. Private production runbooks, private host details, SSH paths, live compose config paths, raw token/provider-key patterns, AppleDouble `._*` metadata, and unexpected source/local-state files are blocked by package validation.\n\nThe `router` binary embeds the Docusaurus build output. At runtime, browser access to `/` redirects to `/docs/`; API and operations routes such as `/v1/*`, `/metrics`, `/admin/*`, `/healthz`, and `/readyz` keep precedence. Authenticated admin report assets, when enabled, are embedded separately under `/admin/reports/` and are not part of public Docusaurus docs.\n\nPublic product docs live under `docs-site/docs/` and are organized as an\noperator journey: overview, getting started, installation, self-managed\nlicensing, configuration, routing, providers and models, API compatibility,\nagents/tools/vision, usage and reports, security and governance, operations,\ntroubleshooting, evaluation, reference, and release/upgrade guidance.\n\nInternal operator and maintainer docs live under `docs/`. Use [docs/DOCS_MAINTENANCE.md](/metrum-ai/router/blob/main/docs/DOCS_MAINTENANCE.md) to decide which internal runbook owns each public section and which verification commands to run. Behavior changes affecting routing, auth, models, CLI/API usage, telemetry, deployment, licensing, security, or production operations normally require both public Docusaurus updates and matching internal/operator doc updates.\n\nDocker packages contain prebuilt image tarballs plus compose deployment assets:\n\n```\nimages/metrum-ai-router-<version>-linux-<arch>.tar\ncompose/docker-compose.yml\ncompose/docker-compose.postgres-localhost.yml\ncompose/Caddyfile.compose\ncompose/.env\ncompose/.env.example\nconfig/config.example.yaml\nconfig/env.example.json\nconfig/scripts/router.ts\ndocs/PACKAGE_README.md\ndocs/BINARY_INSTALL.md\ndocs/DOCKER_COMPOSE_INSTALL.md\ndocs/KUBERNETES_INSTALL.md\ndocs/PACKAGE_VALIDATION.md\ndocs/solution-brief.md\n```\n\nThe packaged config expects the routing script at `config/scripts/router.ts`, so the standard packaged run command is:\n\n```\nbin/metrum-ai-router --config config/config.yaml\n```\n\nSee `docs/DEPLOYMENT.md` for binary deployment guidance with Caddy TLS termination.\n\nFor Docker Compose deployments on AWS/EC2-style hosts, use `make package-docker` and follow `docs/DOCKER_DEPLOYMENT.md`. Docker packages include prebuilt image tarballs for linux/amd64 and linux/arm64, `docker-compose.yml`, an optional localhost-only Postgres override, Caddy config, router config templates, and docs; the target host does not need this source tree or a registry pull. Upgrades of an existing Compose install use `scripts/compose_package_upgrade.py` as documented in that runbook; do not unpack with a handwritten remote glob-move. A deliberate empty Compose Postgres usage store (keep config and caller keys; discard usage after restic) uses `scripts/compose_clean_cutover.py`.\n\nCreate a config from the example:\n\n```\ncp config.example.yaml config.yaml\ngo run ./cmd/metrum-ai-router-token-gen generate \\\n  --owner-user alice \\\n  --project example-project \\\n  --env dev \\\n  --allow <allowed-model-group>[,<allowed-model-group>...]\n```\n\nSave the printed `token` value as the caller's bearer token, add the owner to `users`, add the project to `projects`, add an active `project_memberships` row, and copy the generated `callers:` key entry into `config.yaml`. Tokens use a traceable public prefix plus a random secret suffix, while the router stores only `token_sha256` and logs/exports only `token_id`. Identity and authorization come from the explicit account sections and the key's `owner_user`/` project` references, not from parsing the token prefix. Each user id, project id, caller `id`, `token_sha256`, and non-empty `token_id` must be unique after normalization; token hashes are checked case-insensitively. User, project, and membership statuses support `active`, `disabled`, `suspended`, `removed`, and `archived`; caller key statuses also support `expired` and `rotated`.\n\nProvider keys are read from `env.json` in this project before `${VAR}` references in `config.yaml` are expanded. Real `env.json` is gitignored; use `env.example.json` as the placeholder-only template. Do not paste production or personal provider keys into tracked examples; store real values in ignored `env.json`, the shell environment, or your deployment secret manager. Run `make secret-check` before publishing changes that touch tracked env examples.\n\nRestic/backup secrets belong in ignored `ops.env.json` (see `ops.env.example.json`). Do not mix those into instance `env.json`; every deployment uses the same provider-only shape for instance secrets. Copy the example file locally (`cp ops.env.example.json ops.env.json`) and fill values; never commit the ignored runtime file.\n\n```\ngo run ./cmd/metrum-ai-router --config config.yaml\n```\n\nIf a variable is already set in the shell, the shell value wins over `env.json`. This lets CI or one-off live tests override local secrets without editing files.\n\nIn a packaged deployment, put provider keys in `config/env.json` beside `config/config.yaml`. The same loading rule applies: shell environment values win over `env.json`.\n\nAll Metrum AI Router first-party content is licensed under the Apache License 2.0. Copyright 2026 Metrum AI, Inc. The Apache license grants the rights to use, modify, and distribute those materials; no EULA acceptance or runtime-policy file is a condition of those rights.\n\nThe signed `license.json` described below is an operator-selected runtime policy\ninput. It can gate features or operational limits in a configured deployment,\nbut it is not the software's copyright license and does not restrict the rights\ngranted by Apache-2.0.\n\nNormal release builds enforce offline signed JSON runtime policy under\n`server.license`. Operators generate an Ed25519 keypair, issue `license.json`,\nand configure the paired public key. The router verifies the envelope at\nstartup and on `recheck_interval`, so operators can renew or replace the file\nwithout rebuilding the binary. Runtime YAML cannot disable enforcement in\nrelease builds; deployments should mount the license file read-only and keep\nthe license state file under the deployment state directory.\n\n```\nserver:\n  license:\n    enabled: true\n    path: /app/config/license.json\n    state_path: /app/state/license-state.json\n    instance_fingerprint: \"issued-instance-fingerprint\"\n    recheck_interval: 1h\n    grace_period_on_validation_error: 24h\n```\n\n`/readyz` fails when a required license blocks serving. Caller endpoints return documented `license-*` errors without exposing license payloads, signatures, or keys. Feature gates cover routing, usage reporting, admin reports, security reports, dynamic scoring, TypeScript routing, external policy routing, model-group contracts, retention rollups, and governed content-capture maintenance. Metrics-admin `/metrics` includes safe license gauges, and authorized admin report readers can query `/admin/license/status` for a safe summary only.\n\nSet `instance_fingerprint` only when the operator issues an instance-bound\nlicense for the deployment. It must match the licensed instance scope or\nstartup/readiness will fail with `license-instance-limit-exceeded`.\n\nUse `go run ./cmd/metrum-ai-router-license safe-summary --license license.json` to inspect safe license metadata. `metrum-ai-router-license verify --license license.json --public-key <public-key-file>` is for release/test validation with a supplied public key. Operators use `issue`, `renew`, and `top-up` with a local signing key and the SKU catalog as needed for their deployment. Private signing keys are not required at runtime and must never be copied into router config, logs, images, or source control.\n\nWhen the operator maintains a signed revocation bundle, configure\n`server.license.revocation.mode: file` and mount it at\n`server.license.revocation.path`. Effective `revoked`, `suspended`, or\n`superseded` entries block serving without license grace; `router-license revocation validate` and `revocation safe-summary` provide safe verification.\n\nSelf-managed issuance, renewal, trust rotation, and recovery are documented in\n[Self-Managed Licensing](/metrum-ai/router/blob/main/docs-site/docs/licensing/index.md). Keep private keys\noutside source control and outside runtime containers.\n\nProvider credential variables referenced by the current `config.example.yaml`:\n\n```\nANTHROPIC_API_KEY\nOPENAI_API_KEY\nMOONSHOT_API_KEY\nMINIMAX_API_KEY\nOPENROUTER_API_KEY\nXAI_API_KEY\nBASETEN_API_KEY\nCRUSOE_API_KEY\nFIREWORKS_API_KEY\n```\n\n`env.example.json` may contain additional empty placeholders for optional or\ndeployment-specific providers. An empty placeholder does not activate a\nprovider, prove account entitlement, or make a model eligible for routing.\nMetrum-local CTO package archival keys (`BACKUP_USER`, `BACKUP_PASS`,\n`RESTIC_PASSWORD`) belong only in ignored `ops.env.json` for\n`make dist-backup` / `make package-dist-backup`; leave the example values empty.\n\nProvider adapter notes:\n\n- `anthropic` targets call Anthropic Messages-compatible upstreams.\n- `openai-chat` and`openai-responses` targets call their respective\nOpenAI-compatible API shapes. A model passing one shape is not evidence for\nthe other.\n- Anthropic-compatible, Responses-compatible, and Chat-compatible skins for the same upstream belong in separate provider entries when their endpoints or request contracts differ.\n- `replicate` targets call Replicate Predictions and use an upstream`owner/model-name` identifier.\n- Enterprise-owned vLLM and SGLang services use the same OpenAI-compatible catalog shape and require direct plus router-level validation.\n\nActive provider/model status, capability evidence, and routing weights change\nmore frequently than this overview. Use `config.example.yaml` for the current\nreference catalog, authenticated `/v1/models` for a caller's allowed\ndeployment-defined groups, and deployment-owned validation records for live\nactivation status.\n\nThe router uses two different classes of keys:\n\n- Caller tokens authenticate clients that call this router. A caller sends `Authorization: Bearer <router-token>` or`X-API-Key: <router-token>` . The router hashes the presented token with SHA-256, compares it to configured`callers[].token_sha256` , checks`allow` , rate limits, traffic shaping, quotas, and lifetime token budget, then logs/exports only caller metadata and`token_id` . Config validation rejects duplicate caller`id` , duplicate`token_sha256` values case-insensitively, and duplicate non-empty`token_id` values.\n- Provider API keys authenticate the router to upstream LLM providers. They come from `providers.<name>.api_key` , usually via`${OPENAI_API_KEY}` ,`${OPENROUTER_API_KEY}` ,`${GROQ_API_KEY}` ,`${MOONSHOT_API_KEY}` , and similar values loaded from`env.json` or the shell. The router injects the selected provider key only when calling the selected upstream target.\n\n`callers[].allow` is the per-key allow list for internal router model group names. Model group names are deployment-defined; names such as `default`, `fast`, `small`, `medium`, `high`, `big-coder`, and `vision` are examples from the reference or hosted deployment, not product-required names. Disallowed model requests return `403 model-not-allowed` before provider routing and before any provider API key is used. The authenticated `/v1/models` response is filtered to the caller token's allowed groups.\n\nToken-budget admission reserves the estimated input tokens, tool/schema payload size, structured-output schema payload size, and the caller's requested output cap before an upstream call. Chat Completions uses `max_tokens` or `max_completion_tokens`, Responses uses `max_output_tokens`, and Messages uses `max_tokens`; Anthropic Messages requests without a caller cap reserve the router's injected default output cap. TPM, daily token, monthly token, and lifetime key budgets include in-flight reservations so concurrent large-cap or large-schema requests cannot overshoot the configured budget. Completed requests reconcile the reservation to actual reported usage, failed or canceled requests release it, and cache hits do not consume persisted token quota.\n\nOptional caller traffic shaping smooths short bursts separately from hard RPM/TPM/concurrency and quota checks. `server.traffic_shape.default_caller` can define inherited defaults, and `callers[].traffic_shape` can override or disable shaping for one key. Shaping can limit request starts, estimated input-token throughput, output reservations, and total reserved-token throughput before upstream calls. Deployments can enable bounded queueing with finite `max_wait_ms` and `max_depth` so brief bursts wait for bucket refill while oversized bursts still fail safely. Rejections return safe `429 traffic-shaped` responses with `Retry-After` and a bucket label; usage rows and `request_traffic_shape_events` store only scalar telemetry.\n\nProvider-side limits are pooled only through validated targets. A model group can spread compatible traffic across providers or private upstreams that have separate RPM, TPM, concurrency, quota, or serving envelopes, increasing effective usable capacity compared with a single hard-coded provider. Caller limits still run first, request-shape eligibility decides which targets count for each request, provider traffic shaping protects shared upstream accounts, and fallback is limited to retryable upstream failures.\n\nAdmin reports and `router-usage-report --traffic-tuning-advisor` can turn those safe shaping, upstream attempt, fallback, latency, and cancellation fields into conservative operator recommendations such as enabling queueing, increasing a specific burst/depth, slowing a client, tuning provider shared capacity, or routing around an incompatible upstream target. The advisor uses grouped SQL feature rows instead of report-side raw request-row scans; it is report-only and never modifies config.\n\nSet realistic output caps for each client workflow. Very large caps can be rejected near a token budget even when the prompt is small, because the router admits based on the maximum output the caller asked the upstream to generate. Request-count quotas are unchanged and still count admitted requests independently from token usage.\n\nRaw caller tokens, caller token hashes, and raw provider API keys are not exposed to TypeScript routing scripts, logs, metrics, or responses. Scripts get safe identifiers only: caller `id`, legacy-compatible `user`, canonical `ownerUser`/` username`, `project`, `environment`, public `tokenId`, membership role, key status, and target `keyId`, `apiKeyEnv`, and `keyConfigured`. This is enough to route by validated owner/project metadata or by the configured provider key name without making secrets available to script code.\n\n`config.example.yaml` is the canonical, validated provider/model catalog. Public\nconfiguration guidance lives in\n[`docs-site/docs/configuration/provider-catalog.md`](/metrum-ai/router/blob/main/docs-site/docs/configuration/provider-catalog.md),\nand the activation procedure lives in\n[`docs/onboard-model.md`](/metrum-ai/router/blob/main/docs/onboard-model.md). Keep provider-specific model\nIDs, capability evidence, prices, and source dates there instead of copying a\nsecond catalog into this README.\n\nA provider entry describes one upstream API skin and its reusable model metadata. A model-group target selects a catalog entry and owns the routing weight:\n\n```\nproviders:\n  example_chat:\n    base_url: https://provider.example.com/v1\n    dialect: openai-chat\n    auth_scheme: bearer\n    api_key: ${PROVIDER_API_KEY}\n    api_key_env: PROVIDER_API_KEY\n    key_id: example-chat-default\n    models:\n      example-model:\n        model: provider/example-model\n        input_modalities: [text]\n        output_modalities: [text]\n        tool_support:\n          openai_chat: [tools, tool_choice]\n\nmodels:\n  example-agent:\n    strategy: weighted\n    targets:\n      - provider: example_chat\n        model_ref: example-model\n        weight: 100\n```\n\nThe example names are illustrative; model-group names are deployment-defined.\n`model_ref` is local to its provider. Catalogs contain metadata only—routing\nweights belong under `models.<group>.targets[]`.\n\nThe router loads `env.json` beside the selected config file before expanding\n`${VAR}` references. An already-set process environment variable wins over the\nsame key in `env.json`. `api_key_env` and `key_id` are safe identifiers used for\neligibility and diagnostics; raw provider credentials remain server-side and\nmust not appear in config metadata, logs, reports, or docs.\n\nModel metadata controls eligibility:\n\n- `input_modalities` and`output_modalities` describe validated I/O.\n- `tool_support.openai_chat` ,`tool_support.openai_responses` , and`tool_support.anthropic_messages` are independent per-skin claims.\n- `structured_outputs` , reasoning controls, output-cap behavior, and bridge\nsupport must be declared only for the exact provider/model/dialect surface\nthat passed direct and router-level smokes.\n- pricing and modality metadata are copied into request-time usage records; historical reports use stored request-time costs rather than current prices.\n- `tool_only: true` on a target keeps it out of ordinary text routing. Tool\nrequests use only targets that preserve the caller's tool protocol and bypass\nresponse caching.\n\nCataloging a model does not activate it. Add a target to a restricted smoke group first, validate the exact Chat, Responses, Messages, streaming, tool, image, structured-output, output-cap, and large-payload shapes it will serve, then promote it through the deployment's quality/cost contract. On a partial failure, remove or narrow the failed capability metadata instead of treating a pass on one API skin as evidence for another.\n\nSelf-hosted vLLM and SGLang services use the same catalog and activation rules.\nConfirm the served `/v1/models` ID, parser/chat-template settings, direct\ncompletion and tool behavior, and the equivalent router requests before adding\nthem to a caller-facing group. See\n[`docs/SELF_HOSTED_UPSTREAMS.md`](/metrum-ai/router/blob/main/docs/SELF_HOSTED_UPSTREAMS.md).\n\nFor agent traffic, a group can mix ordinary and tool-only targets:\n\n```\nmodels:\n  example-coding:\n    strategy: weighted\n    targets:\n      - { provider: example_chat, model_ref: example-model, weight: 100 }\n      - provider: example_messages\n        model_ref: example-model\n        weight: 100\n        tool_only: true\n```\n\nNon-tool requests ignore `tool_only` targets. Tool-bearing requests still pass\nrequest-shape eligibility, so the router fails with `no-eligible-target` rather\nthan sending a tool payload through an unvalidated dialect or bridge.\n\nUse `strategy: dynamic_score` when a deployment wants one configurable per-group policy instead of separate hardcoded strategies for cheap-fast routing, latency-aware routing, workload complexity, budget pressure, or evaluation-backed quality preferences. Callers still request a model group they are allowed to use. The router authenticates the caller, validates that group access, filters only that group's `targets[]` for API dialect, tool support, modalities, and explicit max-token safety, then scores only the remaining targets in that same group.\n\n```\nmodels:\n  adaptive-agent:\n    strategy: dynamic_score\n    targets:\n      - { provider: baseten, model_ref: gpt-oss-120b, weight: 60, tags: [validated, coding, tool_capable] }\n      - { provider: minimax, model_ref: m3, weight: 25, tags: [validated, low_cost, tool_capable] }\n      - { provider: openai, model_ref: gpt-5.4-nano, weight: 5, tags: [fallback] }\n    routing_policy:\n      dynamic_score:\n        cold_start_policy: configured_weight\n        min_observations: 20\n        observation_window_seconds: 600\n        max_score_adjustment_percent: 70\n        hard_filters:\n          require_requested_api_skin: true\n          require_input_modalities: true\n          require_tool_support_when_tools_present: true\n          require_honors_max_tokens_when_caller_capped: true\n        signals:\n          request_shape: { enabled: true }\n          prompt_features:\n            enabled: true\n            max_scan_bytes: 16384\n            features: [code, diff, stack_trace, summarize, extract, security_review, tool_agent]\n          complexity: { enabled: true }\n          observed_performance: { enabled: true }\n          cost: { enabled: true }\n          evaluation_metadata: { enabled: true }\n        score_terms:\n          - name: cheapest_fast_enough\n            when: { complexity_lte: standard }\n            expression: \"0.45 * cost_score + 0.25 * latency_score + 0.20 * throughput_score + 0.10 * reliability_score\"\n          - name: complex_quality_floor\n            when: { complexity_gte: complex }\n            require_tags: [validated]\n            expression: \"0.45 * eval_quality_score + 0.25 * reliability_score + 0.20 * latency_score + 0.10 * cost_score\"\n        thresholds:\n          max_error_rate: 0.03\n          max_timeout_rate: 0.02\n          max_p95_latency_ms: 10000\n```\n\nCold start is deterministic: until `min_observations` is reached, targets are ordered by configured group-local weight. After that, the router uses in-memory rolling observations for latency, throughput, error rate, timeout rate, and fallback rate; it does not read the usage database on the hot path. Decision traces contain only safe scalar metadata such as enabled signal names, request-shape buckets, candidate count, selected provider/model, score bucket, observation count, and cold-start mode. They must not contain raw prompts, images, tool outputs, router tokens, token hashes, provider keys, or full upstream headers.\n\nWhen decision telemetry is enabled, usage/admin reports expose safe dynamic-score, policy, fallback, and admission buckets for operations: enabled signal names, score/value/final-score buckets, threshold/filter buckets, policy outcomes/error classes, fallback transition reasons, max-token cap filtering, max-token buckets, input-token buckets, and quota/admission reasons. Request usage rows also store non-secret routing/model-group/policy/pricing fingerprints for reproducibility after config or pricing changes. Daily rollups preserve report-critical buckets in normalized scalar rows so commercial reports can outlive raw request-level detail retention.\n\nRollout should start on a deployment-defined test group with interchangeable validated targets. Use mock or local router smokes for simple text, code/debug prompts, tool calls, forced tool calls, image requests when supported, structured-output requests when supported, and low output caps for each caller API. Roll back by switching the group strategy to `weighted` or by removing score terms and thresholds that are too strict for the workload.\n\nFor structured-output rollout, smoke both Chat Completions `response_format` and Responses `text.format` if both dialects are configured. Also run a negative router smoke against a group with no structured-output-capable target and expect `502 no-eligible-target` with no upstream attempt. If a target claims both tools and structured outputs, include a combined request in rollout validation. Same-dialect Chat and Anthropic streaming proxy native upstream SSE incrementally; OpenAI Responses and cross-dialect bridges retain unary upstream calls with synthesized caller SSE. Schema-constrained incremental chunks remain provider-specific.\n\nFor reasoning routing, see the Docusaurus [Reasoning Routing](/metrum-ai/router/blob/main/docs-site/docs/configuration/reasoning-routing.md) guide and the operator [Smoke Test Matrix](/metrum-ai/router/blob/main/docs/SMOKE_TEST_MATRIX.md). Explicit OpenAI Chat `reasoning_effort`, OpenAI Responses `reasoning`, and Anthropic Messages `thinking` requests must use targets with validated reasoning metadata inside the requested group; ordinary traffic can still use the group's ordinary eligible weighted mix.\n\nFor OpenAI Chat tool clients, for example Warp Agent, configure the client with:\n\n```\nBase URL: https://your-router.example.com/v1\nAPI key: <router caller token>\nModel: <allowed-model-group>\n```\n\nUse whichever deployment-defined model group the caller token allows. If a request includes `tools`, structured-output fields, images, or explicit output caps and no eligible target in that group declares the required support, the router returns `502 no-eligible-target` with a hint to enable an upstream target that supports the requested dialect, tools, structured outputs, modalities, and cap behavior.\n\nModel groups may declare an optional `contract` that makes the group’s workload, API surfaces, hard capability requirements, validation quality floor, and operational thresholds first-class config. Existing groups without a contract behave as before. Contract enforcement is strictly group-local: after authentication and caller allow-list checks, the router filters only the requested group’s already eligible targets, then runs `static`, `weighted`, `failover`, `dynamic_score`, `script`, or `external` on the remaining targets.\n\n```\nmodels:\n  support-chat:\n    strategy: weighted\n    contract:\n      display_name: Support chat\n      caller_visible_notes: Deployment-defined low-latency support group.\n      intended_workloads: [support_chat]\n      supported_api_shapes: [openai_chat]\n      required_capabilities:\n        input_modalities: [text]\n        output_modalities: [text]\n        honors_max_tokens_when_caller_capped: true\n      quality_floor:\n        require_tags: [validated]\n        min_eval_quality_score: 0.90\n        min_eval_pass_rate: 0.95\n        max_eval_age_days: 30\n        allowed_validation_status: [passed]\n      operational_targets:\n        max_p95_latency_ms: 10000\n        max_error_rate: 0.03\n        max_timeout_rate: 0.02\n      reporting:\n        expose_workload_labels: true\n        expose_quality_floor_bucket: true\n    targets:\n      - provider: private-gpu\n        model_ref: support-balanced\n        weight: 70\n        tags: [validated, low_latency]\n        validation:\n          status: passed\n          workload: support_chat\n          validated_at: \"2026-06-25\"\n          quality_score: 0.94\n          pass_rate: 0.98\n          harness: golden-support-set\n      - provider: hosted\n        model_ref: support-fallback\n        weight: 30\n        tags: [validated, fallback]\n        validation:\n          status: passed\n          workload: support_chat\n          validated_at: \"2026-06-25\"\n          quality_score: 0.92\n          pass_rate: 0.96\n          harness: golden-support-set\n```\n\nStartup validation rejects unsupported API shapes, invalid modalities, impossible validation status values, bad dates, out-of-range quality scores/pass rates, negative thresholds, required tags that no target has, declared API shapes that no target serves, and contracts no target can satisfy. Runtime contract failures return the existing `502 no-eligible-target` style response with safe buckets such as `contract-required-api-shape`, `contract-required-modality`, `contract-quality-floor`, `contract-validation-expired`, or `contract-no-validated-target`.\n\n`dynamic_score` can use target `tags` and `validation.quality_score`/` validation.pass_rate` as evaluation hints. TypeScript and external policy strategies receive the same safe contract and target validation metadata after contract filtering, and returned decisions are validated against the filtered target list. Usage rows store only scalar contract buckets, optional workload labels, and target validation status/workload/age buckets.\n\nRoll out contracts on a deployment-defined test group first. Add validation metadata to each intended target, run text/tool/image/structured-output smokes that match the declared contract, confirm no-eligible failures use safe reason buckets, and verify reports show only safe scalar buckets. Roll back by removing or relaxing the `contract` block, removing a too-strict quality floor, or switching the group back to its previous strategy/weights.\n\nFor providers that use Anthropic Messages shape but bearer-token authentication, set `auth_scheme: bearer`:\n\n```\nproviders:\n  minimax_anthropic:\n    base_url: https://api.minimax.io/anthropic\n    dialect: anthropic\n    auth_scheme: bearer\n    api_key: ${MINIMAX_API_KEY}\n    api_key_env: MINIMAX_API_KEY\n    models:\n      m3: { model: MiniMax-M3, tier: heavy }\n  kimi_anthropic:\n    base_url: https://api.moonshot.ai/anthropic\n    dialect: anthropic\n    auth_scheme: bearer\n    api_key: ${MOONSHOT_API_KEY}\n    api_key_env: MOONSHOT_API_KEY\n    models:\n      kimi-k2.7-code: { model: kimi-k2.7-code, tier: heavy }\n  openrouter_anthropic:\n    base_url: https://openrouter.ai/api\n    dialect: anthropic\n    auth_scheme: bearer\n    api_key: ${OPENROUTER_API_KEY}\n    api_key_env: OPENROUTER_API_KEY\n    models:\n```\n\nUse `strategy: script` on a model group and point `script` at a TypeScript file:\n\n```\nmodels:\n  default:\n    strategy: script\n    script: scripts/router.ts\n    targets:\n      - { provider: baseten, model_ref: gpt-oss-120b, weight: 60 }\n      - { provider: minimax, model_ref: m3, weight: 30 }\n      - { provider: kimi, model_ref: kimi-k2.7-code, weight: 10 }\n```\n\nThe script must export `route(ctx)` and return one configured target by index or by `{ provider, model }`. Proxy users still request a deployment-defined model group name; the script chooses one backing target from that group's configured `targets`.\n\nThe script context uses top-level `ctx.text` for normalized request text, plus `ctx.group`, `ctx.request`, `ctx.caller`, and `ctx.targets`. Caller metadata includes `id`, legacy-compatible `user`, canonical `ownerUser`/` username`, `project`, `environment`, public `tokenId`, `membershipRole`, `keyStatus`, and the key allow list. Target metadata includes provider, model, modelRef, baseUrl, dialect, weight, keyId, apiKeyEnv, and keyConfigured. For groups with `pii_filter`, `ctx.text`, normalized request fields, and `ctx.request.raw` are redacted before the script runs, and placeholder mappings are not exposed. Raw provider API keys, raw caller tokens, and caller token hashes are never passed to scripts; returned targets are validated against the configured list. Scripts run synchronously inside the router process, so keep policy local and fast; unrestricted network calls and file access are not part of the script runtime.\n\nRelative TypeScript imports are bundled at router startup, so a script can use local helpers such as `import { scorePrompt } from \"./policy\"`. Keep deployment-owned helpers next to the script, for example `config/scripts/router.ts`, `config/scripts/policy.ts`, and `config/scripts/scoring.ts`.\n\nThird-party dependencies must be installed, locked, and packaged before deployment. The router bundles from the deployment filesystem at startup; it does not run `npm install`, download packages, or resolve network dependencies at runtime. For npm-based policy helpers, manage dependencies under the script directory, package `package.json`, the lockfile, and the resolved dependency tree or a pre-bundled script artifact, and keep that tree free of provider keys, router tokens, and private host credentials. For large dependencies or native modules, prefer pre-bundling during release and deploying the generated entrypoint.\n\nExternal policy calls are opt-in per model group through deployment config:\n\n```\nmodels:\n  default:\n    strategy: script\n    script: scripts/router.ts\n    script_http:\n      enabled: true\n      allow_hosts: [routing-policy.internal.example]\n      timeout_ms: 200\n      max_response_bytes: 65536\n      headers:\n        Authorization: ${ROUTING_POLICY_AUTH_HEADER}\n    targets:\n      - { provider: baseten, model_ref: gpt-oss-120b, tier: cheap, weight: 70 }\n      - { provider: minimax, model_ref: m3, tier: heavy, weight: 30 }\n```\n\nScripts call external policy with `router.fetchJSON(url, options)`, not browser `fetch`. The helper supports `GET` and `POST`, JSON request bodies, JSON responses, script-supplied headers limited to `Accept`, `Content-Type`, and `X-*`, and only hosts in the model group's `script_http.allow_hosts`. HTTPS is required by default. Plain HTTP is accepted only for loopback hosts such as `localhost`, `127.0.0.1`, and `::1`, or when `script_http.allow_http: true` is set for a trusted non-local policy service. Redirects are followed only when each hop keeps an allowed `http`/` https` scheme and an exact allowlisted hostname. Put policy-service auth in deployment config with `script_http.headers`, for example `Authorization: ${ROUTING_POLICY_AUTH_HEADER}`, rather than in script source. `timeout_ms` is capped at `5000`; use smaller values for routing policy because it runs before the upstream model request.\n\nA demo PII-aware routing policy lives in `examples/typescript-pii-policy/`. It detects common PII-like patterns in `ctx.text`, sends matching requests only to configured targets marked `sensitive` or `private`, restricts retry fallbacks to those same sensitive/private targets, sends non-matching requests to a normal target, and returns only safe class labels such as `pii-detected:sensitive-route` or `pii-detected:none`. If PII is detected and no sensitive/private target is eligible, the demo fails closed with `pii-detected:no-sensitive-target`. This is routing only: TypeScript scripts do not redact outbound request content. Use model-group `pii_filter` when the router must redact, restore, or fail requests before upstream calls.\n\nSee [Learned Routing Policy](#learned-routing-policy) for the outcome-trained\nservice. Use `strategy: external` when routing policy should live in a standalone web service instead of in TypeScript. The router sends safe derived request context, safe caller metadata, safe contract metadata when configured, eligible target metadata, validation metadata, pricing, tool capability metadata, and modalities to the configured policy URL, then validates the returned target against the model group's eligible targets. By default the policy payload does not include prompt text, message bodies, image URLs/data, tool schemas, tool outputs, or `request.raw`; route on fields such as `context.textChars`, `context.estimatedTokens`, `context.imageCount`, and `context.toolCount`. Set `external_policy.include_request: true` only for a trusted policy service that is allowed to receive request content. With `pii_filter`, that opt-in request mirror is redacted before dispatch and placeholder mappings are not sent. Raw router tokens, token hashes, and provider API keys are never sent.\n\n```\nmodels:\n  adaptive:\n    strategy: external\n    external_policy:\n      url: https://routing-policy.internal.example/route\n      allow_hosts: [routing-policy.internal.example]\n      timeout_ms: 500\n      max_response_bytes: 65536\n      headers:\n        Authorization: ${ROUTING_POLICY_AUTH_HEADER}\n      on_error: fail_closed\n      include_request: false\n    targets:\n      - { provider: baseten, model_ref: gpt-oss-120b, tier: cheap, weight: 70 }\n      - { provider: minimax, model_ref: m3, tier: heavy, weight: 30 }\n```\n\nPolicy responses use the same selector shape as TypeScript: `targetIndex` or `target`, optional `fallbackIndexes`/` fallbacks`, and optional `classLabel`. Class labels are telemetry labels, not content fields: keep them to short tokens using letters, numbers, `_`, `-`, `.`, and `:`. Unsafe, long, or content-like labels are stored as `unsafe_class_label` before logs or usage rows are written. When a TypeScript script omits `fallbackIndexes` and `fallbacks`, remaining eligible targets are used as retries; when either field is present, the supplied entries are the complete retry set. The default `on_error` behavior is `fail_closed`, returning `502 routing-policy-error`; `fallback` can be configured when the target order is an acceptable default. A runnable demo service lives at `examples/external-routing-policy/prompt_size_policy.py`.\n\n`examples/external-routing-policy/outcome_calibrated_policy.py` is a separate\ntested reference for outcome-calibrated routing. It uses deployment-owned\nexemplar classes, an OpenAI-compatible embedding endpoint, and reviewed outcome\nrecords to emit a reviewable external-policy profile and target-weight patch.\nIt never changes active configuration automatically; unmatched work uses a\ndeployment-designated strong default. The reference requires\n`external_policy.include_request: true` and therefore belongs only in trusted\ninfrastructure, with redacted input when the model group uses `pii_filter`.\n\nExternal policy URLs use the same egress rules as `router.fetchJSON`: HTTPS by default, plaintext HTTP only for loopback hosts or with `external_policy.allow_http: true`, exact-host allowlisting, and redirect revalidation on every hop. A redirect to a host outside `allow_hosts`, including a loopback address that was not explicitly allowed, fails before the redirected service is reached.\n\nNon-loopback private/link-local IPs and nondefault nonlocal ports are also denied. Private policy sidecars use loopback in the router's network namespace.\n\nThe default `scripts/router.ts` does three things:\n\n- Removes targets whose provider key is not configured or whose target weight is zero.\n- Applies named regex rules against safe caller-key metadata and safe target-key metadata.\n- Falls back to weighted random routing across eligible targets, using group target weights as relative probabilities.\n\nFor checked-in request-shape routing (prompt size, tools, images, structured\noutputs, and reasoning signals), use\n`examples/typescript-request-shape/router.ts` instead of assuming\n`scripts/router.ts` implements that policy.\n\nModel groups can configure `pii_filter` rules to replace matched text with typed placeholders before target selection, cache-key generation, routing-policy inputs, and upstream provider calls. The redacted request object is the source of truth for TypeScript script `ctx.request.raw` and for external policy `request`/` text` only when `external_policy.include_request: true` is explicitly enabled; external policy services otherwise receive safe derived context without raw request mirrors. Modes support `redact_only`, `redact_and_restore`, and `fail_on_match`. Response restoration applies to buffered responses; same-dialect native Chat and Anthropic streams preserve placeholders because safely restoring mappings that may span arbitrary SSE chunks would require buffering and defeat native streaming. If a request exceeds `max_replacements_per_request`, the router fails closed with `pii-filter-blocked` before any upstream call. Usage logs and the usage database store only safe scalar metadata such as applied flag, mode, replacement count, and matched-rule count; raw matched values and placeholder mappings remain in memory for the request lifecycle by default.\n\nSee `docs/PII_FILTERING.md` and the Docusaurus PII Filtering page for configuration examples and smoke-test guidance.\n\nPrompt-size / request-shape routing example (`examples/typescript-request-shape/router.ts`):\n\n```\ntype Target = {\n  provider: string;\n  model: string;\n  tier?: string;\n  weight: number;\n  keyConfigured: boolean;\n};\n\ntype RouteContext = {\n  text: string;\n  targets: Target[];\n};\n\nexport function route(ctx: RouteContext) {\n  const eligible = ctx.targets\n    .map((target, index) => ({ target, index }))\n    .filter((entry) => entry.target.keyConfigured && entry.target.weight > 0);\n\n  if (eligible.length === 0) {\n    return { targetIndex: 0, classLabel: \"prompt-size:no-eligible-targets\" };\n  }\n\n  const preferredTier = ctx.text.length > 8000 ? \"heavy\" : \"cheap\";\n  const preferred = eligible.find((entry) => entry.target.tier === preferredTier) || eligible[0];\n\n  return {\n    targetIndex: preferred.index,\n    fallbackIndexes: eligible\n      .filter((entry) => entry.index !== preferred.index)\n      .map((entry) => entry.index),\n    classLabel: `prompt-size:${preferredTier}`,\n  };\n}\n```\n\nCaller metadata enables owner-, project-, environment-, or key-class routing without exposing secrets. Target metadata also lets the script route to targets backed by a specific configured provider key identifier or environment variable name:\n\n```\nexport function route(ctx) {\n  if (\n    /^alice$/.test(ctx.caller?.ownerUser || ctx.caller?.username || \"\") &&\n    /^example-project$/.test(ctx.caller?.project || \"\")\n  ) {\n    const heavyIndex = ctx.targets.findIndex((target) =>\n      target.tier === \"heavy\" &&\n      (\n        (/^openrouter-default$/.test(target.keyId || \"\") &&\n          /^OPENROUTER_API_KEY$/.test(target.apiKeyEnv || \"\")) ||\n        (/^openai-default$/.test(target.keyId || \"\") &&\n          /^OPENAI_API_KEY$/.test(target.apiKeyEnv || \"\"))\n      ) &&\n      target.keyConfigured\n    );\n    if (heavyIndex >= 0) {\n      return { targetIndex: heavyIndex, classLabel: \"key-regex:prod-heavy\" };\n    }\n  }\n  return { targetIndex: 0, classLabel: \"default\" };\n}\n```\n\n`ctx.caller.ownerUser` and `ctx.caller.project` come from validated config references. `ctx.caller.tokenId` is the generated public token id without the secret suffix, for example `rtr_metrum_alice_example-project_prod_key1`; use it for traceable key classes, not identity. Do not route on raw token secrets; the router never passes them to scripts.\n\nCheck which names are present without printing secret values:\n\n```\nenv | grep -E '^(ANTHROPIC|OPENAI|MOONSHOT|KIMI|QWEN|MINIMAX|OPENROUTER|REPLICATE|XAI)_.*=' | sed 's/=.*/=***REDACTED***/'\n```\n\nHealth checks:\n\n```\ncurl http://127.0.0.1:8080/healthz\ncurl http://127.0.0.1:8080/readyz\ncurl -H \"Authorization: Bearer $METRICS_ADMIN_ROUTER_TOKEN\" http://127.0.0.1:8080/metrics\n```\n\n`/metrics` is intentionally restricted to caller subjects authorized for `metrics` `read`; existing caller entries with `metrics_admin: true` receive equivalent Casbin grants at startup. Normal application keys receive `403 metrics-forbidden`. Rejected or unknown model names are reported under bounded labels such as `rejected_model` rather than caller-supplied model text. A temporary migration-status lookup failure keeps authorized scrapes available and emits `smart_llmrouter_migration_status_available{scope=\"usage\"} 0`; it never emits database error text or connection details. Use `/v1/usage` and durable usage reports for caller-scoped usage views.\n\nBrowser-admin HTTP Basic authentication is configured under `server.admin_auth.basic` and is disabled by default. When enabled, `GET /admin/auth/check` validates the first admin identity path: missing or invalid Basic credentials receive `401`, valid credentials without the stub permission receive `403 admin-forbidden`, and valid credentials with `admin:auth:read` receive safe subject metadata. Basic Auth establishes identity such as `basic:admin`; it does not grant broader admin permissions by itself. See [docs/ADMIN_AUTH.md](/metrum-ai/router/blob/main/docs/ADMIN_AUTH.md).\n\nAdministrator browser reports can be enabled under `server.admin_reports` and are served under `/admin/reports/`. They require browser-admin identity plus Casbin policy under `server.admin_auth.authorization`, expose safe usage/performance/cost/savings/cache/fallback/routing/capability/anomaly/troubleshooting/request-drilldown data through a Metrum-branded dark dashboard with a two-row admin header, filters for caller ID, caller IP, requested model, provider/model/dialect, status, cache, project, group, and client, plus shared search, sorting, URL state, spreadsheet-safe CSV export, and a safe version chip populated from `/admin/reports/api/version`. Dedicated troubleshooting tabs include Upstream failures, Shape failures, Fallback health, and User impact, using sanitized upstream error details plus safe request-shape and translation buckets so operators can identify failing provider/model/dialect/status combinations, incompatible request shapes, fallback recovery rate, and affected users or clients without SQL. Request evidence is available through `/admin/reports/api/request-evidence?request_id=<request_id>` and path-style request drilldown; it joins safe normalized rows and reports diagnostic completeness for present, not-applicable, and missing sections. Request and security-event APIs return cursor pagination metadata with stable ordering and filtered totals; aggregate report APIs explicitly return top-N metadata when they intentionally serve bounded ranked summaries. Report data is scoped to the admin's Casbin domain unless an explicit `*` policy domain grants deployment-wide visibility. The browser also includes safe provider catalog/validation status and retention/rollup status tabs, and keeps HTML/Markdown-escaped Markdown export available at `/admin/reports/export.md`; the default detail mode is bounded to recent matching request rows, while `/admin/reports/export.md?mode=summary` renders SQL-backed full-window totals with bounded top-N aggregate sections and no raw request rows. Optional security access reports under `server.admin_reports.security` persist safe scalar access events and require separate `admin:security_reports` policy. Savings baselines under `server.admin_reports.baselines` are source-dated hypothetical comparison prices; actual cost is summed from stored request-time usage rows. See [docs/AUTHORIZATION.md](/metrum-ai/router/blob/main/docs/AUTHORIZATION.md) for metrics, content-capture maintenance, security report, and report policy examples. Ordinary router caller tokens receive `403 reports-forbidden`.\n\nAuthorization policy can come from a deployment-owned file/inline config with `server.admin_auth.authorization.source: static`, or from a validated active policy set in the usage DB with `source: db`. DB-backed policy mode fails closed when no single valid active policy set exists, keeps static policy support intact, and records create, activation, rollback, and validation-failure audit events with safe scalar fields only.\n\nVersion checks:\n\n```\n./router --version\n./router-token-gen --version\n./router-usage-report --version\ncurl http://127.0.0.1:8080/version\n```\n\n`/healthz`, `/readyz`, and `/version` include the router version, commit, and full UTC build timestamp. Metrics-admin `/metrics` exports `smart_llmrouter_build_info`. Browser docs show the running docs package version and build timestamp in the in-page banner, while public docs HTTP responses deliberately omit `X-Smart-LLMRouter-*` build-identity headers. OpenAI-compatible `/v1/*` response bodies do not include router-specific version fields.\n\nUsage is written to both JSONL and a GORM-backed relational database. SQLite is the default for local and new generic Docker Compose/Kubernetes installations; it is created with private `0600` file modes, including sidecars when present. The generic container path is one writer using `/app/state/usage.sqlite`; PostgreSQL is an explicit multi-replica or externally managed database choice. The schema is scalar and relational only: no JSONB, JSON, array, or packed multi-value DB columns.\n\nWhen license enforcement is enabled, request logs and `request_usage` store only safe scalar license metadata such as status, reason, license ID, customer ID, SKU, key ID, expiry, and grace-active flag. They do not store the license payload, detached signature, public/private key bytes, or signing material.\n\nSafe request-shape and provider-translation rows are written with usage when usage DB persistence is enabled; they do not depend on optional decision telemetry. Additional diagnostics such as attempt, trace, upstream-shaping, sanitized upstream-error detail, and terminal-error rows are written when `server.diagnostics.enabled` is true. Each request can have child rows in `request_attempts`, `request_trace_events`, `request_traffic_shape_events`, `request_upstream_shape_events`, `request_shapes`, `request_translation_shapes`, `request_translation_field_events`, `request_upstream_error_details`, and `request_errors`, all keyed by `request_id`. Use the `X-Request-Id` response header or the `request_id` in an error body to join these rows during incident response or query `/admin/reports/api/request-evidence?request_id=<request_id>` with a drilldown-authorized admin for a safe completeness-scored bundle. Diagnostic rows store provider/model/status/timing/error-class data, safe caller and upstream shaping bucket decisions, inbound request-shape counts/buckets/fingerprints, translated upstream attempt-shape counts/buckets, bridge direction (`chat_to_responses` or `responses_to_chat`), translated reasoning-control field, and bounded provider 4xx/5xx fields such as `code`, `type`, `param`, request ID, and categorized provider message when `store_sanitized_upstream_errors` is not explicitly disabled. This setting defaults to true with diagnostics and can be set false to suppress provider detail rows. Chat-to-Responses stateful session activity is recorded with bounded trace event names such as `bridge_session_lookup_hit`, `bridge_session_lookup_miss`, `bridge_session_set`, `bridge_session_delete`, `bridge_session_backend_error`, `bridge_session_previous_response_stale_purged`, and `bridge_session_stateless_retry`. Diagnostics do not store raw prompts, images, image URLs, tool schemas, tool outputs, bearer tokens, provider keys, token hashes, Redis credentials, full upstream headers, raw upstream response bodies, raw session headers, or free-form provider prose. `store_sanitized_upstream_errors` is not content capture: arbitrary upstream bodies are collapsed to a redaction marker, and prompt-like fields, nested upstream bodies, free-form messages, and secret-shaped values are redacted or categorized before JSONL or usage DB persistence.\n\nGoverned content capture is a separate opt-in feature under `server.content_capture`. It is disabled by default and writes redacted request, response, and upstream-error content to `request_content_captures` plus allowlisted headers in `request_content_headers`, both joinable to `request_usage` by `request_id`. Admin delete and retention purge write `request_content_audit_events` and require a caller subject authorized for `content:capture` `delete|purge`; delete-by-request also requires delete authorization in the captured row's caller project/environment domain before rows are removed. Existing caller entries with `content_admin: true` receive equivalent Casbin grants at startup for their own domain. Metrics-admin tokens do not grant content maintenance access. The first slice always requires `redact_before_storage: true`, rejects forbidden header names such as authorization/API-key/token headers, and requires `encryption.enabled: true` with a configured `local_key_id` plus `CONTENT_CAPTURE_LOCAL_KEY` (32-byte hex/base64 local AES key material) when capture is enabled.\n\nCommercial retention policy is configured under `server.retention`. Defaults are conservative with `dry_run: true`; status jobs store active policy versions and rules, legal-hold rows, retention jobs, and per-table counts for `usage_diagnostics`, `decision_telemetry`, `security_access_events`, `content_capture`, and `usage_detail`. Legal holds match by `data_class`, optional `request_id`, and timestamp range. When a reviewed config sets `dry_run: false`, `router-usage-report --retention-run` deletes at most one configured batch per supported table for `usage_diagnostics` and `usage_detail`; other data classes are counted and recorded as blocked. `usage_detail` deletion is blocked unless finalized daily usage rollups continuously cover the candidate window, preserving immutable #122 rollup history. Archive/export, scheduler, admin UI/API workflows, and broader data-class purge execution remain future slices.\n\nUpstream timing is configurable with `server.upstream.timeout_ms`, `server.upstream.default_attempt_timeout_ms`, model-group `attempt_timeout_ms`, and per-target `timeout_ms`. A target timeout overrides a group timeout, and a group timeout overrides the global default attempt timeout. `0` disables the per-attempt cap while preserving the global HTTP client timeout. Successful upstream response bodies are bounded by `server.upstream.max_response_bytes` before decode, cache, or synthesized streaming. Upstream redirects are not followed by the router, so a provider cannot replay request bodies to a different `Location`. If all eligible attempts fail, exhausted upstream timeouts return `504 upstream-timeout`, provider rate limits return `503 upstream-rate-limited`, provider balance/credit/quota/billing exhaustion returns `503 upstream-quota-exhausted`, provider/model/target shared-capacity shaping returns `503 upstream-capacity-throttled`, and other exhausted upstream failures return `502 upstream-failed`. Fallbacks are attempted only after retryable upstream failures such as timeout, network error, rate limit, provider quota/billing exhaustion, or 5xx. Ordinary non-retryable 4xx upstream policy or malformed-request errors stop fallback to avoid replaying the same caller payload to another provider.\n\nEach request row stores the configured input/output price per million tokens for the selected upstream model, the pricing source/update date, and calculated input/output/total USD cost. These values are logged at request time instead of recalculated during reporting, so historical cost reports remain stable after upstream providers change pricing.\n\nThe JSONL file is useful for raw audit/debugging. The relational DB is the source for periodic reports. In container deployments using SQLite, use `/app/logs/requests.jsonl` and `/app/state/usage.sqlite`. In Postgres deployments, the report tool reads from the configured DSN.\n\n`router-usage-report` CLI Markdown instants use UTC RFC3339 with fixed milliseconds (`YYYY-MM-DDTHH:mm:ss.SSSZ`) in the `Period UTC` bounds and `Per-Request Throughput` time column. Hour and day summary labels remain reporting buckets (`YYYY-MM-DD HH:00` and `YYYY-MM-DD`); JSONL, database, browser Markdown exports, browser JSON APIs, and CSV exports retain their existing precision.\n\nFor routine browser inspection, deployments may enable `/admin/reports/`. The browser report UI is disabled by default, embedded in the router binary, uses local Metrum logo/font/chart assets, shows authenticated build metadata from `/admin/reports/api/version`, and calls bounded JSON APIs over the same relational usage DB. The CLI remains the supported path for automation, incident exports, and headless workflows.\n\n`router-usage-report` flags:\n\n```\n--driver NAME   Usage DB driver: sqlite or postgres; defaults to sqlite.\n--db PATH       SQLite usage DB path; defaults to usage.sqlite.\n--dsn DSN       Postgres DSN when --driver=postgres.\n--log PATH      Optional JSONL request log to import before reporting.\n--since DUR     Relative period when --from is omitted, such as 24h, 7d, or 30d.\n--from TIME     Start time, RFC3339, YYYY-MM-DD HH:MM:SS, or YYYY-MM-DD.\n--to TIME       End time; defaults to now.\n--out PATH      Markdown output path; defaults to stdout.\n--token-id ID   Filter to one public router token id.\n--token-id-prefix PREFIX\n                Filter to public router token ids with this prefix.\n--caller-user USER\n                Filter to one caller owner user.\n--caller-project PROJECT\n                Filter to one caller project.\n--caller-environment ENV\n                Filter to one caller environment.\n--resolved-group GROUP\n                Filter to one resolved router model group.\n--client CLIENT Filter to one client, such as codex or claude-code.\n--rollup        Generate relational rollup rows for the selected period instead of markdown.\n--rollup-type TYPE\n                Rollup granularity: hourly, daily, or monthly. Defaults to daily.\n--rollup-finalize\n                Mark the selected rollup window immutable after generation.\n--baseline-id ID\n                Optional savings baseline id to store on rollup rows.\n--baseline-name NAME\n                Optional savings baseline name to store on rollup rows.\n--baseline-version VERSION\n                Optional savings baseline version/source date to store on rollup rows.\n--baseline-input-price-per-million-usd USD\n                Optional baseline input price in USD per million tokens.\n--baseline-output-price-per-million-usd USD\n                Optional baseline output price in USD per million tokens.\n--retention-status\n                Record a dry-run retention status job from --config.\n--retention-run Run retention from --config; deletes one batch per supported table only when config dry_run=false.\n--config PATH   Router config path for --retention-status or --retention-run.\n```\n\nGenerate a markdown report for the last 24 hours:\n\n```\n./router-usage-report --driver sqlite --db /app/state/usage.sqlite --since 24h --out usage-24h.md\n```\n\nGenerate a report from Postgres:\n\n```\n./router-usage-report \\\n  --driver postgres \\\n  --dsn \"$ROUTER_USAGE_DB_DSN\" \\\n  --since 24h \\\n  --out usage-24h.md\n```\n\nGenerate a report for an explicit period and import existing JSONL first. Imports are duplicate-safe by router `request_id`:\n\n```\n./router-usage-report \\\n  --driver sqlite --db /app/state/usage.sqlite \\\n  --log requests.jsonl \\\n  --from 2026-06-14T00:00:00Z \\\n  --to 2026-06-15T00:00:00Z \\\n  --out usage-2026-06-14.md\n```\n\nGenerate a report from the default Docker Compose SQLite deployment:\n\n```\ndocker compose run --rm --no-deps --entrypoint /app/bin/metrum-ai-router-usage-report router \\\n  --driver sqlite \\\n  --db /app/state/usage.sqlite \\\n  --since 24h \\\n  --out /app/logs/usage-24h.md\n```\n\nFor an explicitly configured PostgreSQL deployment, substitute `--driver postgres --dsn \"$ROUTER_USAGE_DB_DSN\"`.\n\nGenerate a report for one benchmark or case study by caller project/environment:\n\n```\n./router-usage-report \\\n  --driver postgres \\\n  --dsn \"$ROUTER_USAGE_DB_DSN\" \\\n  --caller-project harbor-algotune-pca \\\n  --caller-environment case-current-policy-20260615t004637z \\\n  --out harbor-agentic-usage.md\n```\n\nRecord a dry-run retention status job from reviewed router config:\n\n```\n./router-usage-report --retention-status --config config.example.yaml\n```\n\nRun one reviewed retention batch after finalized rollups and legal holds have been checked:\n\n```\n./router-usage-report --retention-run --config config.production.yaml\n```\n\nReports include totals, external provider/model usage, internal router API key usage by `token_id`/owner user/project/environment, caller IP usage, hourly usage by caller IP, client usage, status codes, cache hit/miss/bypass, attempts, fallbacks, token totals, latency, downstream user performance, upstream provider/model/dialect performance, per-request upstream/downstream output-token/sec, per-request upstream/downstream total-token/sec, contract pass/fail buckets, optional contract workload labels, target validation buckets, and cache occupancy snapshots. Raw router tokens and provider API keys are never written to the report.\n\nDurability:\n\n- Durable across container restarts when volumes are preserved: JSONL request logs, relational usage DB rows, per-request throughput fields, and per-request cache snapshots.\n- Not durable across container restarts: in-memory response cache contents and in-process Prometheus counters/gauges.\n\n```\nmake test       # safe default suite: Go, policy, and offline API compatibility tests\nmake docs-build # build customer-facing Docusaurus docs into the Go embed directory\nmake docs-dev   # run the Docusaurus development server\nmake build      # build docs, then ./router, ./router-token-gen, ./router-usage-report, and ./router-migrate\nmake build-go-only # build Go binaries without refreshing embedded docs\nmake build-all  # build docs, then linux amd64 and linux arm64 binaries under dist/build\nmake package    # build linux amd64 and linux arm64 tarballs\nmake package-all # same as package\nmake docker-image # build one metrum-ai-router image for GOOS/GOARCH with docker buildx\nmake package-docker # build linux amd64 and linux arm64 Docker packages\nmake package-docker-all # same as package-docker\nmake e2e-mock   # local mock Claude/Codex C harness\nmake e2e-live-c # live OpenRouter :nitro C-generation e2e through Claude Code and Codex\nmake e2e-live-full # live provider HTTP cache checks plus live CLI C e2e\nmake e2e-compose-live # live provider + Claude/Codex checks through docker compose and Caddy\n```\n\nThe Harbor agentic coding case-study example in `examples/harbor-algotune-pca/` uses `uv tool install harbor`, runs Harbor's `aider/polyglot_python_two-bucket` task through Codex CLI and Claude Code, and emits a markdown usage comparison report. Current production Harbor runs use one reusable Harbor caller token with access to the deployed model groups, then separate results by run matrix, client, model group, timestamps, and usage-report filters. The older per-`{agent, model_group}` token generator remains available only for isolated local or one-off investigations. The current production run is checked in at `docs/harbor-case-study.md`.\n\n`make e2e-live-c` starts the router once per OpenRouter sample target, runs both local CLIs, extracts the generated C source, compiles it with `cc -std=c11 -Wall -Wextra -Werror`, and runs the binary. It reads the project `env.json` before invoking the router. To keep logs and generated C files:\n\n```\nKEEP_LIVE_E2E_WORKDIR=1 make e2e-live-c\n```\n\nTo run one live case:\n\n```\nLIVE_E2E_CASE_REGEX=baseten-gpt-oss-120b make e2e-live-c\n```\n\nThe following commands were tested locally with `Claude Code 2.1.177`, `codex-cli 0.139.0`, router port `18081`, and deployment-defined model groups. Provider-backed smokes require the relevant provider keys in the project `env.json`.\n\nCLI install/update references:\n\n```\n# Codex CLI, official standalone installer/update path:\ncurl -fsSL https://chatgpt.com/codex/install.sh | sh\n\n# Codex CLI, npm install/update path:\nnpm install -g @openai/codex@latest\n\n# Claude Code, npm install/update path:\nnpm install -g @anthropic-ai/claude-code@latest\n```\n\nOn 2026-06-14, the local installs matched the latest npm registry versions: `@openai/codex` `0.139.0` and `@anthropic-ai/claude-code` `2.1.177`.\n\nCreate a temporary router config and caller token:\n\n```\nmake build\n\nexport WORK=/tmp/smart-llmrouter-readme-smoke\nrm -rf \"$WORK\"\nmkdir -p \"$WORK\"\n\n./router-token-gen generate \\\n  --owner-user readme \\\n  --project example-project \\\n  --env dev \\\n  --allow cli-smoke \\\n  --format json > \"$WORK/token.json\"\n\npython3 - <<'PY'\nimport json\nimport os\nimport shlex\nfrom pathlib import Path\n\nwork = Path(os.environ[\"WORK\"])\ngenerated = json.loads((work / \"token.json\").read_text())\n(work / \"token.env\").write_text(\n    f\"ROUTER_TOKEN={shlex.quote(generated['token'])}\\n\"\n    f\"ROUTER_MODEL=cli-smoke\\n\"\n)\n(work / \"config.yaml\").write_text(f\"\"\"server:\n  listen: \":18081\"\n  cache: {{ enabled: false }}\n  logging:\n    path: {work}/requests.jsonl\nstate_path: {work}/state.json\nproviders:\n  baseten:\n    base_url: https://inference.baseten.co/v1\n    dialect: openai-chat\n    api_key: ${{BASETEN_API_KEY}}\n    api_key_env: BASETEN_API_KEY\n    key_id: baseten-readme-smoke\nmodels:\n  cli-smoke:\n    strategy: static\n    targets:\n      - {{ provider: baseten, model: \"openai/gpt-oss-120b\" }}\nusers:\n  - id: readme\n    name: README Smoke\n    type: service_account\n    status: active\nprojects:\n  - id: example-project\n    name: Example Project\n    status: active\nproject_memberships:\n  - user_id: readme\n    project: example-project\n    role: developer\n    status: active\ncallers:\n  - id: readme-example-project-dev\n    owner_user: readme\n    project: example-project\n    environment: dev\n    status: active\n    token_sha256: \"{generated['token_sha256']}\"\n    token_id: \"{generated['token_id']}\"\n    allow: [\"cli-smoke\"]\n    rate: {{ rpm: 120, tpm: 200000, concurrent: 4 }}\n    quota:\n      day: {{ requests: 1000, tokens: 2000000 }}\n      month: {{ tokens: 10000000 }}\n      soft_pct: 80\n    key: {{ lifetime_tokens: 10000000, soft_pct: 90, on_exhaust: disable }}\n\"\"\")\nPY\n```\n\nStart the router in one terminal. This form intentionally reads the project `env.json` for the smoke test, so a stale shell variable does not override the tested provider key:\n\n```\nexport WORK=/tmp/smart-llmrouter-readme-smoke\n\nOPENROUTER_API_KEY=$(python3 - <<'PY'\nimport json\nfrom pathlib import Path\nprint(json.loads(Path(\"env.json\").read_text())[\"OPENROUTER_API_KEY\"])\nPY\n) ./router --config \"$WORK/config.yaml\"\n```\n\nThen run the CLI checks in another terminal:\n\n```\nexport WORK=/tmp/smart-llmrouter-readme-smoke\n\nset -a\n. \"$WORK/token.env\"\nset +a\n```\n\nExample hosted deployment model groups:\n\n```\nsmall      Baseten GPT OSS 120B 58%, MiniMax-M3 28%, Gemma 4%, Kimi 4%, Baseten Nemotron 3%, Baseten GLM 2%, OpenAI GPT-5.4 Nano 1% non-tool.\nmedium     Baseten GPT OSS 120B 51%, MiniMax-M3 25%, Gemma 7%, Kimi 8%, Baseten Nemotron 3%, Baseten GLM 5%, OpenAI GPT-5.4 Nano 1% non-tool.\nhigh       Baseten GPT OSS 120B 45%, MiniMax-M3 26%, Gemma 9%, Kimi 10%, Baseten Nemotron 3%, Baseten GLM 6%, OpenAI GPT-5.4 Nano 1% non-tool; ordinary non-MiniMax targets are request-shape capped at 1 MiB until gt-1mb OpenAI Chat tool payloads pass target-specific validation.\ndefault    Baseten GPT OSS 120B 51%, MiniMax-M3 27%, Gemma 7%, Kimi 6%, Baseten Nemotron 3%, Baseten GLM 5%, OpenAI GPT-5.4 Nano 1% non-tool.\nfast       Baseten GPT OSS 120B 56%, MiniMax-M3 26%, Gemma 4%, Kimi 5%, Baseten Nemotron 3%, Baseten GLM 5%, OpenAI GPT-5.4 Nano 1% non-tool.\nbig-coder  Reasoning-capable code-heavy route: Fireworks GPT OSS 20B 25%, MiniMax M3 Responses 25%, xAI Grok 4.5 15%, Fireworks DeepSeek-V4-Flash 15%, MiniMax M3 Chat 5%, Kimi K2.7 Code 5%, Crusoe GLM 5.2 5%, and OpenAI GPT-5.4 Nano 5% for ordinary text and compatible Chat/Responses traffic; Fireworks GPT OSS 20B and xAI Grok 4.5 handle Chat reasoning and Anthropic-thinking translation, MiniMax M3 Responses handles Responses reasoning, Fireworks Responses Kimi K2.7 Code remains available for Codex/Responses tool traffic, and MiniMax/Kimi Anthropic-compatible targets remain available for Claude Code-style tool traffic. Request-shape metadata can further filter those weights; production-derived opencode/AI SDK Chat `stream_options` requests are gated away from incident-backed Chat targets until exact smokes pass.\n```\n\nThe key used in `ROUTER_TOKEN` must allow the selected `ROUTER_MODEL`. The group names shown above are example hosted deployment names; your deployment can expose different names and access tiers.\n\nClaude Code uses Anthropic-style requests. For this router, set `ANTHROPIC_BASE_URL` to the Anthropic-compatible namespace and set `ANTHROPIC_AUTH_TOKEN`. Do not set `ANTHROPIC_API_KEY` for router traffic; Claude Code uses that variable for direct Anthropic Console API keys via `X-Api-Key`, while this router expects a bearer token.\n\n```\nunset ANTHROPIC_API_KEY\nexport ANTHROPIC_BASE_URL=\"http://127.0.0.1:18081/anthropic\"\nexport ANTHROPIC_AUTH_TOKEN=\"$ROUTER_TOKEN\"\nexport ANTHROPIC_MODEL=\"$ROUTER_MODEL\"\n\nclaude --bare --print --model \"$ROUTER_MODEL\" \\\n  \"Reply with exactly: router claude ok\"\n```\n\nExpected output:\n\n```\nrouter claude ok\n```\n\nExpected log fields include `client=claude-code`, `inbound_dialect=anthropic`, `requested_model=cli-smoke`, and a concrete target provider/model. Provider keys must not appear in output or logs.\n\nContainerized tool smokes:\n[Coding-Agent Client Matrix](/metrum-ai/router/blob/main/docs-site/docs/getting-started/coding-agent-clients.md#containerized-tool-smokes).\n\nCodex is configured with ephemeral provider settings and the OpenAI Responses wire API:\n\n`/v1/codex/models.json` is a Responses-specific, caller-filtered catalog. A model group appears only when it has an ordinary text target eligible for OpenAI Responses, either natively or through a validated `responses_to_chat` bridge. Its advertised tools and reasoning are limited to that same surface. Image metadata is emitted only when the exact group also has an eligible mixed text/image Responses path; a bridge must explicitly validate and enable images, and request-shape image exclusions suppress the claim. A Chat-only or Anthropic-only target is still available to its matching API surface but is not advertised to Codex.\n\n```\nexport METRUM_ROUTER_KEY=\"$ROUTER_TOKEN\"\numask 077\ncurl -fsS \"http://127.0.0.1:18081/v1/codex/models.json\" \\\n  -H \"Authorization: Bearer $METRUM_ROUTER_KEY\" \\\n  -o \"$WORK/metrum-models.json\"\nmkdir -p \"$WORK/codex-work\"\n\ncodex exec --ignore-user-config --ephemeral \\\n  --ignore-rules \\\n  --skip-git-repo-check \\\n  -C \"$WORK/codex-work\" \\\n  -c \"model=\\\"$ROUTER_MODEL\\\"\" \\\n  -c 'model_provider=\"metrum-ai-router\"' \\\n  -c \"model_catalog_json=\\\"$WORK/metrum-models.json\\\"\" \\\n  -c 'model_providers.metrum-ai-router.name=\"Metrum AI Router\"' \\\n  -c 'model_providers.metrum-ai-router.base_url=\"http://127.0.0.1:18081/v1\"' \\\n  -c 'model_providers.metrum-ai-router.env_key=\"METRUM_ROUTER_KEY\"' \\\n  -c 'model_providers.metrum-ai-router.wire_api=\"responses\"' \\\n  \"Reply with exactly: router codex ok\" </dev/null\n```\n\nThe `exec` subcommand is required for `--ignore-user-config`, `--ephemeral`, `--ignore-rules`, and `--skip-git-repo-check`; those flags are not accepted by the top-level interactive `codex` command.\n\nFor interactive Codex, omit the `exec`-only flags and run top-level `codex` with the same provider settings:\n\n```\nexport METRUM_ROUTER_KEY=\"$ROUTER_TOKEN\"\n\ncodex \\\n  -c \"model=\\\"$ROUTER_MODEL\\\"\" \\\n  -c 'model_provider=\"metrum-ai-router\"' \\\n  -c \"model_catalog_json=\\\"$WORK/metrum-models.json\\\"\" \\\n  -c 'model_providers.metrum-ai-router.name=\"Metrum AI Router\"' \\\n  -c 'model_providers.metrum-ai-router.base_url=\"http://127.0.0.1:18081/v1\"' \\\n  -c 'model_providers.metrum-ai-router.env_key=\"METRUM_ROUTER_KEY\"' \\\n  -c 'model_providers.metrum-ai-router.wire_api=\"responses\"'\n```\n\nExpected final assistant output:\n\n```\nrouter codex ok\n```\n\nExpected log fields include `client=codex`, `inbound_dialect=openai-responses`, `requested_model=cli-smoke`, and no leaked credentials. A local Codex installation may print a bubblewrap/user-namespace warning; that is separate from the router request and does not indicate provider failure.\n\nContainerized Codex tool smoke:\n[Coding-Agent Client Matrix](/metrum-ai/router/blob/main/docs-site/docs/getting-started/coding-agent-clients.md#containerized-tool-smokes).\n\nTool-bearing requests bypass the router response cache. They are intentionally routed to the provider every time because tool calls depend on external filesystem, shell, and agent state.\n\n```\ngo test ./...\ngo build ./cmd/metrum-ai-router\n```\n\nThe automated suite uses deterministic mock upstreams. The Claude Code and Codex commands above are the live provider acceptance gates.\n\nFull release validation is live and credit-consuming:\n\n```\nmake e2e-live-full\nmake e2e-compose-live\n```\n\nThese require live provider keys in `env.json` or the shell plus locally installed `claude`, `codex`, Docker, and Docker Compose.\n\n- **Community** : this Apache-2.0 repository. No license key is required by\ndefault. You may use, modify, and distribute the software under Apache-2.0\nwithout payment; Enterprise offerings never condition those Apache rights on\npayment.\n- **Enterprise** : a separate distribution with production validation, named\nsupport, signed releases, and related commercial entitlements.\n\nPrivacy defaults for this software: do not retain prompts or responses by default; do not train on traffic; provider keys stay in your deployment.\n\nSee [Architecture, Platforms, And Limitations](/metrum-ai/router/blob/main/docs-site/docs/reference/architecture-limitations.md)\nfor the full operator-owned boundary list. Streaming accuracy in brief: native\nincremental SSE applies to same-dialect Chat Completions and Anthropic\nMessages; OpenAI Responses and cross-dialect bridges synthesize SSE after a\nunary upstream response.\n\nThe repository-root [LICENSE](/metrum-ai/router/blob/main/LICENSE) contains the Apache License 2.0 terms\nfor Metrum AI Router first-party content. Keep it together with\n[NOTICE](/metrum-ai/router/blob/main/NOTICE), the dependency and asset inventory in\n[THIRD_PARTY_NOTICES.md](/metrum-ai/router/blob/main/THIRD_PARTY_NOTICES.md), and the model-term boundaries\nin [MODEL_LICENSES.md](/metrum-ai/router/blob/main/MODEL_LICENSES.md) when copying or redistributing a\nrelease. Third-party components, assets, and models remain governed by their\nown applicable terms; the Apache-2.0 license does not replace them. See\n[TRADEMARKS.md](/metrum-ai/router/blob/main/TRADEMARKS.md) for mark usage.\n\nOptional signed `license.json` is a separate operator runtime-policy input for\nfeature and operational enforcement when enabled. It is not the software\ncopyright license and not a replacement for any of the files above. See\n[docs/LICENSE.md](/metrum-ai/router/blob/main/docs/LICENSE.md) for the complete scope map and Apache terms.\n\nCurrent MVP capabilities:\n\n- Config-driven model groups with decision-making strategies first: `dynamic_score` , TypeScript`script` , and`external` policy services (including Learned Routing Policy). Optional model-group`contract` gates and request-shape eligibility filtering run before strategy selection. Fallback stays inside the requested group.\n- External policy `shadow` ,`enforce` , and`baseline` modes provide explicit promotion and rollback while selection remains limited to post-eligibility targets.\n- Dynamic-score conversation affinity is enabled by default, caller-isolated, process-local, TTL-bounded, and subordinate to current eligibility.\n- TypeScript routing scripts for custom model-selection logic inside the Go router, with fresh per-decision VMs, a bounded per-group concurrency cap, and cancellation-aware admission.\n- External routing policy services for standalone web-service target selection with safe request, caller, target, pricing, tool, and modality context, connection reuse, bounded timeouts, and reversible baseline/shadow/enforce modes.\n- Conservative traffic-mix strategies remain available: `static` ,`weighted` , and`failover` .\n- `strategy: intelligent` is an experimental baseline-only LLM-selector scaffold (shadow/simulate only; it does not alter target selection). Legacy`latency` ,`cost` , and`semantic` are compatibility-only stubs. See[Deprecated Selectors](/metrum-ai/router/blob/main/docs-site/docs/reference/deprecated-selectors.md) .\n\n- Anthropic Messages, OpenAI Chat Completions, and OpenAI Responses ingress.\n- Anthropic token-count estimate endpoint for Claude Code startup.\n- Bearer-token auth using configured SHA-256 token hashes.\n- Separate caller dialects from upstream provider adapters: callers can use\nAnthropic/OpenAI wire formats while targets route to Anthropic,\nOpenAI-compatible, Replicate, or evidence-gated unary-text Gemini\n`generateContent` adapters.\n- Optional bounded `targets[].region` metadata for selected-target diagnostics;\ndeployments remain responsible for residency enforcement and provider\nvalidation.\n- Server-side provider key injection.\n- Same-dialect OpenAI Chat and Anthropic Messages native SSE proxying with incremental delivery, plus unary upstream proxying and caller-dialect response encoding for OpenAI Responses and cross-dialect bridges.\n- Committed native streams never fall back after the first event; caller cancellation stops the upstream request.\n- In-process LRU+TTL cache for eligible unary responses.\n- Per-caller RPM, TPM, concurrency, traffic shaping, rolling quota, and lifetime key budget enforcement.\n- Disk-persisted quota/key state.\n- JSONL request logs using the SRS schema.\n- Metrics-admin-only Prometheus-compatible `/metrics` with caller/user/project labels.\n- Usage reports, admin browser reports, and optional signed runtime-policy licensing.", "url": "https://wpnews.pro/news/reduce-token-cost-with-metrum-ai-router-routing-intelligence-you-can-train", "canonical_source": "https://github.com/metrum-ai/router", "published_at": "2026-09-12 14:39:20+00:00", "updated_at": "2026-09-12 15:17:09.481334+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "large-language-models", "ai-agents", "mlops"], "entities": ["Metrum AI Router", "Metrum AI", "Codex", "big-coder", "Harbor", "OpenAI", "Learned Routing Policy", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/reduce-token-cost-with-metrum-ai-router-routing-intelligence-you-can-train", "markdown": "https://wpnews.pro/news/reduce-token-cost-with-metrum-ai-router-routing-intelligence-you-can-train.md", "text": "https://wpnews.pro/news/reduce-token-cost-with-metrum-ai-router-routing-intelligence-you-can-train.txt", "jsonld": "https://wpnews.pro/news/reduce-token-cost-with-metrum-ai-router-routing-intelligence-you-can-train.jsonld"}}