Backstage plugin for LiteLLM governance — enables developers to manage virtual API keys and monitor AI model usage directly from Backstage.
This is a Backstage 1.50+ plugin providing a governance interface for LiteLLM proxy. It includes:
Frontend: React components built with the New Frontend System (@backstage/frontend-plugin-api
)Backend: Express router using the New Backend System (@backstage/backend-plugin-api
)
packages/plugin-litellm
- Frontend (
@acarmisc/backstage-plugin-litellm
)packages/plugin-litellm-backend
- Backend (
@acarmisc/backstage-plugin-litellm-backend
)
The LiteLLMHomeWidget
card surfaces the signed-in user's KPIs (USD spent, tokens in/out, active key count) alongside a daily-spend sparkline, with a Today
/ 7d
/ 30d
period selector.
The Overview
tab is the default landing view. It shows the user's identity header (with team chips and key health counters), four KPI tiles (Total Spend, Total Requests, Success Rate, Total Tokens), and the Costs
chart with per-model daily spend.
The full-page version below shows the complete chart grid (Daily Spend by Model, Daily Token Usage, Daily Requests, Daily Success Rate, Cumulative Spend vs Budget):
Browse, edit, block, and revoke your virtual keys. Each row shows alias, key ID, creation/expiry dates, budget bar, TPM/RPM, and the models it can call.
Mint a scoped key with its own budget, team binding, model list, and TPM/RPM caps. Failures (e.g. duplicate alias) surface inline with the upstream error.
Browse every model the proxy exposes, with per-model input/output cost and max input/output token limits. The team filter scopes the list to the team the key will be bound to.
A smaller variant of the home widget for surfaces that only need a one-click shortcut into the key-mint flow.
This plugin is designed to be used within a Backstage monorepo. It uses workspace dependencies and requires the Backstage CLI to build.
From your Backstage monorepo root:
yarn add file:../backstage-govai/packages/plugin-litellm
yarn add file:../backstage-govai/packages/plugin-litellm-backend
Copy the packages directly into your Backstage plugins/
directory and add them to your workspace.
Set these in your shell or deployment environment before starting Backstage. Backstage's config system supports ${ENV_VAR}
substitution in app-config.yaml
:
LITELLM_BASE_URL=http://litellm-proxy:4000 # LiteLLM proxy URL
LITELLM_MASTER_KEY=sk-... # LiteLLM admin master key
Add to app-config.yaml
. All keys live under the litellm
top-level namespace:
litellm:
baseUrl: ${LITELLM_BASE_URL}
masterKey: ${LITELLM_MASTER_KEY}
userIdDomain: example.com # optional
provisioning:
enabled: false # default
defaults:
maxBudget: 10 # default: 10
budgetDuration: 30d # default: "30d"
models: [] # default: [] (all models)
teams: [] # default: [] (no teams)
userRole: internal_user # default: "internal_user"
roles:
- group: group:default/ai-power-users # Backstage group entity ref
maxBudget: 100
budgetDuration: 30d
models:
- gpt-4o
- claude-3-5-sonnet
userRole: internal_user
keyGeneration:
allowUnlimitedBudget: false # default
teamRequired: true # default
Config key reference:
| Key | Type | Required | Default | Description |
|---|---|---|---|---|
litellm.baseUrl |
||||
| string | yes | — | LiteLLM proxy base URL | |
litellm.publicBaseUrl |
||||
| string | no | — | Publicly reachable proxy URL for snippet generation | |
litellm.masterKey |
||||
| string | yes | — | Admin master key (@visibility secret ) |
|
litellm.userIdDomain |
||||
| string | no | — | Email domain for LiteLLM user IDs | |
litellm.provisioning.enabled |
||||
| boolean | no | false |
||
| Enable autoprovisioning | ||||
litellm.provisioning.defaults.maxBudget |
||||
| number | no | 10 |
||
| Max spend (USD) per reset period | ||||
litellm.provisioning.defaults.budgetDuration |
||||
| string | no | "30d" |
||
| Spend-reset period | ||||
litellm.provisioning.defaults.models |
||||
| string[] | no | [] |
||
| Allowed model IDs (empty = all) | ||||
litellm.provisioning.defaults.teams |
||||
| string[] | no | [] |
||
| Team IDs to join on creation | ||||
litellm.provisioning.defaults.userRole |
||||
| string | no | "internal_user" |
||
| LiteLLM role | ||||
litellm.provisioning.defaults.tpmLimit |
||||
| number | no | — | Tokens-per-minute cap | |
litellm.provisioning.defaults.rpmLimit |
||||
| number | no | — | Requests-per-minute cap | |
litellm.provisioning.defaults.metadata |
||||
| object | no | {} |
||
| Extra metadata on user record | ||||
litellm.provisioning.roles[].group |
||||
| string | yes* | — | Backstage group entity ref | |
litellm.provisioning.roles[].maxBudget |
||||
| number | no | — | Overrides default for group | |
litellm.provisioning.roles[].budgetDuration |
||||
| string | no | — | Overrides default for group | |
litellm.provisioning.roles[].models |
||||
| string[] | no | — | Overrides default for group | |
litellm.provisioning.roles[].teams |
||||
| string[] | no | — | Overrides default for group | |
litellm.provisioning.roles[].userRole |
||||
| string | no | — | Overrides default for group | |
litellm.provisioning.roles[].tpmLimit |
||||
| number | no | — | Overrides default for group | |
litellm.provisioning.roles[].rpmLimit |
||||
| number | no | — | Overrides default for group | |
litellm.provisioning.roles[].metadata |
||||
| object | no | — | Merged over default metadata | |
litellm.keyGeneration.allowUnlimitedBudget |
||||
| boolean | no | false |
||
| Show the "Unlimited budget" checkbox in the Generate New Key form | ||||
litellm.keyGeneration.teamRequired |
||||
| boolean | no | true |
||
| Require a team to be selected before a key can be generated |
*required when the roles
array is present
In packages/backend/src/index.ts
:
backend.add(import('@acarmisc/backstage-plugin-litellm-backend'));
The plugin uses the Backstage New Frontend System. Add the plugin package as an extension in packages/app/src/App.tsx
or equivalent:
import { litellmPlugin, LiteLLMPage } from '@acarmisc/backstage-plugin-litellm';
// Add the route:
<Route path="/litellm" element={<LiteLLMPage />} />
You can also register it as a plugin extension using the New Frontend System.
LiteLLMHomeWidget
is a compact card you can drop onto any Backstage homepage. It shows the signed-in user's data — identity is resolved server-side from the Backstage Bearer token, so no userId
prop is required and no additional backend endpoint is needed.
KPIs displayed: USD spent · Tokens in · Tokens out · Key count, plus a daily-spend sparkline (hidden when there is no daily data). A small period selector (Today
/ 7d
/ 30d
) lives in the card header.
import { LiteLLMHomeWidget } from '@acarmisc/backstage-plugin-litellm';
// In your HomePage composition:
<LiteLLMHomeWidget defaultPeriod="7d" />
Props:
| Prop | Type | Default | Description |
|---|---|---|---|
defaultPeriod |
|||
| `'today' | '7d' | '30d'` | |
'7d' |
|||
| Period shown on first render | |||
title |
|||
string |
|||
'LiteLLM Usage' |
|||
| Card title override |
The widget requires the same backend setup as the full LiteLLMPage
(backend plugin configured and the user provisioned in LiteLLM).
When litellm.provisioning.enabled
is true
, the backend automatically creates a LiteLLM user the first time a Backstage user hits any plugin endpoint (user info, keys, teams, or usage). The flow is:
- The backend resolves the caller's Backstage identity from the request token (
user:default/<name>
). - It checks whether that identity already exists in LiteLLM via
/user/info
. - If not found, it looks up the user's Backstage catalog entity to fetch their profile (email, display name) and group memberships.
- It applies any matching
provisioning.roles
override (first match wins), then calls/user/new
on LiteLLM with the effective defaults. - A concurrent single-flight lock prevents duplicate
/user/new
calls when several endpoints fire in parallel on the same page load.
Backstage catalog prerequisites:
- The user
must exist as a. The catalog is the source of truth for email, display name, and group memberships.
User
entity in the Backstage catalog - Group memberships (used for role matching) are resolved from the
memberOf
relations on the user entity. These are typically populated by a catalog provider such as the LDAP, GitHub, or Microsoft Graph org provider. - If
userIdDomain
is set, the entity name (e.g.john.doe
fromuser:default/john.doe
) is combined with the domain to produce the LiteLLMuser_id
(e.g.john.doe@example.com
). Make sure LiteLLM users were created with matching IDs if you are migrating an existing deployment. - If a user signs in without a catalog entity (e.g.
dangerouslyAllowSignInWithoutUserInCatalog
is set), provisioning still proceeds but the LiteLLM user record will lack email, display name, and team-role resolution — they will receive the default settings.
Minimum working example with autoprovisioning enabled:
litellm:
baseUrl: ${LITELLM_BASE_URL}
masterKey: ${LITELLM_MASTER_KEY}
provisioning:
enabled: true
defaults:
maxBudget: 5
budgetDuration: 30d
Build from within your Backstage monorepo:
yarn workspace @acarmisc/backstage-plugin-litellm build
yarn workspace @acarmisc/backstage-plugin-litellm-backend build
yarn workspace @acarmisc/backstage-plugin-litellm test
For frontend-only iteration:
cd packages/plugin-litellm
yarn start
The backend provides the following endpoints (all prefixed with /api/litellm
).
A machine-readable OpenAPI 3.1 contract is served at /api/litellm/openapi.json
— point any OpenAPI-compatible renderer (Stoplight, Swagger UI, Redoc) at it instead of maintaining this table by hand.
| Endpoint | Method | Purpose |
|---|---|---|
/health |
||
| GET | Health check | |
/config |
||
| GET | Public LiteLLM proxy base URL (for snippet generation) | |
/openapi.json |
||
| GET | OpenAPI 3.1 contract for this backend surface | |
/user/info |
||
| GET | Get current user info and quotas | |
/keys |
||
| GET | List user's virtual keys | |
/keys/generate |
||
| POST | Generate a new virtual key | |
/keys/:keyId |
||
| DELETE | Revoke/delete a virtual key (caller must own it) | |
/keys/:keyId/update |
||
| POST | Update alias / models / budget / limits (caller must own it) | |
/keys/:keyId/block |
||
| POST | Suspend a key without revoking it (caller must own it) | |
/keys/:keyId/unblock |
||
| POST | Re-enable a blocked key (caller must own it) | |
/keys/:keyId/reset_spend |
||
| POST | Zero out a key's spend counter (caller must own it) | |
/models |
||
| GET | List available LLM models | |
/teams |
||
| GET | List teams the current user belongs to | |
/teams/:teamId/usage |
||
| GET | Usage metrics for a team (start_date , end_date required) |
|
/usage |
||
| GET | Get usage metrics and analytics for the current user | |
/audit |
||
| GET | Audit logs (gated by litellm.audit.group membership) |
|
/provisioning/preview |
||
| GET | Resolve which role a Backstage group maps to (dry-run, audit-group-gated) |
The UI endpoints above authenticate via the Backstage identity system. The
CLI bridge endpoints below are gated behind litellm.bridge.enabled
and authenticate with a raw Keycloak access token instead (see CLI Bridge):
| Endpoint | Method | Purpose |
|---|---|---|
/bridge/health |
||
| GET | Bridge health + configured clientId (no auth) |
|
/bridge/keys |
||
| GET | List the caller's virtual keys | |
/bridge/keys |
||
| POST | Mint a virtual key for the caller | |
/bridge/models |
||
| GET | List available LLM models |
The bridge lets CLI clients (the Abby CLI) list and mint LiteLLM virtual keys without ever holding the LiteLLM master key. The Backstage backend keeps the master key (as it already does for the UI); the CLI authenticates with its Keycloak access token — the same realm Backstage uses.
Request flow for /api/litellm/bridge/*
:
- CLI sends
Authorization: Bearer <keycloak-access-token>
. - The bridge verifies the JWT against the realm JWKS (
createRemoteJWKSet
), checking the issuer and that the token was issued for the configuredclientId
(viaazp
, falling back toaud
). Failure →401
. - The caller is resolved to a LiteLLM
user_id
(email → preferred_username → sub) and ensured to exist — provisioned from the JWT claims iflitellm.provisioning.enabled
, otherwise404
(log in to Backstage once first). - Keys are listed/minted via the existing master-key-authed client. Minted
keys are stamped with ownership metadata (
created_via: abby-cli
,created_by
,created_at_iso
).
Unlike the UI routes, bridge routes do not call Backstage's
auth.authenticate
— they verify the raw Keycloak JWT themselves.
Configuration (app-config.yaml
):
litellm:
bridge:
enabled: true # default false
issuer: https://auth.example.com/realms/solution-innovation # required when enabled
clientId: abby-cli # default abby-cli
When enabled
is true but issuer
is missing the backend fails fast at startup; the bridge routes are not mounted otherwise.
-
Built with React and Material-UI
-
Uses Backstage API client for backend communication
-
Components:
LiteLLMPage -
Main plugin page
DashboardHeader -
Header with user context
KeysTable -
Display and manage virtual keys
UsageStats -
Usage analytics with date range selector
TeamUsage -
Team-specific usage breakdown
-
Express-based router
-
Communicates with LiteLLM proxy API
-
Handles authentication via Backstage identity system
-
Provides user context resolution and API proxying
Key Management: Generate, view, and revoke virtual API keys** Usage Analytics**: Track API usage with configurable date ranges (today, 7 days, 30 days)** Team Context**: Optional team-based key generation and usage tracking** User Info**: Display user quotas and current usage limits** Model Selection**: Browse available LLM models configured in LiteLLM** At-a-glance Dashboard**: The profile header shows the current user, team membership, a live counter of total / expired / expiring-soon keys, and a one-click "Generate New Key" shortcutInline Generation Errors: Key-generation failures (e.g. a duplicate alias) surface directly in the dialog, with a client-side warning when the typed alias already matches one of your keys; the backend preserves the upstream status andparam
(e.g.400
/key_alias
)
Push a tag matching the pattern <package>@<version>
to trigger automated npm publish + GitHub Release:
$EDITOR packages/plugin-litellm/package.json # or plugin-litellm-backend
git commit -am "release: litellm vX.Y.Z"
git push origin main
git tag litellm@X.Y.Z # or litellm-backend@X.Y.Z
git push origin litellm@X.Y.Z
The CI workflow verifies the tag version matches package.json
, builds, publishes to npm, and auto-creates a GitHub Release with generated release notes.
This message is displayed in the Teams panel when the authenticated user exists in LiteLLM but belongs to no LiteLLM teams. It is an informational UI state, not an error — the user is provisioned and can still generate keys and view usage.
Why it happens:
- The user was provisioned with
provisioning.defaults.teams: []
(the default), so no teams were assigned at creation time. - Alternatively the user was created manually in LiteLLM without team membership.
How to fix:
- Add the user to a LiteLLM team via the LiteLLM admin UI or API.
- Or set
litellm.provisioning.defaults.teams
(or a matching role override) to include the relevant LiteLLM team IDs before the user's first sign-in. Users already provisioned will not be retroactively re-assigned — update them via LiteLLM directly.
The backend returns a 404 with { "error": "User not found in LiteLLM", "hint": "...", "provisioning": false }
when:
- The user does not exist in LiteLLM,
and
litellm.provisioning.enabled
isfalse
(the default).
Fix: Either enable autoprovisioning (litellm.provisioning.enabled: true
) or create the user manually in LiteLLM using an ID that matches the Backstage entity name (plus userIdDomain
if configured).
The backend derives the LiteLLM user_id
from the Backstage token using the formula:
user_id = <entity-name> [ + "@" + userIdDomain ]
For example, user:default/john.doe
with userIdDomain: example.com
produces john.doe@example.com
. If LiteLLM has the user stored under a different ID (e.g. the full email was used as the entity name), the lookup will fail.
Fix: Align the LiteLLM user IDs with what the plugin derives, or adjust userIdDomain
. If the Backstage entity name is already in email form (e.g. user:default/john.doe@example.com
), do not set userIdDomain
— the plugin detects the @
and skips the domain suffix to avoid double-appending.
Ensure proper Material-UI theme configuration in your parent Backstage app.
Verify that LITELLM_MASTER_KEY
has permissions to list models on the LiteLLM proxy.
Check LiteLLM proxy connectivity and master key validity. The backend health endpoint (GET /api/litellm/health
) returns the provisioning status and can confirm the plugin is reachable.
Usage analytics refresh when the date range selector is changed. If data appears stale, change the range and change it back to trigger a reload.