Spotify Backstage Meet LiteLLM for Corporate AI Governance

A new Backstage plugin from @acarmisc provides a governance interface for LiteLLM proxy, enabling developers to manage virtual API keys and monitor AI model usage directly within Backstage 1.50+. The plugin includes frontend React components and a backend Express router, offering features such as KPI dashboards, key management, and model browsing, with configuration via environment variables like LITELLM_BASE_URL and LITELLM_MASTER_KEY.

Backstage plugin for LiteLLM governance — enables developers to manage virtual API keys and monitor AI model usage directly from Backstage. This is a Backstage 1.50+ plugin providing a governance interface for LiteLLM proxy. It includes: Frontend : React components built with the New Frontend System @backstage/frontend-plugin-api Backend : Express router using the New Backend System @backstage/backend-plugin-api packages/plugin-litellm - Frontend @acarmisc/backstage-plugin-litellm packages/plugin-litellm-backend - Backend @acarmisc/backstage-plugin-litellm-backend The LiteLLMHomeWidget card surfaces the signed-in user's KPIs USD spent, tokens in/out, active key count alongside a daily-spend sparkline, with a Today / 7d / 30d period selector. The Overview tab is the default landing view. It shows the user's identity header with team chips and key health counters , four KPI tiles Total Spend, Total Requests, Success Rate, Total Tokens , and the Costs chart with per-model daily spend. The full-page version below shows the complete chart grid Daily Spend by Model, Daily Token Usage, Daily Requests, Daily Success Rate, Cumulative Spend vs Budget : Browse, edit, block, and revoke your virtual keys. Each row shows alias, key ID, creation/expiry dates, budget bar, TPM/RPM, and the models it can call. Mint a scoped key with its own budget, team binding, model list, and TPM/RPM caps. Failures e.g. duplicate alias surface inline with the upstream error. Browse every model the proxy exposes, with per-model input/output cost and max input/output token limits. The team filter scopes the list to the team the key will be bound to. A smaller variant of the home widget for surfaces that only need a one-click shortcut into the key-mint flow. This plugin is designed to be used within a Backstage monorepo . It uses workspace dependencies and requires the Backstage CLI to build. From your Backstage monorepo root: yarn add file:../backstage-govai/packages/plugin-litellm yarn add file:../backstage-govai/packages/plugin-litellm-backend Copy the packages directly into your Backstage plugins/ directory and add them to your workspace. Set these in your shell or deployment environment before starting Backstage. Backstage's config system supports ${ENV VAR} substitution in app-config.yaml : LITELLM BASE URL=http://litellm-proxy:4000 LiteLLM proxy URL LITELLM MASTER KEY=sk-... LiteLLM admin master key Add to app-config.yaml . All keys live under the litellm top-level namespace: litellm: Required — base URL of your LiteLLM proxy instance. @visibility backend baseUrl: ${LITELLM BASE URL} Optional — publicly reachable LiteLLM proxy URL, used to build ready-to-paste curl / OpenAI-SDK snippets in the "Key Generated" dialog. Falls back to baseUrl the internal URL when omitted. @visibility backend publicBaseUrl: https://llm-gw.example.com Required — LiteLLM master key for admin operations. Never exposed to the frontend marked @visibility secret . masterKey: ${LITELLM MASTER KEY} Optional — email domain appended to the Backstage user entity name to form the LiteLLM user id. When set, a user entity "user:default/john.doe" maps to "john.doe@example.com" in LiteLLM. Omit to use the bare entity name. @visibility backend userIdDomain: example.com optional Optional — autoprovisioning of LiteLLM users on first access. provisioning: Whether to automatically create a LiteLLM user when the Backstage user is not yet known to LiteLLM. Disabled by default. enabled: false default defaults: Max lifetime spend USD before the account is blocked. maxBudget: 10 default: 10 Spend-reset period after which the spend counter resets. Accepts LiteLLM duration strings: "30d", "7d", "1h", etc. budgetDuration: 30d default: "30d" LiteLLM model IDs the new user is allowed to call. An empty list means all models configured in the proxy are allowed. models: default: all models LiteLLM team IDs to enrol the new user in automatically. teams: default: no teams LiteLLM role assigned to every provisioned user. Valid values: proxy admin, proxy admin viewer, internal user, internal user viewer, team. userRole: internal user default: "internal user" Tokens per minute hard cap omit for no per-user limit . tpmLimit: 100000 Requests per minute hard cap omit for no per-user limit . rpmLimit: 1000 Arbitrary key-value metadata stored on the LiteLLM user record. metadata: cost centre: engineering Optional — role-based provisioning overrides. Evaluated in order; first matching group wins. Fields omitted here fall back to defaults above. roles: - group: group:default/ai-power-users Backstage group entity ref maxBudget: 100 budgetDuration: 30d models: - gpt-4o - claude-3-5-sonnet userRole: internal user Optional — controls for the "Generate New Key" form in the frontend. keyGeneration: Show the "Unlimited budget" checkbox. When false, a positive max budget is always required. allowUnlimitedBudget: false default Require a team to be selected before a key can be generated. Set to false to allow personal, team-less keys. teamRequired: true default Config key reference: | Key | Type | Required | Default | Description | |---|---|---|---|---| litellm.baseUrl | string | yes | — | LiteLLM proxy base URL | litellm.publicBaseUrl | string | no | — | Publicly reachable proxy URL for snippet generation | litellm.masterKey | string | yes | — | Admin master key @visibility secret | litellm.userIdDomain | string | no | — | Email domain for LiteLLM user IDs | litellm.provisioning.enabled | boolean | no | false | Enable autoprovisioning | litellm.provisioning.defaults.maxBudget | number | no | 10 | Max spend USD per reset period | litellm.provisioning.defaults.budgetDuration | string | no | "30d" | Spend-reset period | litellm.provisioning.defaults.models | string | no | | Allowed model IDs empty = all | litellm.provisioning.defaults.teams | string | no | | Team IDs to join on creation | litellm.provisioning.defaults.userRole | string | no | "internal user" | LiteLLM role | litellm.provisioning.defaults.tpmLimit | number | no | — | Tokens-per-minute cap | litellm.provisioning.defaults.rpmLimit | number | no | — | Requests-per-minute cap | litellm.provisioning.defaults.metadata | object | no | {} | Extra metadata on user record | litellm.provisioning.roles .group | string | yes | — | Backstage group entity ref | litellm.provisioning.roles .maxBudget | number | no | — | Overrides default for group | litellm.provisioning.roles .budgetDuration | string | no | — | Overrides default for group | litellm.provisioning.roles .models | string | no | — | Overrides default for group | litellm.provisioning.roles .teams | string | no | — | Overrides default for group | litellm.provisioning.roles .userRole | string | no | — | Overrides default for group | litellm.provisioning.roles .tpmLimit | number | no | — | Overrides default for group | litellm.provisioning.roles .rpmLimit | number | no | — | Overrides default for group | litellm.provisioning.roles .metadata | object | no | — | Merged over default metadata | litellm.keyGeneration.allowUnlimitedBudget | boolean | no | false | Show the "Unlimited budget" checkbox in the Generate New Key form | litellm.keyGeneration.teamRequired | boolean | no | true | Require a team to be selected before a key can be generated | required when the roles array is present In packages/backend/src/index.ts : backend.add import '@acarmisc/backstage-plugin-litellm-backend' ; The plugin uses the Backstage New Frontend System. Add the plugin package as an extension in packages/app/src/App.tsx or equivalent: js import { litellmPlugin, LiteLLMPage } from '@acarmisc/backstage-plugin-litellm'; // Add the route: <Route path="/litellm" element={<LiteLLMPage / } / You can also register it as a plugin extension using the New Frontend System. LiteLLMHomeWidget is a compact card you can drop onto any Backstage homepage. It shows the signed-in user's data — identity is resolved server-side from the Backstage Bearer token, so no userId prop is required and no additional backend endpoint is needed. KPIs displayed: USD spent · Tokens in · Tokens out · Key count, plus a daily-spend sparkline hidden when there is no daily data . A small period selector Today / 7d / 30d lives in the card header. js import { LiteLLMHomeWidget } from '@acarmisc/backstage-plugin-litellm'; // In your HomePage composition: <LiteLLMHomeWidget defaultPeriod="7d" / Props: | Prop | Type | Default | Description | |---|---|---|---| defaultPeriod | 'today' | '7d' | '30d' | '7d' | Period shown on first render | title | string | 'LiteLLM Usage' | Card title override | The widget requires the same backend setup as the full LiteLLMPage backend plugin configured and the user provisioned in LiteLLM . When litellm.provisioning.enabled is true , the backend automatically creates a LiteLLM user the first time a Backstage user hits any plugin endpoint user info, keys, teams, or usage . The flow is: - The backend resolves the caller's Backstage identity from the request token user:default/<name . - It checks whether that identity already exists in LiteLLM via /user/info . - If not found, it looks up the user's Backstage catalog entity to fetch their profile email, display name and group memberships. - It applies any matching provisioning.roles override first match wins , then calls /user/new on LiteLLM with the effective defaults. - A concurrent single-flight lock prevents duplicate /user/new calls when several endpoints fire in parallel on the same page load. Backstage catalog prerequisites: - The user must exist as a . The catalog is the source of truth for email, display name, and group memberships. User entity in the Backstage catalog - Group memberships used for role matching are resolved from the memberOf relations on the user entity. These are typically populated by a catalog provider such as the LDAP, GitHub, or Microsoft Graph org provider. - If userIdDomain is set, the entity name e.g. john.doe from user:default/john.doe is combined with the domain to produce the LiteLLM user id e.g. john.doe@example.com . Make sure LiteLLM users were created with matching IDs if you are migrating an existing deployment. - If a user signs in without a catalog entity e.g. dangerouslyAllowSignInWithoutUserInCatalog is set , provisioning still proceeds but the LiteLLM user record will lack email, display name, and team-role resolution — they will receive the default settings. Minimum working example with autoprovisioning enabled: litellm: baseUrl: ${LITELLM BASE URL} masterKey: ${LITELLM MASTER KEY} provisioning: enabled: true defaults: maxBudget: 5 budgetDuration: 30d Build from within your Backstage monorepo: yarn workspace @acarmisc/backstage-plugin-litellm build yarn workspace @acarmisc/backstage-plugin-litellm-backend build yarn workspace @acarmisc/backstage-plugin-litellm test For frontend-only iteration: cd packages/plugin-litellm yarn start The backend provides the following endpoints all prefixed with /api/litellm . A machine-readable OpenAPI 3.1 contract is served at /api/litellm/openapi.json — point any OpenAPI-compatible renderer Stoplight, Swagger UI, Redoc at it instead of maintaining this table by hand. | Endpoint | Method | Purpose | |---|---|---| /health | GET | Health check | /config | GET | Public LiteLLM proxy base URL for snippet generation | /openapi.json | GET | OpenAPI 3.1 contract for this backend surface | /user/info | GET | Get current user info and quotas | /keys | GET | List user's virtual keys | /keys/generate | POST | Generate a new virtual key | /keys/:keyId | DELETE | Revoke/delete a virtual key caller must own it | /keys/:keyId/update | POST | Update alias / models / budget / limits caller must own it | /keys/:keyId/block | POST | Suspend a key without revoking it caller must own it | /keys/:keyId/unblock | POST | Re-enable a blocked key caller must own it | /keys/:keyId/reset spend | POST | Zero out a key's spend counter caller must own it | /models | GET | List available LLM models | /teams | GET | List teams the current user belongs to | /teams/:teamId/usage | GET | Usage metrics for a team start date , end date required | /usage | GET | Get usage metrics and analytics for the current user | /audit | GET | Audit logs gated by litellm.audit.group membership | /provisioning/preview | GET | Resolve which role a Backstage group maps to dry-run, audit-group-gated | The UI endpoints above authenticate via the Backstage identity system. The CLI bridge endpoints below are gated behind litellm.bridge.enabled and authenticate with a raw Keycloak access token instead see CLI Bridge cli-bridge-abby : | Endpoint | Method | Purpose | |---|---|---| /bridge/health | GET | Bridge health + configured clientId no auth | /bridge/keys | GET | List the caller's virtual keys | /bridge/keys | POST | Mint a virtual key for the caller | /bridge/models | GET | List available LLM models | The bridge lets CLI clients the Abby CLI list and mint LiteLLM virtual keys without ever holding the LiteLLM master key . The Backstage backend keeps the master key as it already does for the UI ; the CLI authenticates with its Keycloak access token — the same realm Backstage uses. Request flow for /api/litellm/bridge/ : - CLI sends Authorization: Bearer <keycloak-access-token . - The bridge verifies the JWT against the realm JWKS createRemoteJWKSet , checking the issuer and that the token was issued for the configured clientId via azp , falling back to aud . Failure → 401 . - The caller is resolved to a LiteLLM user id email → preferred username → sub and ensured to exist — provisioned from the JWT claims if litellm.provisioning.enabled , otherwise 404 log in to Backstage once first . - Keys are listed/minted via the existing master-key-authed client. Minted keys are stamped with ownership metadata created via: abby-cli , created by , created at iso . Unlike the UI routes, bridge routes do not call Backstage's auth.authenticate — they verify the raw Keycloak JWT themselves. Configuration app-config.yaml : litellm: bridge: enabled: true default false issuer: https://auth.example.com/realms/solution-innovation required when enabled clientId: abby-cli default abby-cli When enabled is true but issuer is missing the backend fails fast at startup; the bridge routes are not mounted otherwise. - Built with React and Material-UI - Uses Backstage API client for backend communication - Components: LiteLLMPage - Main plugin page DashboardHeader - Header with user context KeysTable - Display and manage virtual keys UsageStats - Usage analytics with date range selector TeamUsage - Team-specific usage breakdown - Express-based router - Communicates with LiteLLM proxy API - Handles authentication via Backstage identity system - Provides user context resolution and API proxying Key Management : Generate, view, and revoke virtual API keys Usage Analytics : Track API usage with configurable date ranges today, 7 days, 30 days Team Context : Optional team-based key generation and usage tracking User Info : Display user quotas and current usage limits Model Selection : Browse available LLM models configured in LiteLLM At-a-glance Dashboard : The profile header shows the current user, team membership, a live counter of total / expired / expiring-soon keys, and a one-click "Generate New Key" shortcut Inline Generation Errors : Key-generation failures e.g. a duplicate alias surface directly in the dialog, with a client-side warning when the typed alias already matches one of your keys; the backend preserves the upstream status and param e.g. 400 / key alias Push a tag matching the pattern <package @<version to trigger automated npm publish + GitHub Release: Bump the version in package.json $EDITOR packages/plugin-litellm/package.json or plugin-litellm-backend git commit -am "release: litellm vX.Y.Z" git push origin main Cut and push the tag git tag litellm@X.Y.Z or litellm-backend@X.Y.Z git push origin litellm@X.Y.Z The CI workflow verifies the tag version matches package.json , builds, publishes to npm, and auto-creates a GitHub Release with generated release notes. This message is displayed in the Teams panel when the authenticated user exists in LiteLLM but belongs to no LiteLLM teams. It is an informational UI state, not an error — the user is provisioned and can still generate keys and view usage. Why it happens: - The user was provisioned with provisioning.defaults.teams: the default , so no teams were assigned at creation time. - Alternatively the user was created manually in LiteLLM without team membership. How to fix: - Add the user to a LiteLLM team via the LiteLLM admin UI or API. - Or set litellm.provisioning.defaults.teams or a matching role override to include the relevant LiteLLM team IDs before the user's first sign-in. Users already provisioned will not be retroactively re-assigned — update them via LiteLLM directly. The backend returns a 404 with { "error": "User not found in LiteLLM", "hint": "...", "provisioning": false } when: - The user does not exist in LiteLLM, and litellm.provisioning.enabled is false the default . Fix: Either enable autoprovisioning litellm.provisioning.enabled: true or create the user manually in LiteLLM using an ID that matches the Backstage entity name plus userIdDomain if configured . The backend derives the LiteLLM user id from the Backstage token using the formula: user id = <entity-name + "@" + userIdDomain For example, user:default/john.doe with userIdDomain: example.com produces john.doe@example.com . If LiteLLM has the user stored under a different ID e.g. the full email was used as the entity name , the lookup will fail. Fix: Align the LiteLLM user IDs with what the plugin derives, or adjust userIdDomain . If the Backstage entity name is already in email form e.g. user:default/john.doe@example.com , do not set userIdDomain — the plugin detects the @ and skips the domain suffix to avoid double-appending. Ensure proper Material-UI theme configuration in your parent Backstage app. Verify that LITELLM MASTER KEY has permissions to list models on the LiteLLM proxy. Check LiteLLM proxy connectivity and master key validity. The backend health endpoint GET /api/litellm/health returns the provisioning status and can confirm the plugin is reachable. Usage analytics refresh when the date range selector is changed. If data appears stale, change the range and change it back to trigger a reload.