{"slug": "spotify-backstage-meet-litellm-for-corporate-ai-governance", "title": "Spotify Backstage Meet LiteLLM for Corporate AI Governance", "summary": "A new Backstage plugin from @acarmisc provides a governance interface for LiteLLM proxy, enabling developers to manage virtual API keys and monitor AI model usage directly within Backstage 1.50+. The plugin includes frontend React components and a backend Express router, offering features such as KPI dashboards, key management, and model browsing, with configuration via environment variables like LITELLM_BASE_URL and LITELLM_MASTER_KEY.", "body_md": "Backstage plugin for LiteLLM governance — enables developers to manage virtual API keys and monitor AI model usage directly from Backstage.\n\nThis is a Backstage 1.50+ plugin providing a governance interface for LiteLLM proxy. It includes:\n\n**Frontend**: React components built with the New Frontend System (`@backstage/frontend-plugin-api`\n\n)**Backend**: Express router using the New Backend System (`@backstage/backend-plugin-api`\n\n)\n\n`packages/plugin-litellm`\n\n- Frontend (`@acarmisc/backstage-plugin-litellm`\n\n)`packages/plugin-litellm-backend`\n\n- Backend (`@acarmisc/backstage-plugin-litellm-backend`\n\n)\n\nThe `LiteLLMHomeWidget`\n\ncard surfaces the signed-in user's KPIs (USD spent, tokens in/out, active key count) alongside a daily-spend sparkline, with a `Today`\n\n/ `7d`\n\n/ `30d`\n\nperiod selector.\n\nThe `Overview`\n\ntab is the default landing view. It shows the user's identity header (with team chips and key health counters), four KPI tiles (Total Spend, Total Requests, Success Rate, Total Tokens), and the `Costs`\n\nchart with per-model daily spend.\n\nThe full-page version below shows the complete chart grid (Daily Spend by Model, Daily Token Usage, Daily Requests, Daily Success Rate, Cumulative Spend vs Budget):\n\nBrowse, edit, block, and revoke your virtual keys. Each row shows alias, key ID, creation/expiry dates, budget bar, TPM/RPM, and the models it can call.\n\nMint a scoped key with its own budget, team binding, model list, and TPM/RPM caps. Failures (e.g. duplicate alias) surface inline with the upstream error.\n\nBrowse every model the proxy exposes, with per-model input/output cost and max input/output token limits. The team filter scopes the list to the team the key will be bound to.\n\nA smaller variant of the home widget for surfaces that only need a one-click shortcut into the key-mint flow.\n\nThis plugin is designed to be used **within a Backstage monorepo**. It uses workspace dependencies and requires the Backstage CLI to build.\n\nFrom your Backstage monorepo root:\n\n```\nyarn add file:../backstage-govai/packages/plugin-litellm\nyarn add file:../backstage-govai/packages/plugin-litellm-backend\n```\n\nCopy the packages directly into your Backstage `plugins/`\n\ndirectory and add them to your workspace.\n\nSet these in your shell or deployment environment before starting Backstage. Backstage's config system supports `${ENV_VAR}`\n\nsubstitution in `app-config.yaml`\n\n:\n\n```\nLITELLM_BASE_URL=http://litellm-proxy:4000     # LiteLLM proxy URL\nLITELLM_MASTER_KEY=sk-...                      # LiteLLM admin master key\n```\n\nAdd to `app-config.yaml`\n\n. All keys live under the `litellm`\n\ntop-level namespace:\n\n```\nlitellm:\n  # Required — base URL of your LiteLLM proxy instance.\n  # @visibility backend\n  baseUrl: ${LITELLM_BASE_URL}\n\n  # Optional — publicly reachable LiteLLM proxy URL, used to build\n  # ready-to-paste curl / OpenAI-SDK snippets in the \"Key Generated\" dialog.\n  # Falls back to baseUrl (the internal URL) when omitted.\n  # @visibility backend\n  # publicBaseUrl: https://llm-gw.example.com\n\n  # Required — LiteLLM master key for admin operations.\n  # Never exposed to the frontend (marked @visibility secret).\n  masterKey: ${LITELLM_MASTER_KEY}\n\n  # Optional — email domain appended to the Backstage user entity name to form\n  # the LiteLLM user_id. When set, a user entity \"user:default/john.doe\" maps\n  # to \"john.doe@example.com\" in LiteLLM. Omit to use the bare entity name.\n  # @visibility backend\n  userIdDomain: example.com   # optional\n\n  # Optional — autoprovisioning of LiteLLM users on first access.\n  provisioning:\n    # Whether to automatically create a LiteLLM user when the Backstage user\n    # is not yet known to LiteLLM. Disabled by default.\n    enabled: false   # default\n\n    defaults:\n      # Max lifetime spend (USD) before the account is blocked.\n      maxBudget: 10          # default: 10\n\n      # Spend-reset period after which the spend counter resets.\n      # Accepts LiteLLM duration strings: \"30d\", \"7d\", \"1h\", etc.\n      budgetDuration: 30d    # default: \"30d\"\n\n      # LiteLLM model IDs the new user is allowed to call.\n      # An empty list means all models configured in the proxy are allowed.\n      models: []             # default: [] (all models)\n\n      # LiteLLM team IDs to enrol the new user in automatically.\n      teams: []              # default: [] (no teams)\n\n      # LiteLLM role assigned to every provisioned user.\n      # Valid values: proxy_admin, proxy_admin_viewer, internal_user,\n      #               internal_user_viewer, team.\n      userRole: internal_user   # default: \"internal_user\"\n\n      # Tokens per minute hard cap (omit for no per-user limit).\n      # tpmLimit: 100000\n\n      # Requests per minute hard cap (omit for no per-user limit).\n      # rpmLimit: 1000\n\n      # Arbitrary key-value metadata stored on the LiteLLM user record.\n      # metadata:\n      #   cost_centre: engineering\n\n    # Optional — role-based provisioning overrides.\n    # Evaluated in order; first matching group wins.\n    # Fields omitted here fall back to defaults above.\n    roles:\n      - group: group:default/ai-power-users   # Backstage group entity ref\n        maxBudget: 100\n        budgetDuration: 30d\n        models:\n          - gpt-4o\n          - claude-3-5-sonnet\n        userRole: internal_user\n\n  # Optional — controls for the \"Generate New Key\" form in the frontend.\n  keyGeneration:\n    # Show the \"Unlimited budget\" checkbox. When false, a positive max\n    # budget is always required.\n    allowUnlimitedBudget: false   # default\n\n    # Require a team to be selected before a key can be generated.\n    # Set to false to allow personal, team-less keys.\n    teamRequired: true   # default\n```\n\n**Config key reference:**\n\n| Key | Type | Required | Default | Description |\n|---|---|---|---|---|\n`litellm.baseUrl` |\nstring | yes | — | LiteLLM proxy base URL |\n`litellm.publicBaseUrl` |\nstring | no | — | Publicly reachable proxy URL for snippet generation |\n`litellm.masterKey` |\nstring | yes | — | Admin master key (`@visibility secret` ) |\n`litellm.userIdDomain` |\nstring | no | — | Email domain for LiteLLM user IDs |\n`litellm.provisioning.enabled` |\nboolean | no | `false` |\nEnable autoprovisioning |\n`litellm.provisioning.defaults.maxBudget` |\nnumber | no | `10` |\nMax spend (USD) per reset period |\n`litellm.provisioning.defaults.budgetDuration` |\nstring | no | `\"30d\"` |\nSpend-reset period |\n`litellm.provisioning.defaults.models` |\nstring[] | no | `[]` |\nAllowed model IDs (empty = all) |\n`litellm.provisioning.defaults.teams` |\nstring[] | no | `[]` |\nTeam IDs to join on creation |\n`litellm.provisioning.defaults.userRole` |\nstring | no | `\"internal_user\"` |\nLiteLLM role |\n`litellm.provisioning.defaults.tpmLimit` |\nnumber | no | — | Tokens-per-minute cap |\n`litellm.provisioning.defaults.rpmLimit` |\nnumber | no | — | Requests-per-minute cap |\n`litellm.provisioning.defaults.metadata` |\nobject | no | `{}` |\nExtra metadata on user record |\n`litellm.provisioning.roles[].group` |\nstring | yes* | — | Backstage group entity ref |\n`litellm.provisioning.roles[].maxBudget` |\nnumber | no | — | Overrides default for group |\n`litellm.provisioning.roles[].budgetDuration` |\nstring | no | — | Overrides default for group |\n`litellm.provisioning.roles[].models` |\nstring[] | no | — | Overrides default for group |\n`litellm.provisioning.roles[].teams` |\nstring[] | no | — | Overrides default for group |\n`litellm.provisioning.roles[].userRole` |\nstring | no | — | Overrides default for group |\n`litellm.provisioning.roles[].tpmLimit` |\nnumber | no | — | Overrides default for group |\n`litellm.provisioning.roles[].rpmLimit` |\nnumber | no | — | Overrides default for group |\n`litellm.provisioning.roles[].metadata` |\nobject | no | — | Merged over default metadata |\n`litellm.keyGeneration.allowUnlimitedBudget` |\nboolean | no | `false` |\nShow the \"Unlimited budget\" checkbox in the Generate New Key form |\n`litellm.keyGeneration.teamRequired` |\nboolean | no | `true` |\nRequire a team to be selected before a key can be generated |\n\n*required when the `roles`\n\narray is present\n\nIn `packages/backend/src/index.ts`\n\n:\n\n```\nbackend.add(import('@acarmisc/backstage-plugin-litellm-backend'));\n```\n\nThe plugin uses the Backstage New Frontend System. Add the plugin package as an extension in `packages/app/src/App.tsx`\n\nor equivalent:\n\n``` js\nimport { litellmPlugin, LiteLLMPage } from '@acarmisc/backstage-plugin-litellm';\n\n// Add the route:\n<Route path=\"/litellm\" element={<LiteLLMPage />} />\n```\n\nYou can also register it as a plugin extension using the New Frontend System.\n\n`LiteLLMHomeWidget`\n\nis a compact card you can drop onto any Backstage homepage. It shows **the signed-in user's** data — identity is resolved server-side from the Backstage Bearer token, so no `userId`\n\nprop is required and no additional backend endpoint is needed.\n\n**KPIs displayed:** USD spent · Tokens in · Tokens out · Key count, plus a daily-spend sparkline (hidden when there is no daily data). A small period selector (`Today`\n\n/ `7d`\n\n/ `30d`\n\n) lives in the card header.\n\n``` js\nimport { LiteLLMHomeWidget } from '@acarmisc/backstage-plugin-litellm';\n\n// In your HomePage composition:\n<LiteLLMHomeWidget defaultPeriod=\"7d\" />\n```\n\n**Props:**\n\n| Prop | Type | Default | Description |\n|---|---|---|---|\n`defaultPeriod` |\n`'today' | '7d' | '30d'` |\n`'7d'` |\nPeriod shown on first render |\n`title` |\n`string` |\n`'LiteLLM Usage'` |\nCard title override |\n\nThe widget requires the same backend setup as the full `LiteLLMPage`\n\n(backend plugin configured and the user provisioned in LiteLLM).\n\nWhen `litellm.provisioning.enabled`\n\nis `true`\n\n, the backend automatically creates a LiteLLM user the first time a Backstage user hits any plugin endpoint (user info, keys, teams, or usage). The flow is:\n\n- The backend resolves the caller's Backstage identity from the request token (\n`user:default/<name>`\n\n). - It checks whether that identity already exists in LiteLLM via\n`/user/info`\n\n. - If not found, it looks up the user's Backstage catalog entity to fetch their profile (email, display name) and group memberships.\n- It applies any matching\n`provisioning.roles`\n\noverride (first match wins), then calls`/user/new`\n\non LiteLLM with the effective defaults. - A concurrent single-flight lock prevents duplicate\n`/user/new`\n\ncalls when several endpoints fire in parallel on the same page load.\n\n**Backstage catalog prerequisites:**\n\n- The user\n**must exist as a**. The catalog is the source of truth for email, display name, and group memberships.`User`\n\nentity in the Backstage catalog - Group memberships (used for role matching) are resolved from the\n`memberOf`\n\nrelations on the user entity. These are typically populated by a catalog provider such as the LDAP, GitHub, or Microsoft Graph org provider. - If\n`userIdDomain`\n\nis set, the entity name (e.g.`john.doe`\n\nfrom`user:default/john.doe`\n\n) is combined with the domain to produce the LiteLLM`user_id`\n\n(e.g.`john.doe@example.com`\n\n). Make sure LiteLLM users were created with matching IDs if you are migrating an existing deployment. - If a user signs in without a catalog entity (e.g.\n`dangerouslyAllowSignInWithoutUserInCatalog`\n\nis set), provisioning still proceeds but the LiteLLM user record will lack email, display name, and team-role resolution — they will receive the default settings.\n\n**Minimum working example with autoprovisioning enabled:**\n\n```\nlitellm:\n  baseUrl: ${LITELLM_BASE_URL}\n  masterKey: ${LITELLM_MASTER_KEY}\n  provisioning:\n    enabled: true\n    defaults:\n      maxBudget: 5\n      budgetDuration: 30d\n```\n\nBuild from within your Backstage monorepo:\n\n```\nyarn workspace @acarmisc/backstage-plugin-litellm build\nyarn workspace @acarmisc/backstage-plugin-litellm-backend build\nyarn workspace @acarmisc/backstage-plugin-litellm test\n```\n\nFor frontend-only iteration:\n\n```\ncd packages/plugin-litellm\nyarn start\n```\n\nThe backend provides the following endpoints (all prefixed with `/api/litellm`\n\n).\nA machine-readable OpenAPI 3.1 contract is served at `/api/litellm/openapi.json`\n\n— point any OpenAPI-compatible renderer (Stoplight, Swagger UI, Redoc) at it\ninstead of maintaining this table by hand.\n\n| Endpoint | Method | Purpose |\n|---|---|---|\n`/health` |\nGET | Health check |\n`/config` |\nGET | Public LiteLLM proxy base URL (for snippet generation) |\n`/openapi.json` |\nGET | OpenAPI 3.1 contract for this backend surface |\n`/user/info` |\nGET | Get current user info and quotas |\n`/keys` |\nGET | List user's virtual keys |\n`/keys/generate` |\nPOST | Generate a new virtual key |\n`/keys/:keyId` |\nDELETE | Revoke/delete a virtual key (caller must own it) |\n`/keys/:keyId/update` |\nPOST | Update alias / models / budget / limits (caller must own it) |\n`/keys/:keyId/block` |\nPOST | Suspend a key without revoking it (caller must own it) |\n`/keys/:keyId/unblock` |\nPOST | Re-enable a blocked key (caller must own it) |\n`/keys/:keyId/reset_spend` |\nPOST | Zero out a key's spend counter (caller must own it) |\n`/models` |\nGET | List available LLM models |\n`/teams` |\nGET | List teams the current user belongs to |\n`/teams/:teamId/usage` |\nGET | Usage metrics for a team (`start_date` , `end_date` required) |\n`/usage` |\nGET | Get usage metrics and analytics for the current user |\n`/audit` |\nGET | Audit logs (gated by `litellm.audit.group` membership) |\n`/provisioning/preview` |\nGET | Resolve which role a Backstage group maps to (dry-run, audit-group-gated) |\n\nThe UI endpoints above authenticate via the Backstage identity system. The\n**CLI bridge** endpoints below are gated behind `litellm.bridge.enabled`\n\nand\nauthenticate with a raw Keycloak access token instead (see [CLI Bridge](#cli-bridge-abby)):\n\n| Endpoint | Method | Purpose |\n|---|---|---|\n`/bridge/health` |\nGET | Bridge health + configured `clientId` (no auth) |\n`/bridge/keys` |\nGET | List the caller's virtual keys |\n`/bridge/keys` |\nPOST | Mint a virtual key for the caller |\n`/bridge/models` |\nGET | List available LLM models |\n\nThe bridge lets CLI clients (the **Abby** CLI) list and mint LiteLLM virtual\nkeys **without ever holding the LiteLLM master key**. The Backstage backend\nkeeps the master key (as it already does for the UI); the CLI authenticates\nwith its Keycloak access token — the same realm Backstage uses.\n\nRequest flow for `/api/litellm/bridge/*`\n\n:\n\n- CLI sends\n`Authorization: Bearer <keycloak-access-token>`\n\n. - The bridge verifies the JWT against the realm JWKS (\n`createRemoteJWKSet`\n\n), checking the issuer and that the token was issued for the configured`clientId`\n\n(via`azp`\n\n, falling back to`aud`\n\n). Failure →`401`\n\n. - The caller is resolved to a LiteLLM\n`user_id`\n\n(email → preferred_username → sub) and ensured to exist — provisioned from the JWT claims if`litellm.provisioning.enabled`\n\n, otherwise`404`\n\n(log in to Backstage once first). - Keys are listed/minted via the existing master-key-authed client. Minted\nkeys are stamped with ownership metadata (\n`created_via: abby-cli`\n\n,`created_by`\n\n,`created_at_iso`\n\n).\n\nUnlike the UI routes, bridge routes do **not** call Backstage's\n`auth.authenticate`\n\n— they verify the raw Keycloak JWT themselves.\n\nConfiguration (`app-config.yaml`\n\n):\n\n```\nlitellm:\n  bridge:\n    enabled: true                                   # default false\n    issuer: https://auth.example.com/realms/solution-innovation  # required when enabled\n    clientId: abby-cli                              # default abby-cli\n```\n\nWhen `enabled`\n\nis true but `issuer`\n\nis missing the backend fails fast at\nstartup; the bridge routes are not mounted otherwise.\n\n- Built with React and Material-UI\n- Uses Backstage API client for backend communication\n- Components:\n`LiteLLMPage`\n\n- Main plugin page`DashboardHeader`\n\n- Header with user context`KeysTable`\n\n- Display and manage virtual keys`UsageStats`\n\n- Usage analytics with date range selector`TeamUsage`\n\n- Team-specific usage breakdown\n\n- Express-based router\n- Communicates with LiteLLM proxy API\n- Handles authentication via Backstage identity system\n- Provides user context resolution and API proxying\n\n**Key Management**: Generate, view, and revoke virtual API keys** Usage Analytics**: Track API usage with configurable date ranges (today, 7 days, 30 days)** Team Context**: Optional team-based key generation and usage tracking** User Info**: Display user quotas and current usage limits** Model Selection**: Browse available LLM models configured in LiteLLM** At-a-glance Dashboard**: The profile header shows the current user, team membership, a live counter of total / expired / expiring-soon keys, and a one-click \"Generate New Key\" shortcut**Inline Generation Errors**: Key-generation failures (e.g. a duplicate alias) surface directly in the dialog, with a client-side warning when the typed alias already matches one of your keys; the backend preserves the upstream status and`param`\n\n(e.g.`400`\n\n/`key_alias`\n\n)\n\nPush a tag matching the pattern `<package>@<version>`\n\nto trigger automated npm publish + GitHub Release:\n\n```\n# Bump the version in package.json\n$EDITOR packages/plugin-litellm/package.json   # or plugin-litellm-backend\ngit commit -am \"release: litellm vX.Y.Z\"\ngit push origin main\n\n# Cut and push the tag\ngit tag litellm@X.Y.Z              # or litellm-backend@X.Y.Z\ngit push origin litellm@X.Y.Z\n```\n\nThe CI workflow verifies the tag version matches `package.json`\n\n, builds, publishes to npm, and auto-creates a GitHub Release with generated release notes.\n\nThis message is displayed in the Teams panel when the authenticated user exists in LiteLLM but belongs to no LiteLLM teams. It is an informational UI state, not an error — the user is provisioned and can still generate keys and view usage.\n\n**Why it happens:**\n\n- The user was provisioned with\n`provisioning.defaults.teams: []`\n\n(the default), so no teams were assigned at creation time. - Alternatively the user was created manually in LiteLLM without team membership.\n\n**How to fix:**\n\n- Add the user to a LiteLLM team via the LiteLLM admin UI or API.\n- Or set\n`litellm.provisioning.defaults.teams`\n\n(or a matching role override) to include the relevant LiteLLM team IDs before the user's first sign-in. Users already provisioned will not be retroactively re-assigned — update them via LiteLLM directly.\n\nThe backend returns a 404 with `{ \"error\": \"User not found in LiteLLM\", \"hint\": \"...\", \"provisioning\": false }`\n\nwhen:\n\n- The user does not exist in LiteLLM,\n**and** `litellm.provisioning.enabled`\n\nis`false`\n\n(the default).\n\n**Fix:** Either enable autoprovisioning (`litellm.provisioning.enabled: true`\n\n) or create the user manually in LiteLLM using an ID that matches the Backstage entity name (plus `userIdDomain`\n\nif configured).\n\nThe backend derives the LiteLLM `user_id`\n\nfrom the Backstage token using the formula:\n\n```\nuser_id = <entity-name> [ + \"@\" + userIdDomain ]\n```\n\nFor example, `user:default/john.doe`\n\nwith `userIdDomain: example.com`\n\nproduces `john.doe@example.com`\n\n. If LiteLLM has the user stored under a different ID (e.g. the full email was used as the entity name), the lookup will fail.\n\n**Fix:** Align the LiteLLM user IDs with what the plugin derives, or adjust `userIdDomain`\n\n. If the Backstage entity name is already in email form (e.g. `user:default/john.doe@example.com`\n\n), do **not** set `userIdDomain`\n\n— the plugin detects the `@`\n\nand skips the domain suffix to avoid double-appending.\n\nEnsure proper Material-UI theme configuration in your parent Backstage app.\n\nVerify that `LITELLM_MASTER_KEY`\n\nhas permissions to list models on the LiteLLM proxy.\n\nCheck LiteLLM proxy connectivity and master key validity. The backend health endpoint (`GET /api/litellm/health`\n\n) returns the provisioning status and can confirm the plugin is reachable.\n\nUsage analytics refresh when the date range selector is changed. If data appears stale, change the range and change it back to trigger a reload.", "url": "https://wpnews.pro/news/spotify-backstage-meet-litellm-for-corporate-ai-governance", "canonical_source": "https://github.com/acarmisc/backstage-plugin-litellm-govai", "published_at": "2026-08-28 17:51:47+00:00", "updated_at": "2026-08-28 18:18:53.503882+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure"], "entities": ["Backstage", "LiteLLM", "@acarmisc"], "alternates": {"html": "https://wpnews.pro/news/spotify-backstage-meet-litellm-for-corporate-ai-governance", "markdown": "https://wpnews.pro/news/spotify-backstage-meet-litellm-for-corporate-ai-governance.md", "text": "https://wpnews.pro/news/spotify-backstage-meet-litellm-for-corporate-ai-governance.txt", "jsonld": "https://wpnews.pro/news/spotify-backstage-meet-litellm-for-corporate-ai-governance.jsonld"}}