{"slug": "lago-open-source-sdk-bill-on-top-of-your-llm-token-cost-with-no-middleware", "title": "Lago Open-source SDK: Bill on top of your LLM token cost with no middleware", "summary": "Lago released an open-source SDK that wraps existing LLM clients to automatically extract token usage data and send it to Lago's billing platform without requiring middleware or API changes. The SDK supports AWS Bedrock and Mistral providers with p99 overhead under 5 milliseconds, buffering usage events in memory and flushing them in batches while surviving provider or Lago outages through exponential backoff. The tool enables developers to bill customers based on LLM token consumption by attaching subscription IDs per call, per context, or as a default fallback.", "body_md": "Instrument LLM clients and emit usage events to [Lago](https://www.getlago.com) for billing.\n\n```\n                  ┌──────────────┐\nyour code ──────► │ wrapped client│ ──► provider (Bedrock / Mistral / …)\n                  └──────┬───────┘\n                         │ (extract usage)\n                         ▼\n                  ┌──────────────┐\n                  │  Lago events │ ──► api.getlago.com\n                  └──────────────┘\n```\n\n- Wraps your existing LLM client in place — no API surface change for your application code.\n- Extracts usage from each response into a normalized shape (\n`CanonicalUsage`\n\n). - Buffers events in memory, flushes them in batches to Lago's\n`/events/batch`\n\nendpoint. - Survives provider/Lago outages with exponential backoff and a bounded buffer.\n- p99 wrap-overhead under 5 ms — your call is never blocked on Lago.\n\n```\npip install lago-agent-sdk\n```\n\nFor Bedrock support: `pip install 'lago-agent-sdk[bedrock]'`\n\n(adds `boto3`\n\n).\nFor Mistral support: `pip install 'lago-agent-sdk[mistral]'`\n\n(adds `mistralai`\n\n).\n\n``` python\nimport boto3\nfrom lago_agent_sdk import LagoSDK\n\nsdk = LagoSDK(\n    api_key=\"<YOUR_LAGO_API_KEY>\",\n    api_url=\"https://api.getlago.com/api/v1/\",\n    default_subscription_id=\"sub_acme\",\n)\nclient = sdk.wrap(boto3.client(\"bedrock-runtime\", region_name=\"eu-west-1\"))\n\nresp = client.converse(\n    modelId=\"eu.amazon.nova-lite-v1:0\",\n    messages=[{\"role\": \"user\", \"content\": [{\"text\": \"Hello\"}]}],\n)\nsdk.flush()\n```\n\nThe wrapped client behaves identically to the original — same arguments, same return shape, same exceptions. The SDK adds an in-memory queue that batches events to Lago in the background.\n\n``` python\nfrom mistralai.client import Mistral\nfrom lago_agent_sdk import LagoSDK\n\nsdk = LagoSDK(api_key=\"...\", default_subscription_id=\"sub_acme\")\nclient = sdk.wrap(Mistral(api_key=\"...\"))\n\nresp = client.chat.complete(\n    model=\"mistral-small-latest\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n)\nsdk.flush()\n```\n\nThree ways to set the `external_subscription_id`\n\n, in priority order:\n\n```\n# 1. Per-call override (highest precedence)\nclient.converse(..., extra_lago={\"subscription\": \"sub_acme\", \"dimensions\": {\"feature\": \"summarize\"}})\n\n# 2. Context-bound (use in middleware to set once per request)\nsdk.set_subscription(\"sub_acme\")\n# all calls in this thread/asyncio task → sub_acme\n\n# 3. Default at init (fallback)\nsdk = LagoSDK(api_key=\"...\", default_subscription_id=\"sub_default\")\n```\n\nBacked by `contextvars`\n\nfor safe propagation across `asyncio`\n\ntasks.\n\n| Provider | Access | Status |\n|---|---|---|\n| AWS Bedrock | `Converse` (sync + stream) |\n✓ |\n| AWS Bedrock | `InvokeModel` (sync + stream), 7 model families |\n✓ |\n| Mistral | native SDK (`chat.complete` + `chat.stream` ) |\n✓ |\n| OpenAI | native SDK | Phase 2 |\n| Anthropic | native SDK | Phase 2 |\n| Google Gemini | native SDK | Phase 2 |\n| LiteLLM | callback bridge | Phase 4 |\n\n`CanonicalUsage`\n\ncarries 10 numeric fields. Which ones populate depends on the provider:\n\n| Field | Lago metric code | Bedrock | Mistral native |\n|---|---|---|---|\n| input | `llm_input_tokens` |\n✓ | ✓ |\n| output | `llm_output_tokens` |\n✓ | ✓ |\n| cache_read | `llm_cached_input_tokens` |\n✓ (Anthropic) | ✓ (when cache hits) |\n| cache_write | `llm_cache_creation_tokens` |\n✓ (Anthropic) | ✗ |\n| cache_write_5m / 1h | `llm_cache_write_5m/1h_tokens` |\n✓ (Anthropic InvokeModel) | ✗ |\n| reasoning | `llm_reasoning_tokens` |\n✗ (folded into output) | ✗ (folded into output) |\n| tool_calls | `llm_tool_calls` |\n✓ | ✓ |\n| image_input / audio_input | `llm_image/audio_input_tokens` |\n✗ | ✗ |\n\nReasoning, image, and audio fields will populate when Phase 2 native OpenAI ships.\n\nThe SDK never breaks your LLM call. If anything in instrumentation fails (adapter bug, Lago down, network error), the SDK swallows it, logs a warning, and your call returns normally.\n\nConfigurable via `LagoConfig.on_error`\n\ncallback to integrate with Sentry, Datadog, etc.:\n\n``` python\nfrom lago_agent_sdk import LagoConfig, LagoSDK\n\ndef on_error(exc: Exception, where: str) -> None:\n    sentry.capture_exception(exc, tags={\"sdk_phase\": where})\n\nsdk = LagoSDK(\n    api_key=\"...\",\n    config=LagoConfig(api_key=\"...\", on_error=on_error),\n)\n```\n\nThe SDK ships with default metric codes (`llm_input_tokens`\n\n, `llm_output_tokens`\n\n, etc.). You need to register matching billable metrics in your Lago tenant before events count toward charges. See [Lago docs — Billable Metrics](https://docs.getlago.com/api-reference/billable-metrics/create).\n\n```\ngit clone https://github.com/getlago/lago-agent-sdk-python\ncd lago-agent-sdk-python\npython -m venv venv && source venv/bin/activate\npip install -e '.[dev]'\npytest\n```\n\nRun live integration tests (requires real credentials):\n\n```\nAWS_BEARER_TOKEN_BEDROCK=\"...\" \\\nMISTRAL_API_KEY=\"...\" \\\nLAGO_API_URL=\"https://api.getlago.com/api/v1/\" \\\nLAGO_API_KEY=\"...\" \\\nLAGO_EXTERNAL_SUBSCRIPTION_ID=\"sub_...\" \\\npytest tests/integration\n```\n\nFound a vulnerability? See [SECURITY.md](/getlago/lago-agent-sdk-python/blob/main/SECURITY.md).", "url": "https://wpnews.pro/news/lago-open-source-sdk-bill-on-top-of-your-llm-token-cost-with-no-middleware", "canonical_source": "https://github.com/getlago/lago-agent-sdk-python", "published_at": "2026-05-27 00:17:38+00:00", "updated_at": "2026-05-27 00:39:44.028046+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "ai-products", "mlops"], "entities": ["Lago", "Bedrock", "Mistral", "boto3", "Amazon Nova"], "alternates": {"html": "https://wpnews.pro/news/lago-open-source-sdk-bill-on-top-of-your-llm-token-cost-with-no-middleware", "markdown": "https://wpnews.pro/news/lago-open-source-sdk-bill-on-top-of-your-llm-token-cost-with-no-middleware.md", "text": "https://wpnews.pro/news/lago-open-source-sdk-bill-on-top-of-your-llm-token-cost-with-no-middleware.txt", "jsonld": "https://wpnews.pro/news/lago-open-source-sdk-bill-on-top-of-your-llm-token-cost-with-no-middleware.jsonld"}}