{"slug": "show-hn-spanlight-open-source-llm-observability-platform", "title": "Show HN: Spanlight – open-source LLM observability platform", "summary": "Spanlight, an open-source LLM observability platform built on a single Postgres database, launched publicly with a live demo running on real Claude calls under a small monthly budget. Spanlight records every LLM call — prompts, completions, latency, time to first token, tokens, cost and errors — grouped into traces and sessions, and accepts data via Python and TypeScript SDKs, any OpenTelemetry exporter over OTLP/HTTP, or by changing one base_url to its built-in gateway for OpenAI and Anthropic. The platform adds cost auditing against Anthropic or OpenAI admin-key bills, 15 deterministic failure detectors, budgets that can block spend, versioned prompts and evals with 95% confidence intervals, and Postgres row-level security isolating each project's data.", "body_md": "**Open-source observability for LLM applications, on one Postgres.**\n\n[Live demo](https://spanlight.dev) · [Docs](https://spanlight.dev/docs/) · [Features](#features) · [Getting started](#getting-started) · [How it works](#how-it-works) · [Discussions](https://github.com/dheerajkarwasra/spanlight/discussions)\n\nSpanlight records every LLM call your application makes: prompts and completions, latency, time to first token, tokens, cost and errors, grouped into traces and user sessions. Send traces with the Python or TypeScript SDK, any OpenTelemetry exporter over OTLP/HTTP, or by changing one `base_url` to the built-in gateway. Then go past tracing: audit what each call cost, check it against the provider's bill, find out why calls fail, and review yesterday's traffic every morning.\n\nTip\n\nWant to look before you install? The [live demo](https://spanlight.dev) runs on real Claude calls under a small monthly budget.\n\n- **Trace explorer and sessions.** Search traces by name or by what the model was sent and said, read a span waterfall with agent, tool and model steps, compare two traces step by step, and replay multi-turn conversations as chat with cost and latency per turn. Inputs and outputs are kept up to 1 MB each, so long RAG contexts stay readable.\n- **Cost you can audit.** Prices come from a versioned table that covers cache writes, long-prompt tiers, batch, flex and priority rates, and web search fees. Every call has a line-by-line cost breakdown. A cost that cannot be known shows as*unknown* with the reason, never as $0.\n- **Checked against the bill.** Connect an Anthropic or OpenAI admin key, or upload a CSV, and Spanlight compares traced spend with what the provider billed, every day.\n- **Attribution and budgets.** Tag traces with up to 32 metadata pairs (customer, feature, team) and see spend, calls and errors per value. Budgets can alert on, or block, spend per project, gateway key, user, model, metadata pair or trace name.\n- **The Doctor and provider health.** Fifteen deterministic detectors flag known failure patterns (retry storms, ignored`Retry-After` , truncated outputs, tool loops, cost and latency spikes and more), each with its evidence and a suggested fix. Provider health tells a provider outage or rate limit apart from your own bug, with the provider's status page incidents alongside.\n- **A gateway for OpenAI and Anthropic.** Gemini and Bedrock work through their OpenAI-compatible endpoints, with cost unknown until you add a price for their models. The official SDKs work against it unchanged, streaming and tool calls included. It adds sealed provider credentials, routes with retries and fallbacks, per-key rate limits, an exact-match cache, and an Integration Lab that injects real provider failures on non-production keys.\n- **Prompts, evals and a daily review.** Versioned prompts with labels, a playground, exact replay of any recorded call, datasets, experiments with 95% confidence intervals on pass rates, and a nightly queue of the traces worth reading. LLM judges show their agreement with your reviewers (κ, F1, confusion matrix).\n- **OpenTelemetry native.**`POST /v1/otlp/traces` (OTLP over HTTP, as JSON or protobuf; gRPC through an[OpenTelemetry Collector](https://spanlight.dev/docs/otlp/#sending-from-a-grpc-exporter) ) reads the GenAI conventions (`gen_ai.*` , also written by LiteLLM), OpenInference, OpenLLMetry (Traceloop) and the Vercel AI SDK, and counts usage once across nested spans. Recorded fixtures cover LangChain, LangGraph, LlamaIndex, Pydantic AI and the OpenAI Agents SDK. A Python SDK is on PyPI; a TypeScript SDK for Node.js is in[`sdks/typescript/`](https://github.com/dheerajkarwasra/spanlight/blob/main/sdks/typescript) (its npm release job is ready; install it from a tarball until the first release).\n- **Built for teams, private by design.** Organizations, projects, roles, scoped API keys and an audit log. Postgres row-level security isolates every project's data, and secrets are redacted from payloads before storage.\n- **Self-hosted on one database.** Postgres is the only datastore, the job queue included. The core is MIT licensed.\n\nNote\n\nSpanlight is pre-1.0. The current server release is 0.8.1 ([changelog](https://github.com/dheerajkarwasra/spanlight/blob/main/CHANGELOG.md)), and the Python SDK on PyPI is 0.3.0.\n\n- [Docker](https://docs.docker.com/get-docker/) with Compose v2\n- Python 3.10 or newer for the SDK\n- An OpenAI or Anthropic API key in your own application\n\n```\ngit clone https://github.com/dheerajkarwasra/spanlight.git\ncd spanlight\ncp deploy/.env.example deploy/.env    # fill in POSTGRES_PASSWORD, APP_DB_PASSWORD and SECRET_KEY\ndocker compose -f deploy/compose.yaml --env-file deploy/.env up -d\n```\n\nThis pulls the released images named in `deploy/.env` (`SPANLIGHT_IMAGE` and `SPANLIGHT_WEB_IMAGE`). To build from the checkout instead, comment out those two lines and add `--build`.\n\nOpen [http://localhost:8080](http://localhost:8080), create an account, and follow the setup wizard to create a project and an API key.\n\nTip\n\nGenerate each secret with `python3 -c \"import secrets; print(secrets.token_urlsafe(32))\"`. To host it instead, see [deploy to Railway](https://github.com/dheerajkarwasra/spanlight/blob/main/docs/deploy/railway.md).\n\nInstall the SDK with the extra for your provider (`anthropic` or `openai`):\n\n```\npip install \"spanlight[anthropic]\"\npython\nimport spanlight\nfrom anthropic import Anthropic\n\nspanlight.init(api_key=\"spl_live_…\", host=\"http://localhost:8080\")\nclient = spanlight.wrap_anthropic(Anthropic())\n\n@spanlight.observe()\ndef answer(question: str) -> str:\n    reply = client.messages.create(\n        model=\"claude-haiku-5-5\",\n        max_tokens=300,\n        messages=[{\"role\": \"user\", \"content\": question}],\n    )\n    return reply.content[0].text\n```\n\n## No SDK: use the gateway\n\nAdd a provider credential and a gateway key in the dashboard (**Gateway**), then point your existing client at Spanlight:\n\n``` python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8080/gw/v1\", api_key=\"spl_gw_…\")\n```\n\nFor Anthropic, use `Anthropic(base_url=\"http://localhost:8080/gw\", api_key=\"spl_gw_…\")`.\n\n## No SDK: use OpenTelemetry\n\nPoint any OTLP/HTTP exporter at `http://localhost:8080/v1/otlp/traces` with the header `Authorization: Bearer spl_live_…`. JSON and protobuf are both accepted. See the [OTLP guide](https://spanlight.dev/docs/otlp/) for the attributes each library sends.\n\nThe wizard shows your first trace live. From there you have the overview dashboard, the trace explorer, sessions, cost, attribution and the Doctor.\n\n**In one sentence:** your app sends a record of every LLM call to Spanlight, Spanlight prices it, stores it in Postgres and checks it for problems, and you read the results in the dashboard.\n\n```\nflowchart LR\n    app[\"Your app\"]\n    subgraph spanlight[\"Spanlight (one Docker image)\"]\n        web[\"web<br/>dashboard and docs\"]\n        api[\"api<br/>receives calls, prices them,<br/>answers the dashboard\"]\n        gw[\"gateway<br/>optional proxy to providers\"]\n        worker[\"worker<br/>background jobs\"]\n        pg[(\"Postgres<br/>the only database\")]\n    end\n    llm[\"OpenAI / Anthropic /<br/>OpenAI-compatible<br/>(Gemini, Bedrock)\"]\n\n    app -- \"SDK or OpenTelemetry\" --> api\n    app -- \"or change base_url\" --> gw\n    gw --> llm\n    gw --> api\n    web --> api\n    api --> pg\n    worker --> pg\n```\n\nPick one of three ways in. They all end up in the same place:\n\n| Way in | What you change | Good for | \n|---|---|---|\n| **Python SDK** | `spanlight.init()` and wrap your OpenAI or Anthropic client | Python apps; adds your own steps with `@observe` | \n| **OpenTelemetry** | Point any OTLP/HTTP exporter at `/v1/otlp/traces` | Any language or framework (LangChain, LlamaIndex, Vercel AI SDK, …) | \n| **Gateway** | Change your client's `base_url` to Spanlight | No code changes; also adds retries, fallbacks, caching and budgets | \n\nThe SDK sends calls in the background, compressed and in batches, and never throws an error into your code.\n\nWhen a batch arrives, the api does five things, in one database transaction:\n\n1. **Checks** the data and rejects anything malformed, with a reason.\n2. **Cleans** it: removes secrets such as API keys from prompts and outputs, and keeps each input and output up to 1 MB.\n3. **Prices** it from a versioned price table: cached tokens, cache writes, long-prompt rates, batch discounts and web-search fees. If a price can't be known, the cost is stored as*unknown* , never as $0.\n4. **Stores** it in Postgres. Sending the same batch twice changes nothing, so SDK retries never double-count.\n5. **Totals** each trace (all the calls of one request) so the dashboard reads fast.\n\nThe worker runs on its own schedule, so the dashboard stays fast:\n\n| Every | Job | \n|---|---|\n| 30 seconds | send alert and notification emails, Slack and webhook messages | \n| minute | evaluate alert rules; pick traces for automatic evaluation | \n| 5 minutes | hourly rollups for charts; read provider status pages | \n| 15 minutes | the Doctor's 15 detectors (retry storms, truncated outputs, cost spikes, …); spend by customer or feature; daily cost totals | \n| hour | delete traces past the project's retention; clean expired sessions and gateway cache | \n| day | fetch the provider's bill and compare it with traced spend; queue yesterday's traces worth reviewing | \n\nThe dashboard shows traces, sessions, costs, Doctor findings, alerts, evals and the gateway, for your projects only. Every project's data is isolated by Postgres row-level security, so even a bug in a query cannot show one team another team's traces.\n\nEverything above, the job queue included, runs on a single Postgres database. There is no ClickHouse, Redis or separate queue to install, back up or upgrade. The trade-off is scale: one api process handles about 2,000 spans per second on a laptop, which covers most teams; the [decision records](https://github.com/dheerajkarwasra/spanlight/blob/main/docs/decisions) explain each choice.\n\n| Path | What it is | \n|---|---|\n| [`backend/`](https://github.com/dheerajkarwasra/spanlight/blob/main/backend) | FastAPI, SQLAlchemy 2 (async), Alembic, the Postgres job queue and the CLI | \n| [`frontend/`](https://github.com/dheerajkarwasra/spanlight/blob/main/frontend) | React 19, TypeScript, TanStack Router and Query, Tailwind v4, Radix, Recharts | \n| [`sdks/python/`](https://github.com/dheerajkarwasra/spanlight/blob/main/sdks/python) | The `spanlight` SDK:`@observe` , spans, and the OpenAI and Anthropic wrappers | \n| [`sdks/typescript/`](https://github.com/dheerajkarwasra/spanlight/blob/main/sdks/typescript) | The `spanlight` npm package for Node.js:`observe` ,`withSpan` , and the OpenAI and Anthropic wrappers | \n| [`deploy/`](https://github.com/dheerajkarwasra/spanlight/blob/main/deploy) | Dockerfile, Caddyfile, Compose file, Prometheus example and Grafana dashboard | \n| [`docs-site/`](https://github.com/dheerajkarwasra/spanlight/blob/main/docs-site) | The documentation site, served by every instance at `/docs/` | \n| [`docs/`](https://github.com/dheerajkarwasra/spanlight/blob/main/docs) | Decision records, runbooks, security documents and API notes | \n\n- [Documentation](https://spanlight.dev/docs/) : quickstart, SDK, OTLP, gateway, cost, self-hosting, configuration and the API reference. Every instance also serves it at`/docs/` .\n- [Python SDK](https://github.com/dheerajkarwasra/spanlight/blob/main/sdks/python/README.md) : the`@observe` and`span` API, the wrappers and delivery guarantees.\n- [TypeScript SDK](https://github.com/dheerajkarwasra/spanlight/blob/main/sdks/typescript/README.md) : the Node.js wrappers,`observe` and`withSpan` , and delivery guarantees.\n- [API notes](https://github.com/dheerajkarwasra/spanlight/blob/main/docs/api-deviations.md) : error codes, ingestion rules and metric definitions as implemented.\n- [Runbooks](https://github.com/dheerajkarwasra/spanlight/blob/main/docs/runbooks) and[security](https://github.com/dheerajkarwasra/spanlight/blob/main/docs/security) : operating, restoring and securing an instance.\n- [Backend](https://github.com/dheerajkarwasra/spanlight/blob/main/backend/README.md) and[contributor](https://github.com/dheerajkarwasra/spanlight/blob/main/CONTRIBUTING.md) guides: local setup, the test database and the checks CI runs.\n\nHave a question or an idea? Start a thread in [Discussions](https://github.com/dheerajkarwasra/spanlight/discussions). Found a bug? [Open an issue](https://github.com/dheerajkarwasra/spanlight/issues/new/choose).", "url": "https://wpnews.pro/news/show-hn-spanlight-open-source-llm-observability-platform", "canonical_source": "https://github.com/dheerajkarwasra/spanlight", "published_at": "2026-11-12 13:12:46+00:00", "updated_at": "2026-11-12 13:21:46.115550+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "mlops", "developer-tools", "ai-agents"], "entities": ["Spanlight", "Postgres", "OpenAI", "Anthropic", "OpenTelemetry", "LiteLLM", "LangChain", "Vercel AI SDK"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-spanlight-open-source-llm-observability-platform", "markdown": "https://wpnews.pro/news/show-hn-spanlight-open-source-llm-observability-platform.md", "text": "https://wpnews.pro/news/show-hn-spanlight-open-source-llm-observability-platform.txt", "jsonld": "https://wpnews.pro/news/show-hn-spanlight-open-source-llm-observability-platform.jsonld"}}