Open WebUI Observability & Monitoring with OpenTelemetry

SigNoz announced that Open WebUI, a self-hosted AI interface, now supports observability and monitoring through OpenTelemetry, enabling real-time tracing, metrics, and logs for API requests, database queries, and outbound calls without code changes. Users can optionally integrate OpenLIT via Open WebUI Pipelines to track LLM token usage and cost per model. The setup requires a SigNoz Cloud account or self-hosted instance, Docker, and a recent Open WebUI deployment, with configuration via an OpenTelemetry Collector and Docker Compose.

What is Open WebUI Monitoring? Open WebUI monitoring gives you real-time visibility into your self-hosted Open WebUI https://openwebui.com/ deployment using OpenTelemetry https://opentelemetry.io/ . Open WebUI's backend ships with OpenTelemetry built in, auto-instrumenting FastAPI, SQLAlchemy, and its HTTP clients, so it emits traces, metrics, and logs for API requests, database queries, and outbound calls without any code changes. If you also want GenAI telemetry model, tokens, and cost for every LLM call, you can optionally add OpenLIT https://openlit.io/ through Open WebUI Pipelines. With full Open WebUI observability in SigNoz, you can trace API requests end to end, monitor database and endpoint latency, track LLM token usage and cost per model, set alerts on errors and latency, and keep your deployment reliable. Prerequisites - A SigNoz Cloud account https://signoz.io/teams/ with an active ingestion key or Self Hosted SigNoz instance https://signoz.io/docs/install/self-host/ - Docker and Docker Compose installed and running on your system - A running Open WebUI deployment a recent version with OpenTelemetry support . Follow the Open WebUI docs https://docs.openwebui.com/ if you don't have it yet - Only for the optional LLM telemetry add-on: an OpenAI API key or another OpenAI-compatible provider Monitor Open WebUI with OpenTelemetry Open WebUI's OpenTelemetry stack is bundled inside its backend and turned on with environment variables, so there is no instrumentation package to install. Open WebUI's exporter does not attach custom headers, so it sends OTLP to a lightweight OpenTelemetry Collector that adds your SigNoz ingestion key and forwards the data. The three steps below wire both together with Docker Compose and get data into SigNoz. Once that works, you can optionally add LLM telemetry with OpenLIT add-llm-telemetry-with-openlit-optional for per-call model, token, and cost data. Step 1: Configure the OpenTelemetry Collector Create otel-collector-config.yaml . The Collector receives OTLP from Open WebUI and OpenLIT and exports it to SigNoz with your ingestion key. otel-collector-config.yaml receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: batch: timeout: 5s exporters: otlp/signoz: endpoint: https://ingest.<region .signoz.cloud:443 tls: insecure: false headers: signoz-ingestion-key: <your-ingestion-key service: pipelines: traces: receivers: otlp processors: batch exporters: otlp/signoz metrics: receivers: otlp processors: batch exporters: otlp/signoz logs: receivers: otlp processors: batch exporters: otlp/signoz Verify these values: <region : Your SigNoz Cloud region https://signoz.io/docs/ingestion/signoz-cloud/overview/ endpoint . <your-ingestion-key : Your SigNoz ingestion key https://signoz.io/docs/ingestion/signoz-cloud/keys/ . Step 2: Start the stack with Docker Compose Create docker-compose.yaml . It runs Open WebUI and the Collector on one network, with Open WebUI exporting its native OTLP to the Collector. services: otel-collector: image: otel/opentelemetry-collector-contrib:latest command: "--config=/etc/otel-collector-config.yaml" volumes: - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml:ro ports: - "4317:4317" - "4318:4318" open-webui: image: ghcr.io/open-webui/open-webui:main depends on: otel-collector environment: ENABLE OTEL: "true" ENABLE OTEL TRACES: "true" ENABLE OTEL METRICS: "true" ENABLE OTEL LOGS: "true" OTEL SERVICE NAME: "open-webui" OTEL EXPORTER OTLP ENDPOINT: "http://otel-collector:4318" OTEL OTLP SPAN EXPORTER: "http" OTEL EXPORTER OTLP INSECURE: "true" ports: - "3000:8080" volumes: - open-webui-data:/app/backend/data volumes: open-webui-data: Start everything: docker compose up -d The Open WebUI environment variables above control the native export: | Variable | Required | Value | What it does | |---|---|---|---| ENABLE OTEL | Yes | true | Turns on Open WebUI's OpenTelemetry stack. | ENABLE OTEL TRACES / ENABLE OTEL METRICS / ENABLE OTEL LOGS | No | true default false | Enable each signal. | OTEL SERVICE NAME | No | open-webui | The service.name Open WebUI reports. | OTEL EXPORTER OTLP ENDPOINT | Yes | http://otel-collector:4318 | The Collector's OTLP HTTP endpoint. | OTEL OTLP SPAN EXPORTER | No | http default grpc | Selects the OTLP/HTTP exporter for traces. | OTEL EXPORTER OTLP INSECURE | Yes | true | Plaintext OTLP to the Collector on the local network. | Step 3: Generate telemetry - Open Open WebUI at http://localhost:3000 and create the first admin account. - Browse the app and start a chat to exercise the API. App activity produces FastAPI, database, and HTTP spans. An idle instance emits little, so use the app before checking SigNoz, and allow a few seconds for export. Your Open WebUI telemetry is now in SigNoz. Add LLM Telemetry with OpenLIT Optional Add LLM Telemetry with OpenLIT Optional add-llm-telemetry-with-openlit-optional Open WebUI's native instrumentation captures API, database, and HTTP spans, but not token usage, cost, or the model behind each call. To capture those, run an Open WebUI Pipelines server with an OpenLIT-instrumented OpenAI client. This adds a container and some manual wiring, so set it up after the steps above are sending data. Create the pipeline Create pipelines/openai openlit pipeline.py : python """ title: OpenAI GenAI OpenLIT requirements: openlit, openai """ import os from typing import Generator, List, Union from pydantic import BaseModel class Pipeline: class Valves BaseModel : OPENAI API KEY: str = "" OPENAI MODEL: str = "gpt-4o-mini" OTLP ENDPOINT: str = "http://otel-collector:4318" def init self : self.id = "openai-genai" self.name = "OpenAI GenAI OpenLIT " self.valves = self.Valves OPENAI API KEY=os.getenv "OPENAI API KEY", "" , OTLP ENDPOINT=os.getenv "OTLP ENDPOINT", "http://otel-collector:4318" , self.client = None async def on startup self : import openlit from openai import OpenAI Instruments the OpenAI client and exports gen ai. spans to the Collector. openlit.init otlp endpoint=self.valves.OTLP ENDPOINT, application name="open-webui-llm" self.client = OpenAI api key=self.valves.OPENAI API KEY def pipe self, user message: str, model id: str, messages: List dict , body: dict - Union str, Generator : resp = self.client.chat.completions.create model=self.valves.OPENAI MODEL, messages=messages, stream=True, stream options={"include usage": True}, for chunk in resp: if chunk.choices and chunk.choices 0 .delta.content: yield chunk.choices 0 .delta.content Add the Pipelines service Add this service to the docker-compose.yaml from Step 2. It exports GenAI telemetry to the same Collector: pipelines: image: ghcr.io/open-webui/pipelines:main depends on: otel-collector environment: OPENAI API KEY: ${OPENAI API KEY} OTLP ENDPOINT: "http://otel-collector:4318" volumes: - ./pipelines:/app/pipelines ports: - "9099:9099" Then add pipelines to the open-webui service's depends on so it starts after the Pipelines server: depends on: otel-collector, pipelines Restart the stack: docker compose up -d Connect the pipeline - In Open WebUI, go to Admin Panel → Settings → Connections and add an OpenAI-compatible connection with the URL http://pipelines:9099 and the API key 0p3n-w3bu . - Start a chat with the OpenAI GenAI OpenLIT model. Each chat now produces GenAI spans carrying gen ai. attributes model, token usage, cost under the service open-webui-llm . View Open WebUI Traces in SigNoz Once configured, Open WebUI emits traces for app requests under the service open-webui , plus traces for LLM calls under open-webui-llm if you added the OpenLIT step. Traces are available in SigNoz under the Traces tab: Click a trace to see the detailed view, including all spans, events, and attributes. With OpenLIT added, LLM spans carry gen ai. attributes such as the model, token usage, cost, and the prompt and response messages. View Open WebUI Metrics in SigNoz Open WebUI also emits OpenTelemetry metrics. Explore them under the Metrics tab, where you can see HTTP client and server durations, alongside GenAI metrics such as token usage, request cost, and time to first token if you added OpenLIT: Select a metric to inspect its metadata, type, and attributes: Open WebUI Observability Dashboard You can also import our custom Open WebUI dashboard https://signoz.io/docs/dashboards/dashboard-templates/open-webui-dashboard/ , which provides ready-made panels for application health, database performance, and LLM usage, tokens, cost, and latency by model, along with import instructions to get started quickly. The LLM panels populate once you add the optional OpenLIT step add-llm-telemetry-with-openlit-optional . Troubleshooting Open WebUI Observability Troubleshooting Open WebUI Observability troubleshooting-open-webui-observability No traces or metrics in SigNoz - Confirm you used the app after starting the stack. An idle instance emits little telemetry. - Check the Collector logs for export errors: docker compose logs otel-collector . - Verify ENABLE OTEL=true is set on the Open WebUI container and that OTEL EXPORTER OTLP ENDPOINT points at the Collector. - OpenTelemetry batches data before sending, so wait 10-30 seconds after generating activity. No LLM gen ai spans - These spans only appear if you completed the optional OpenLIT step add-llm-telemetry-with-openlit-optional . - Confirm the Pipelines server loaded the pipeline and that Open WebUI's OpenAI connection points at http://pipelines:9099 with the API key 0p3n-w3bu . - Make sure you chatted with the OpenAI GenAI OpenLIT model, not another connection. - Check the Pipelines logs: docker compose logs pipelines . Auth errors 401 / 403 Re-check the ingestion key in the Collector's signoz-ingestion-key header. It must be the exact key from your SigNoz Ingestion Settings, with no extra spaces or quotes, and the region in the endpoint must match your account. Setup OpenTelemetry Collector Optional Setup OpenTelemetry Collector Optional setup-opentelemetry-collector-optional What is the OpenTelemetry Collector? Think of the OTel Collector as a middleman between your app and SigNoz. Instead of your application sending data directly to SigNoz, it sends everything to the Collector first, which then forwards it along. This guide already routes Open WebUI through a Collector because Open WebUI cannot attach the SigNoz ingestion key itself. Why use it? Cleaning up data - Filter out noisy traces you don't care about, or remove sensitive info before it leaves your servers. Keeping your app lightweight - Let the Collector handle batching, retries, and compression instead of your application code. Adding context automatically - The Collector can tag your data with useful info like which Kubernetes pod or cloud region it came from. Future flexibility - Want to send data to multiple backends later? The Collector makes that easy without changing your app. For more details, see Why use the OpenTelemetry Collector? https://signoz.io/docs/opentelemetry-collection-agents/opentelemetry-collector/why-to-use-collector/ and the Collector configuration guide https://signoz.io/docs/opentelemetry-collection-agents/opentelemetry-collector/configuration/ . Related integrations Instrument the rest of your self-hosted inference stack: Monitor Ollama with OpenTelemetry https://signoz.io/docs/ollama-monitoring/ - track local model inference latency, tokens, and resource usage Hugging Face observability with OpenTelemetry https://signoz.io/docs/huggingface-observability/ - trace inference API calls and local pipeline runs Monitor Baseten with OpenTelemetry https://signoz.io/docs/baseten-monitoring/ - trace model deployments and inference calls running on Baseten LangChain and LangGraph observability with OpenTelemetry https://signoz.io/docs/langchain-observability/ - trace chains, agents, graph nodes, and tool calls LiteLLM observability with OpenTelemetry https://signoz.io/docs/litellm-observability/ - trace calls across 100+ models through either the SDK or the proxy Browse all LLM observability integrations https://signoz.io/docs/llm-observability/ to instrument the rest of your stack.