# vLLM Monitoring and Observability with OpenTelemetry > Source: > Published: 2026-09-20 00:00:00+00:00 ## Overview vLLM is an inference server that you run yourself. It serves models over an OpenAI-compatible API. It reports its own health in two ways: Prometheus metrics on an HTTP endpoint, and OpenTelemetry traces for each request. This guide sends both to SigNoz. The metrics answer questions that only the server can answer. How many tokens per second is the GPU producing? How full is the key-value cache, the memory pool that holds attention state for active requests? How many requests wait in the queue? ## Prerequisites - A vLLM server that you can restart - An OpenTelemetry Collector that can reach the vLLM host. See [Install the OpenTelemetry Collector](https://signoz.io/docs/opentelemetry-collection-agents/get-started/) - An instance of SigNoz ([Cloud](https://signoz.io/teams/) or[Self-Hosted](https://signoz.io/docs/install/self-host/) ) ## Send vLLM metrics to SigNoz ### Step 1: Confirm the metrics endpoint vLLM serves Prometheus metrics at `/metrics` on the same port as the API, and it needs no flag to turn them on. Confirm the endpoint responds: ``` curl -s http://localhost:8000/metrics | head ``` If you changed the port, use that port instead of `8000`. ### Step 2: Add a scrape job to the Collector Append this scrape job to your existing `otel-collector-config.yaml`. Do not replace the whole file. ``` otel-collector-config.yaml receivers: prometheus: config: scrape_configs: - job_name: vllm scrape_interval: 30s static_configs: - targets: [':8000'] metric_relabel_configs: # Drop the Prometheus _created timestamp series, which carry no signal. - source_labels: [__name__] regex: '.*_created' action: drop ``` **Verify these values:** - `` : The hostname or IP address of the machine that runs vLLM. Keep the `metric_relabel_configs` block. vLLM exposes a `_created` series next to every counter and histogram, recording when the metric was first observed. On a default server these are 111 of the 405 exposed series. Dropping them removes about a quarter of the ingested data and loses no signal. Then add the receiver to your metrics pipeline: ``` otel-collector-config.yaml service: pipelines: metrics: receivers: [prometheus] # append prometheus to your existing receivers list processors: [batch] exporters: [otlphttp] ``` ### Step 3: Restart the Collector Restart the Collector so that it loads the new scrape job, then watch its logs for scrape errors. ``` sudo systemctl restart otelcol-contrib sudo journalctl -u otelcol-contrib -f docker compose up -d docker compose logs -f kubectl rollout restart deployment/-k8s-infra-otel-deployment -n kubectl logs -f deployment/-k8s-infra-otel-deployment -n ``` The SigNoz K8s Infra chart installs two Collectors. Add the scrape job to the Deployment, which handles cluster-wide scraping, rather than to the per-node `-k8s-infra-otel-agent` DaemonSet. Replace `` with your Helm release name and `` with the namespace that runs it. If you run your own Collector under a different name, use that name instead. The MSI installs the Collector as a Windows service named `otelcol-contrib`, with `OpenTelemetry Collector` as its display name. Run this in PowerShell: ``` Restart-Service -Name otelcol-contrib Get-EventLog -LogName Application -Source otelcol-contrib -Newest 20 ``` Send a request to vLLM so that the server has something to report. ## ## Export Request Traces (Optional) Metrics tell you how the server behaves as a whole. Traces show where time went inside a single request. Traces go straight from vLLM to SigNoz and do not pass through the Collector, so the scrape job above is unaffected. Set three environment variables, then add one flag to the launch command: ``` export OTEL_SERVICE_NAME=vllm export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf export OTEL_EXPORTER_OTLP_TRACES_HEADERS="signoz-ingestion-key=" vllm serve \ --host 0.0.0.0 \ --port 8000 \ --otlp-traces-endpoint https://ingest..signoz.cloud:443/v1/traces ``` **Verify these values:** - `` : The model you serve, for example`Qwen/Qwen2.5-7B-Instruct` . - `` : Your[SigNoz Cloud region](https://signoz.io/docs/ingestion/signoz-cloud/overview/#endpoint) . - `` : Your SigNoz[ingestion key](https://signoz.io/docs/ingestion/signoz-cloud/keys/) . You must set the protocol and header variables. vLLM sends no authentication header of its own, and it defaults to gRPC. Restart the server after you change the launch command. ## Validate Open **Metrics** > **Metrics Explorer** and filter with `service.name = 'vllm'`. The Prometheus receiver takes that value from the `job_name` in your scrape job. Metrics appear within one scrape interval. If you enabled traces, open **Traces** and filter on the value you set for `OTEL_SERVICE_NAME`. Each request produces one span named `llm_request`. ## ## Metrics Reference | Metric | Type | What it tells you | |---|---|---| | `vllm:generation_tokens_total` | Counter | Output tokens produced. Apply a rate to get tokens per second. | | `vllm:prompt_tokens_total` | Counter | Prefill tokens processed. | | `vllm:prompt_tokens_cached_total` | Counter | Prompt tokens served from cache rather than recomputed. | | `vllm:time_to_first_token_seconds` | Histogram | Delay before the first output token. Drives how fast a streaming response feels. | | `vllm:inter_token_latency_seconds` | Histogram | Delay between output tokens after the first one. | | `vllm:e2e_request_latency_seconds` | Histogram | Total request duration. | | `vllm:request_queue_time_seconds` | Histogram | Time a request spent waiting before it ran. | | `vllm:num_requests_running` | Gauge | Requests in the current execution batch. | | `vllm:num_requests_waiting` | Gauge | Requests waiting to start. A number above zero means the server is saturated. | | `vllm:kv_cache_usage_perc` | Gauge | Fraction of the key-value cache in use, from 0 to 1. | | `vllm:prefix_cache_queries_total` | Counter | Prompt tokens looked up in the prefix cache. | | `vllm:prefix_cache_hits_total` | Counter | Prompt tokens found in the prefix cache. | | `vllm:num_preemptions_total` | Counter | Requests the scheduler preempted and re-ran. | Every metric carries `model_name` and `engine` labels. Group by `model_name` when one server hosts several models. vLLM has no ready-made throughput gauge, so read tokens per second as a rate over `vllm:generation_tokens_total`. It also has no prefix cache hit rate metric. Divide `vllm:prefix_cache_hits_total` by `vllm:prefix_cache_queries_total` to get one. Metric names vary between vLLM releases. The names above come from v0.29.0. For the full list, see the [vLLM metrics reference](https://docs.vllm.ai/en/latest/usage/metrics.html). ## ## Troubleshooting ### The metrics endpoint returns 404 Likely cause: the request went to the wrong port, or the server runs behind a proxy that does not forward `/metrics`. Fix: use the port vLLM serves the API on, which is `8000` by default. Verify: `curl -s http://localhost:8000/metrics | head` prints lines that start with `# HELP`. ### The Collector logs a connection refused error Likely cause: vLLM is bound to `127.0.0.1`, so nothing outside the host can reach it. Fix: start the server with `--host 0.0.0.0`, and open port `8000` to the Collector host in your firewall. Verify: run `curl -s http://:8000/metrics | head` from the machine that runs the Collector. ### Metrics arrive but every value is zero Likely cause: the server has served no requests since it started. Counters and gauges stay at zero until traffic arrives. Fix: send a request to the server. Verify: `vllm:prompt_tokens_total` rises in Metrics Explorer. ### Traces do not appear Likely cause: the server is still exporting over gRPC, which cannot reach SigNoz Cloud. Fix: set `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf` and `OTEL_EXPORTER_OTLP_TRACES_HEADERS` in the environment of the vLLM process, then restart it. Verify: spans named `llm_request` appear in the Traces explorer. ## Next steps - [vLLM dashboard template](https://signoz.io/docs/dashboards/dashboard-templates/vllm-dashboard/) - [Set up alerts](https://signoz.io/docs/setup-alerts-notification/) on queue depth and time to first token - [LLM observability](https://signoz.io/docs/llm-observability/) for tracing the application that calls this server ## Get Help If you need help with the steps in this topic, please reach out to us on [SigNoz Community Slack](https://signoz.io/slack/). If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at [cloud-support@signoz.io](mailto:cloud-support@signoz.io).