{"slug": "vllm-monitoring-and-observability-with-opentelemetry", "title": "vLLM Monitoring and Observability with OpenTelemetry", "summary": "SigNoz published a guide for monitoring self-hosted vLLM inference servers by sending Prometheus metrics and OpenTelemetry request traces to SigNoz. The guide has operators scrape vLLM's /metrics endpoint on port 8000 every 30 seconds via an OpenTelemetry Collector, dropping the '_created' timestamp series that account for 111 of the 405 exposed series and roughly a quarter of ingested data. Traces are exported directly from vLLM to SigNoz using the OTEL_SERVICE_NAME and OTEL_EXPORT environment variables, bypassing the Collector.", "body_md": "## Overview\n\nvLLM is an inference server that you run yourself. It serves models over an OpenAI-compatible API. It reports its own health in two ways: Prometheus metrics on an HTTP endpoint, and OpenTelemetry traces for each request. This guide sends both to SigNoz.\n\nThe metrics answer questions that only the server can answer. How many tokens per second is the GPU producing? How full is the key-value cache, the memory pool that holds attention state for active requests? How many requests wait in the queue?\n\n## Prerequisites\n\n- A vLLM server that you can restart\n- An OpenTelemetry Collector that can reach the vLLM host. See [Install the OpenTelemetry Collector](https://signoz.io/docs/opentelemetry-collection-agents/get-started/)\n- An instance of SigNoz ([Cloud](https://signoz.io/teams/) or[Self-Hosted](https://signoz.io/docs/install/self-host/) )\n\n## Send vLLM metrics to SigNoz\n\n### Step 1: Confirm the metrics endpoint\n\nvLLM serves Prometheus metrics at `/metrics` on the same port as the API, and it\nneeds no flag to turn them on. Confirm the endpoint responds:\n\n```\ncurl -s http://localhost:8000/metrics | head\n```\n\nIf you changed the port, use that port instead of `8000`.\n\n### Step 2: Add a scrape job to the Collector\n\nAppend this scrape job to your existing `otel-collector-config.yaml`. Do not\nreplace the whole file.\n\n```\notel-collector-config.yaml\nreceivers:\n  prometheus:\n    config:\n      scrape_configs:\n        - job_name: vllm\n          scrape_interval: 30s\n          static_configs:\n            - targets: ['<vllm-host>:8000']\n          metric_relabel_configs:\n            # Drop the Prometheus _created timestamp series, which carry no signal.\n            - source_labels: [__name__]\n              regex: '.*_created'\n              action: drop\n```\n\n**Verify these values:**\n\n- `<vllm-host>` : The hostname or IP address of the machine that runs vLLM.\n\nKeep the `metric_relabel_configs` block. vLLM exposes a `_created` series next to\nevery counter and histogram, recording when the metric was first observed. On a\ndefault server these are 111 of the 405 exposed series. Dropping them removes\nabout a quarter of the ingested data and loses no signal.\n\nThen add the receiver to your metrics pipeline:\n\n```\notel-collector-config.yaml\nservice:\n  pipelines:\n    metrics:\n      receivers: [prometheus] # append prometheus to your existing receivers list\n      processors: [batch]\n      exporters: [otlphttp]\n```\n\n### Step 3: Restart the Collector\n\nRestart the Collector so that it loads the new scrape job, then watch its logs for scrape errors.\n\n```\nsudo systemctl restart otelcol-contrib\nsudo journalctl -u otelcol-contrib -f\ndocker compose up -d\ndocker compose logs -f\nkubectl rollout restart deployment/<release>-k8s-infra-otel-deployment -n <collector-namespace>\nkubectl logs -f deployment/<release>-k8s-infra-otel-deployment -n <collector-namespace>\n```\n\nThe SigNoz K8s Infra chart installs two Collectors. Add the scrape job to the\nDeployment, which handles cluster-wide scraping, rather than to the per-node\n`<release>-k8s-infra-otel-agent` DaemonSet. Replace `<release>` with your Helm\nrelease name and `<collector-namespace>` with the namespace that runs it. If you\nrun your own Collector under a different name, use that name instead.\n\nThe MSI installs the Collector as a Windows service named `otelcol-contrib`, with\n`OpenTelemetry Collector` as its display name. Run this in PowerShell:\n\n```\nRestart-Service -Name otelcol-contrib\nGet-EventLog -LogName Application -Source otelcol-contrib -Newest 20\n```\n\nSend a request to vLLM so that the server has something to report.\n\n## ## Export Request Traces (Optional)\n\nMetrics tell you how the server behaves as a whole. Traces show where time went inside a single request. Traces go straight from vLLM to SigNoz and do not pass through the Collector, so the scrape job above is unaffected.\n\nSet three environment variables, then add one flag to the launch command:\n\n```\nexport OTEL_SERVICE_NAME=vllm\nexport OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf\nexport OTEL_EXPORTER_OTLP_TRACES_HEADERS=\"signoz-ingestion-key=<your-ingestion-key>\"\n \nvllm serve <your-model> \\\n  --host 0.0.0.0 \\\n  --port 8000 \\\n  --otlp-traces-endpoint https://ingest.<region>.signoz.cloud:443/v1/traces\n```\n\n**Verify these values:**\n\n- `<your-model>` : The model you serve, for example`Qwen/Qwen2.5-7B-Instruct` .\n- `<region>` : Your[SigNoz Cloud region](https://signoz.io/docs/ingestion/signoz-cloud/overview/#endpoint) .\n- `<your-ingestion-key>` : Your SigNoz[ingestion key](https://signoz.io/docs/ingestion/signoz-cloud/keys/) .\n\nYou must set the protocol and header variables. vLLM sends no authentication header of its own, and it defaults to gRPC.\n\nRestart the server after you change the launch command.\n\n## Validate\n\nOpen **Metrics** > **Metrics Explorer** and filter with `service.name = 'vllm'`.\nThe Prometheus receiver takes that value from the `job_name` in your scrape job.\nMetrics appear within one scrape interval.\n\nIf you enabled traces, open **Traces** and filter on the value you set for\n`OTEL_SERVICE_NAME`. Each request produces one span named `llm_request`.\n\n## ## Metrics Reference\n\n| Metric | Type | What it tells you | \n|---|---|---|\n| `vllm:generation_tokens_total` | Counter | Output tokens produced. Apply a rate to get tokens per second. | \n| `vllm:prompt_tokens_total` | Counter | Prefill tokens processed. | \n| `vllm:prompt_tokens_cached_total` | Counter | Prompt tokens served from cache rather than recomputed. | \n| `vllm:time_to_first_token_seconds` | Histogram | Delay before the first output token. Drives how fast a streaming response feels. | \n| `vllm:inter_token_latency_seconds` | Histogram | Delay between output tokens after the first one. | \n| `vllm:e2e_request_latency_seconds` | Histogram | Total request duration. | \n| `vllm:request_queue_time_seconds` | Histogram | Time a request spent waiting before it ran. | \n| `vllm:num_requests_running` | Gauge | Requests in the current execution batch. | \n| `vllm:num_requests_waiting` | Gauge | Requests waiting to start. A number above zero means the server is saturated. | \n| `vllm:kv_cache_usage_perc` | Gauge | Fraction of the key-value cache in use, from 0 to 1. | \n| `vllm:prefix_cache_queries_total` | Counter | Prompt tokens looked up in the prefix cache. | \n| `vllm:prefix_cache_hits_total` | Counter | Prompt tokens found in the prefix cache. | \n| `vllm:num_preemptions_total` | Counter | Requests the scheduler preempted and re-ran. | \n\nEvery metric carries `model_name` and `engine` labels. Group by `model_name` when\none server hosts several models.\n\nvLLM has no ready-made throughput gauge, so read tokens per second as a rate over\n`vllm:generation_tokens_total`. It also has no prefix cache hit rate metric.\nDivide `vllm:prefix_cache_hits_total` by `vllm:prefix_cache_queries_total` to get\none.\n\nMetric names vary between vLLM releases. The names above come from v0.29.0. For\nthe full list, see the [vLLM metrics reference](https://docs.vllm.ai/en/latest/usage/metrics.html).\n\n## ## Troubleshooting\n\n### The metrics endpoint returns 404\n\nLikely cause: the request went to the wrong port, or the server runs behind a\nproxy that does not forward `/metrics`.\n\nFix: use the port vLLM serves the API on, which is `8000` by default.\n\nVerify: `curl -s http://localhost:8000/metrics | head` prints lines that start\nwith `# HELP`.\n\n### The Collector logs a connection refused error\n\nLikely cause: vLLM is bound to `127.0.0.1`, so nothing outside the host can reach\nit.\n\nFix: start the server with `--host 0.0.0.0`, and open port `8000` to the Collector\nhost in your firewall.\n\nVerify: run `curl -s http://<vllm-host>:8000/metrics | head` from the machine that\nruns the Collector.\n\n### Metrics arrive but every value is zero\n\nLikely cause: the server has served no requests since it started. Counters and gauges stay at zero until traffic arrives.\n\nFix: send a request to the server.\n\nVerify: `vllm:prompt_tokens_total` rises in Metrics Explorer.\n\n### Traces do not appear\n\nLikely cause: the server is still exporting over gRPC, which cannot reach SigNoz Cloud.\n\nFix: set `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf` and\n`OTEL_EXPORTER_OTLP_TRACES_HEADERS` in the environment of the vLLM process, then\nrestart it.\n\nVerify: spans named `llm_request` appear in the Traces explorer.\n\n## Next steps\n\n- [vLLM dashboard template](https://signoz.io/docs/dashboards/dashboard-templates/vllm-dashboard/)\n- [Set up alerts](https://signoz.io/docs/setup-alerts-notification/) on queue depth and time to first token\n- [LLM observability](https://signoz.io/docs/llm-observability/) for tracing the application that calls this server\n\n## Get Help\n\nIf you need help with the steps in this topic, please reach out to us on [SigNoz Community Slack](https://signoz.io/slack/). If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at [cloud-support@signoz.io](mailto:cloud-support@signoz.io).", "url": "https://wpnews.pro/news/vllm-monitoring-and-observability-with-opentelemetry", "canonical_source": "https://signoz.io/docs/vllm-observability", "published_at": "2026-09-20 00:00:00+00:00", "updated_at": "2026-09-22 10:55:58.706708+00:00", "lang": "en", "topics": ["ai-infrastructure", "mlops", "ai-tools"], "entities": ["vLLM", "SigNoz", "OpenTelemetry", "OpenTelemetry Collector", "Prometheus"], "alternates": {"html": "https://wpnews.pro/news/vllm-monitoring-and-observability-with-opentelemetry", "markdown": "https://wpnews.pro/news/vllm-monitoring-and-observability-with-opentelemetry.md", "text": "https://wpnews.pro/news/vllm-monitoring-and-observability-with-opentelemetry.txt", "jsonld": "https://wpnews.pro/news/vllm-monitoring-and-observability-with-opentelemetry.jsonld"}}