{"slug": "sglang-monitoring-and-observability-with-opentelemetry", "title": "SGLang Monitoring and Observability with OpenTelemetry", "summary": "SigNoz published a guide for monitoring self-hosted SGLang inference servers by sending Prometheus metrics and OpenTelemetry request traces to its observability platform. The setup requires launching SGLang with the --enable-metrics flag, which exposes metrics at /metrics on the API port (default 30000), and adding a Prometheus scrape job with a 30s scrape_interval to an OpenTelemetry Collector's otel-collector-config.yaml. Optional request tracing is enabled with the --enable-trace and --otlp flags plus the OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf and OTEL_EXPORTER_OTLP_TRACES_HEADERS environment variables, sending traces directly from SGLang to SigNoz without passing through the Collector.", "body_md": "## Overview\n\nSGLang is an inference server that you run yourself. It serves models over an OpenAI-compatible API. It reports its own health in two ways: Prometheus metrics on an HTTP endpoint, and OpenTelemetry traces for each request. This guide sends both to SigNoz.\n\nThe metrics answer questions that only the server can answer. How many tokens per second is the GPU producing? How full is the key-value cache, the memory pool that holds attention state for active requests? How many requests wait in the queue?\n\n## Prerequisites\n\n- An SGLang server that you can restart\n- An OpenTelemetry Collector that can reach the SGLang host. See [Install the OpenTelemetry Collector](https://signoz.io/docs/opentelemetry-collection-agents/get-started/)\n- An instance of SigNoz ([Cloud](https://signoz.io/teams/) or[Self-Hosted](https://signoz.io/docs/install/self-host/) )\n\n## Send SGLang metrics to SigNoz\n\n### Step 1: Start SGLang with metrics enabled\n\nSGLang does not expose metrics by default. Add `--enable-metrics` to the launch\ncommand:\n\n```\npython -m sglang.launch_server \\\n  --model-path <your-model> \\\n  --host 0.0.0.0 \\\n  --port 30000 \\\n  --enable-metrics\n```\n\n**Verify these values:**\n\n- `<your-model>` : The model you serve, for example`Qwen/Qwen2.5-7B-Instruct` .\n\nSGLang serves metrics at `/metrics` on the same port as the API. Confirm the endpoint\nresponds before you move on:\n\n```\ncurl -s http://localhost:30000/metrics | head\n```\n\n### Step 2: Add a scrape job to the Collector\n\nAppend this scrape job to your existing `otel-collector-config.yaml`. Do not\nreplace the whole file.\n\n```\notel-collector-config.yaml\nreceivers:\n  prometheus:\n    config:\n      scrape_configs:\n        - job_name: sglang\n          scrape_interval: 30s\n          static_configs:\n            - targets: ['<sglang-host>:30000']\n```\n\n**Verify these values:**\n\n- `<sglang-host>` : The hostname or IP address of the machine that runs SGLang.\n\nThen add the receiver to your metrics pipeline:\n\n```\notel-collector-config.yaml\nservice:\n  pipelines:\n    metrics:\n      receivers: [prometheus] # append prometheus to your existing receivers list\n      processors: [batch]\n      exporters: [otlphttp]\n```\n\n### Step 3: Restart the Collector\n\nRestart the Collector so that it loads the new scrape job, then watch its logs for scrape errors.\n\n```\nsudo systemctl restart otelcol-contrib\nsudo journalctl -u otelcol-contrib -f\ndocker compose up -d\ndocker compose logs -f\nkubectl rollout restart deployment/<release>-k8s-infra-otel-deployment -n <collector-namespace>\nkubectl logs -f deployment/<release>-k8s-infra-otel-deployment -n <collector-namespace>\n```\n\nThe SigNoz K8s Infra chart installs two Collectors. Add the scrape job to the\nDeployment, which handles cluster-wide scraping, rather than to the per-node\n`<release>-k8s-infra-otel-agent` DaemonSet. Replace `<release>` with your Helm\nrelease name and `<collector-namespace>` with the namespace that runs it. If you\nrun your own Collector under a different name, use that name instead.\n\nThe MSI installs the Collector as a Windows service named `otelcol-contrib`, with\n`OpenTelemetry Collector` as its display name. Run this in PowerShell:\n\n```\nRestart-Service -Name otelcol-contrib\nGet-EventLog -LogName Application -Source otelcol-contrib -Newest 20\n```\n\nSend a request to SGLang so that the server has something to report.\n\n## ## Export Request Traces (Optional)\n\nMetrics tell you how the server behaves as a whole. Traces show where time went inside a single request. Traces go straight from SGLang to SigNoz and do not pass through the Collector, so the scrape job above is unaffected.\n\nSet two environment variables, then add two flags to the launch command from Step 1:\n\n```\nexport OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf\nexport OTEL_EXPORTER_OTLP_TRACES_HEADERS=\"signoz-ingestion-key=<your-ingestion-key>\"\n \npython -m sglang.launch_server \\\n  --model-path <your-model> \\\n  --host 0.0.0.0 \\\n  --port 30000 \\\n  --enable-metrics \\\n  --enable-trace \\\n  --otlp-traces-endpoint https://ingest.<region>.signoz.cloud:443/v1/traces\n```\n\n**Verify these values:**\n\n- `<region>` : Your[SigNoz Cloud region](https://signoz.io/docs/ingestion/signoz-cloud/overview/#endpoint) .\n- `<your-ingestion-key>` : Your SigNoz[ingestion key](https://signoz.io/docs/ingestion/signoz-cloud/keys/) .\n\nYou must set both environment variables. SGLang sends no authentication header of its own, and it defaults to gRPC.\n\nRestart SGLang after you change the launch command.\n\n## Validate\n\nOpen **Metrics** > **Metrics Explorer** and filter with `service.name = 'sglang'`.\nThe Prometheus receiver takes that value from the `job_name` in your scrape job.\nMetrics appear within one scrape interval.\n\nIf you enabled traces, open **Traces** and filter with the same\n`service.name = 'sglang'`. SGLang sets that name itself and ignores\n`OTEL_SERVICE_NAME`, so every SGLang server reports as `sglang`.\n\n## ## Metrics Reference\n\n| Metric | Type | What it tells you | \n|---|---|---|\n| `sglang:gen_throughput` | Gauge | Output tokens generated per second. This is the tokens-per-second figure. | \n| `sglang:prompt_tokens_total` | Counter | Prefill tokens processed. | \n| `sglang:generation_tokens_total` | Counter | Output tokens produced. | \n| `sglang:time_to_first_token_seconds` | Histogram | Delay before the first output token. Drives how fast a streaming response feels. | \n| `sglang:inter_token_latency_seconds` | Histogram | Delay between output tokens after the first one. | \n| `sglang:e2e_request_latency_seconds` | Histogram | Total request duration. | \n| `sglang:num_running_reqs` | Gauge | Requests the server is decoding right now. | \n| `sglang:num_queue_reqs` | Gauge | Requests waiting to start. A number above zero means the server is saturated. | \n| `sglang:token_usage` | Gauge | Fraction of the key-value cache pool in use, from 0 to 1. | \n| `sglang:cache_hit_rate` | Gauge | Fraction of prefill tokens served from the prefix cache. | \n\nEvery metric carries `model_name`, `engine_type`, `tp_rank`, `pp_rank`, and\n`moe_ep_rank` labels. The token counters add `is_streaming`. Group by `model_name`\nwhen one server hosts several models.\n\nOn a server that uses tensor parallelism, only rank 0 records request metrics by\ndefault, so a sum across ranks counts each request once. The\n`--enable-metrics-for-all-schedulers` flag makes every rank record separately,\nwhich doubles those sums unless you filter on `tp_rank`. Add `tp_rank = '0'` to\nyour own queries against the scheduler gauges. The dashboard template already\ndoes this.\n\nSGLang also splits the queue gauges when you run it with\n`--enable-priority-scheduling`. It reports the total under `priority = ''` and a\nbreakdown under `priority = '<int>'`, so summing every series counts each request\ntwice. Filter on `priority = ''` for totals. This filter also works on servers\nthat run without priority scheduling, where the label is absent.\n\nMetric names vary between SGLang releases. The names above come from v0.5.19. For\nthe full list, see the [SGLang production metrics reference](https://docs.sglang.io/references/production_metrics).\n\n## ## Troubleshooting\n\n### The metrics endpoint returns 404\n\nLikely cause: the server started without `--enable-metrics`.\n\nFix: add the flag and restart SGLang.\n\nVerify: `curl -s http://localhost:30000/metrics | head` prints lines that start\nwith `# HELP`.\n\n### The Collector logs a connection refused error\n\nLikely cause: SGLang is bound to `127.0.0.1`, so nothing outside the host can\nreach it.\n\nFix: start the server with `--host 0.0.0.0`, and open port `30000` to the\nCollector host in your firewall.\n\nVerify: run `curl -s http://<sglang-host>:30000/metrics | head` from the machine\nthat runs the Collector.\n\n### Metrics arrive but every value is zero\n\nLikely cause: the server has served no requests since it started. Counters and gauges stay at zero until traffic arrives.\n\nFix: send a request to the server.\n\nVerify: `sglang:prompt_tokens_total` rises in Metrics Explorer.\n\n### Traces do not appear\n\nLikely cause: the server is still exporting over gRPC, which cannot reach SigNoz Cloud.\n\nFix: set `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf` and\n`OTEL_EXPORTER_OTLP_TRACES_HEADERS` in the environment of the SGLang process, then\nrestart it.\n\nVerify: spans with `service.name = sglang` appear in the Traces explorer.\n\n## Next steps\n\n- [SGLang dashboard template](https://signoz.io/docs/dashboards/dashboard-templates/sglang-dashboard/)\n- [Set up alerts](https://signoz.io/docs/setup-alerts-notification/) on queue depth and time to first token\n- [LLM observability](https://signoz.io/docs/llm-observability/) for tracing the application that calls this server\n\n## Get Help\n\nIf you need help with the steps in this topic, please reach out to us on [SigNoz Community Slack](https://signoz.io/slack/). If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at [cloud-support@signoz.io](mailto:cloud-support@signoz.io).", "url": "https://wpnews.pro/news/sglang-monitoring-and-observability-with-opentelemetry", "canonical_source": "https://signoz.io/docs/sglang-observability", "published_at": "2026-09-17 00:00:00+00:00", "updated_at": "2026-09-22 11:26:29.163739+00:00", "lang": "en", "topics": ["ai-infrastructure", "mlops", "ai-tools"], "entities": ["SGLang", "SigNoz", "OpenTelemetry", "Prometheus", "OpenTelemetry Collector", "Qwen/Qwen2.5-7B-Instruct"], "alternates": {"html": "https://wpnews.pro/news/sglang-monitoring-and-observability-with-opentelemetry", "markdown": "https://wpnews.pro/news/sglang-monitoring-and-observability-with-opentelemetry.md", "text": "https://wpnews.pro/news/sglang-monitoring-and-observability-with-opentelemetry.txt", "jsonld": "https://wpnews.pro/news/sglang-monitoring-and-observability-with-opentelemetry.jsonld"}}