{"slug": "migrating-from-node-exporter-to-grafana-alloy-one-server-at-a-time", "title": "Migrating from node_exporter to Grafana Alloy, One Server at a Time", "summary": "A developer from the Irin monitoring stack details a migration from node_exporter to Grafana Alloy, consolidating telemetry into a single agent. The process involves installing Alloy alongside node_exporter, configuring its prometheus.exporter.unix component, and switching from Prometheus scraping to remote write, one server at a time. This approach minimizes risk and preserves visibility while enabling future logs and traces without additional agents.", "body_md": "If you've been monitoring Linux servers for any length of time, there's a good chance **node_exporter** was the first thing you installed. It's lightweight, reliable, and exposes a huge amount of machine metrics for Prometheus to scrape. For years, it has been the default answer.\n\nAs your infrastructure grows, though, your monitoring stack usually grows with it. First comes log collection. Then traces. Before long you're running `node_exporter`\n\n, a log shipper, and maybe another telemetry agent. Each component has its own configuration, service unit, upgrade cycle, and failure modes.\n\nGrafana Alloy changes that by consolidating those responsibilities into a single telemetry agent.\n\nThis post walks through migrating from `node_exporter`\n\nto Alloy on a real fleet, one server at a time, while maintaining continuous visibility throughout the process. These are the exact steps that survived contact with production on the Irin monitoring stack, not the idealized version that looks clean in a diagram.\n\n**TL;DR** If you're already running `node_exporter`\n\n, don't replace it overnight. Install Grafana Alloy alongside it, configure Alloy's built-in prometheus.exporter.unix component, verify that metrics are reaching your remote Prometheus instance, and only then retire node_exporter.\n\nMigrating one server at a time minimizes risk, preserves visibility, and positions your infrastructure for logs, traces, and future telemetry without deploying additional agents.\n\nBefore getting started, it's worth understanding what actually changes.\n\nThis isn't simply replacing one monitoring agent with another.\n\n`node_exporter`\n\nis a server. It listens on a port, typically 9100,and waits for Prometheus to connect and scrape metrics. That means every monitored machine needs an open endpoint, network connectivity from Prometheus, firewall rules, and scrape configurations.\n\nAlloy flips that model around.\n\nInstead of waiting for Prometheus to connect, Alloy collects metrics locally and pushes them to a remote endpoint using Prometheus Remote Write.\n\nOn my stack, that outbound traffic travels through a Cloudflare Tunnel. Nothing reaches into the monitored servers. There are no metrics ports exposed to the LAN, no inbound firewall rules to maintain, and no scrape network that has to remain routable. The user’s metrics are exposed through the secure tunnel, and the monitoring stack has no access to the server.\n\nThat shift is the real migration, you're not replacing a binary, you're changing the direction your telemetry flows. Once you frame it that way, the rest of the migration makes much more sense.\n\nAlloy is configured using River, which is less like a traditional configuration file and more like a telemetry pipeline. Each component performs one task before handing data to the next component. As you begin to put your model together, the configuration becomes surprisingly readable.\n\nThe component that replaces node_exporter is `prometheus.exporter.unix`\n\n.\n\nUnder the hood it's using the same collector code as `node_exporter`\n\n, so the metrics themselves remain familiar.\n\nA minimal configuration looks like this:\n\n```\n// Collect host metrics.\nprometheus.exporter.unix \"host\" {\n}\n\n// Scrape those locally collected metrics.\nprometheus.scrape \"host\" {\n  targets    = prometheus.exporter.unix.host.targets\n  forward_to = [prometheus.remote_write.default.receiver]\n}\n\n// Push metrics to Prometheus Remote Write.\nprometheus.remote_write \"default\" {\n  endpoint {\n    url = \"https://metrics.example.internal/api/v1/write\"\n\n    # Production deployments typically authenticate here using\n    # Basic Auth, bearer tokens, or mTLS.\n  }\n}\n```\n\nRead from top to bottom, it tells a story:\n\nInstead of maintaining a YAML file full of scrape targets, you're wiring components together into a pipeline. That same pattern extends naturally to logs, traces, profiling, and nearly every other telemetry signal Alloy supports.\n\nOne thing to remember during migration: the metrics themselves stay the same, but some labels—particularly the `instance`\n\nlabel—may change depending on how Alloy identifies the host.\n\nThat's expected, and it's important when you start verifying your migration.\n\nThe temptation is to deploy Alloy everywhere and immediately disable node_exporter, but the best way is to do it slowly. The safest migration pattern is a canary. Pick a non-critical server, install Alloy beside node_exporter, and let both run simultaneously while you verify that the new telemetry path is working correctly. The resource overhead is negligible, but the confidence you gain is enormous.\n\nRunning both agents briefly means you always have a known-good monitoring path while validating the new one. Only after you've confirmed that Alloy is producing fresh, accurate metrics should you retire `node_exporter`\n\n. Once you've done that successfully a few times, batching servers becomes much less stressful.\n\nChoose a stable server where a few minutes of metric oddities wouldn't be catastrophic, but would still be noticeable.\n\nInstall Grafana Alloy using your distribution's package manager.\n\nMost Linux distributions install Alloy as a systemd service automatically, with the primary configuration file located at:\n\n`/etc/alloy/config.alloy`\n\nEnable the service, but leave node_exporter exactly as it is.\n\n`sudo systemctl enable alloy`\n\n`sudo systemctl start alloy`\n\nAt this point nothing has changed from Prometheus' perspective, you’re just running two collector concurrently.\n\nCreate your River configuration and point the `remote_write`\n\nendpoint at your Prometheus receiver.\n\nStart Alloy and verify the service is healthy.\n\n```\nsudo systemctl status alloy --no-pager\n```\n\nThe `--no-pager`\n\noption simply prints the status and returns you to the shell instead of opening the interactive pager, making it much friendlier for automation and scripting. Alloy also exposes a built-in UI on port 12345. Opening it gives you a live view of every component in your telemetry pipeline.\n\nIf `prometheus.exporter.unix`\n\nand `prometheus.remote_write`\n\nare healthy, your data is flowing.\n\nIf `remote_write`\n\nis unhealthy, the problem is usually one of three things:\n\n• Incorrect endpoint URL\n\n• Network connectivity\n\n• Authentication\n\n...in roughly that order.\n\nNow both `node_exporter`\n\nand Alloy are reporting metrics.\n\nFirst, verify Alloy locally.\n\n```\ncurl http://localhost:12345/metrics | head\n```\n\nor check for a specific metric:\n\n```\ncurl http://localhost:12345/metrics | grep node_memory_MemAvailable_bytes\n```\n\nThen open Prometheus or Grafana and confirm the new series is arriving. Since Alloy pushes via remote_write instead of being scraped, the standard up metric won't reflect this host anymore — up is generated per scrape target, and there's no scrape target here. Checking it after cutover will look like the host vanished, even though everything is working.\n\nInstead, check for freshness directly:\n\n```\ntime() - timestamp(node_uname_info{instance=\"canary-host\"})\n```\n\nA small, stable number means data is arriving on schedule. A growing number means the host has stopped pushing.\n\nThen compare a real metric such as available memory:\n\n```\nnode_memory_MemAvailable_bytes\n```\n\nDon't expect the labels to match perfectly — the important part is that the values agree and the new series continues updating.\n\nIf your dashboards, recording rules, or alerting rules explicitly reference labels such as job=\"node_exporter\" — or reference up for these hosts — now is a good time to identify them before removing the old exporter.\n\nOnce you're confident Alloy is working correctly, stop the old exporter.\n\n```\nsudo systemctl stop node_exporter\nsudo systemctl disable node_exporter\n```\n\n`stop`\n\nends the running process.\n\ndisable prevents it from quietly returning after the next reboot.\n\nThen remove the server's scrape job from Prometheus and reload the configuration.\n\n```\ncurl -X POST http://localhost:9090/-/reload\n```\n\nReloading avoids interrupting metric collection for every other host while updating the configuration.\n\nGive the server at least one scrape interval under the new path.\n\nVerify that:\n\n• Metrics remain fresh.\n\n• Alerts continue behaving normally.\n\n• Dashboards still populate.\n\n• No recording rules broke because of label changes.\n\nOnce everything looks healthy, repeat the process on the next server.\n\nAfter a handful of successful migrations, you'll have enough confidence to migrate small batches instead of individual hosts.\n\nThere are a few issues that show up repeatedly during migrations:\n\n• Removing node_exporter before verifying Alloy.\n\n• Forgetting to reload Prometheus after removing scrape targets.\n\n• Alert rules or dashboards still referencing the old job label.\n\n• Firewall rules blocking outbound Remote Write traffic.\n\n• Assuming label changes won't affect existing dashboards.\n\nNone of these are difficult to fix, but catching them during a canary migration is far less stressful than discovering them after migrating twenty servers.\n\nThe obvious benefit is consolidation.\n\nInstead of deploying separate agents for metrics, logs, and traces, Alloy provides a single telemetry pipeline that grows with your infrastructure.\n\nIf you need logs, you can add a Loki component, for OTLP traces, add another component. The overall architecture doesn't change.\n\nThe less obvious benefit is security. Once every monitored machine pushes telemetry outward, you can close your metrics ports entirely. There's no longer a listening endpoint on every server, no scrape network to maintain, and no inbound firewall rule whose only purpose is monitoring.\n\nFor anyone managing customer infrastructure—or simply trying to reduce attack surface—that's arguably the biggest improvement Alloy brings.\n\nFive years ago, exposing port 9100 across an internal network wasn't unusual, but today we're steadily moving toward zero-trust networking, outbound-only connectivity, and centralized telemetry pipelines.\n\nGrafana Alloy isn't compelling because it replaces node_exporter. It's compelling because it aligns your monitoring architecture with where modern infrastructure is already heading, and strengthens your security posture by removing exposed enpoints.\n\nDone this way, there should never be a moment when a server isn't being watched.", "url": "https://wpnews.pro/news/migrating-from-node-exporter-to-grafana-alloy-one-server-at-a-time", "canonical_source": "https://dev.to/irinobservability/migrating-from-nodeexporter-to-grafana-alloy-one-server-at-a-time-58an", "published_at": "2026-07-08 12:46:19+00:00", "updated_at": "2026-07-08 12:58:53.843740+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "mlops"], "entities": ["Grafana Alloy", "node_exporter", "Prometheus", "Cloudflare Tunnel", "Irin"], "alternates": {"html": "https://wpnews.pro/news/migrating-from-node-exporter-to-grafana-alloy-one-server-at-a-time", "markdown": "https://wpnews.pro/news/migrating-from-node-exporter-to-grafana-alloy-one-server-at-a-time.md", "text": "https://wpnews.pro/news/migrating-from-node-exporter-to-grafana-alloy-one-server-at-a-time.txt", "jsonld": "https://wpnews.pro/news/migrating-from-node-exporter-to-grafana-alloy-one-server-at-a-time.jsonld"}}