Monitor LLM Costs with Prometheus & Grafana (Without a Proxy)

LLMeter, an open-source tool, enables developers to monitor LLM costs in Prometheus and Grafana without routing traffic through a proxy. By exposing a /metrics endpoint, it allows Prometheus to scrape cost data directly, integrating LLM spending into existing observability dashboards. The tool emphasizes async, out-of-band monitoring to avoid adding latency or points of failure.

PLATAFORMA: Dev.to If you're running any observability stack, you've probably got Prometheus and Grafana at the core. You've got your CPU usage, your memory, your network I/O... but what about your single biggest, most volatile line item? Your LLM bill. Getting that data into your existing dashboards is a pain. Most solutions want you to route all your traffic through their proxy, adding a point of failure and latency you don't need. It doesn't have to be that complicated. I'm a big believer in async, out-of-band monitoring. Your app should fire and forget logging events, not wait for a monitoring service to respond. This is why when I built LLMeter, I made sure it could plug into the tools you already use. Here's how you can pipe your LLM costs into Prometheus without a proxy. LLMeter exposes a /metrics endpoint that Prometheus can scrape, just like it does for your other services. All you need to do is add a new job to your prometheus.yml : scrape configs: - job name: 'llmeter' scrape interval: 15s static configs: - targets: 'your-llmeter-instance.local:3000' That's it. Now you can build Grafana dashboards that show your LLM costs right next to your other metrics. You can see how a spike in traffic to a certain endpoint correlates with a spike in your Anthropic bill. You can finally see the real cost of that new AI feature you just shipped. This approach gives you: Stop flying blind. Your LLM bill is part of your infrastructure cost. It's time to start treating it that way.