PLATAFORMA: Dev.to
If you're running any observability stack, you've probably got Prometheus and Grafana at the core. You've got your CPU usage, your memory, your network I/O... but what about your single biggest, most volatile line item? Your LLM bill.
Getting that data into your existing dashboards is a pain. Most solutions want you to route all your traffic through their proxy, adding a point of failure and latency you don't need.
It doesn't have to be that complicated.
I'm a big believer in async, out-of-band monitoring. Your app should fire and forget logging events, not wait for a monitoring service to respond. This is why when I built LLMeter, I made sure it could plug into the tools you already use.
Here's how you can pipe your LLM costs into Prometheus without a proxy.
LLMeter exposes a /metrics
endpoint that Prometheus can scrape, just like it does for your other services. All you need to do is add a new job to your prometheus.yml
:
scrape_configs:
- job_name: 'llmeter'
scrape_interval: 15s
static_configs:
- targets: ['your-llmeter-instance.local:3000']
That's it. Now you can build Grafana dashboards that show your LLM costs right next to your other metrics. You can see how a spike in traffic to a certain endpoint correlates with a spike in your Anthropic bill. You can finally see the real cost of that new AI feature you just shipped.
This approach gives you:
Stop flying blind. Your LLM bill is part of your infrastructure cost. It's time to start treating it that way.