Member-only story
Part of the ongoing “FinOps for Production AI” series.
Background #
I’ve spent most of this series on the operational surfaces nobody puts on the whiteboard: cost per inference, attribution for agent workloads, the hidden invoice behind probabilistic retrieval. This post is about the gap those pieces keep circling — the gap between the system you designed and the system you pay for.
Every AI system has two architectures. The first is the one in the diagram: the tidy boxes and arrows, the inference gateway, the batch pipeline, the GPU pool with sensible labels. The second is the one on the invoice — and it never looks like the diagram. On the invoice there’s a namespace nobody remembers creating, a GPU node that’s been warm since March, and a batch worker that costs more per month than the feature it supports.
I was reminded how good the open tooling for closing that gap has gotten while reading a recent CNCF projects roundup — OpenCost has quietly become one of the most underused projects in the ecosystem, and it’s the one I want to dig into here. Because the interesting thing about cost attribution isn’t the dashboards. It’s what happens to a team the first time they see their own number.