Imagine you hire a researcher to answer a question.
They hand you a neat one-page report and say, "That took me 10 minutes."
What they don't tell you:
That is how most AI agents report their cost today.
An agent step rarely succeeds on the first call. The model answers in prose when you asked for JSON, so the step asks again. A reviewer rejects the answer, so the review runs again.
Your provider bills every one of those calls. Most tracking only keeps the call whose answer you ended up using. The failed attempts are spent, billed, and missing from your own numbers.
Here is one small job, two steps, with every call on the receipt:
step attempt result cost
extract 1 failed: prose instead of JSON $0.005
extract 2 ok $0.006
review 1 failed: answer rejected $0.027
review 2 ok $0.028
──────
what you paid $0.066
what success-only shows $0.034
The dashboard says $0.034. The provider charges $0.066. In this run, 48% of the spend is invisible, and nothing about the answer tells you it happened.
The numbers are illustrative, but the shape is not. Any retry that happens below the layer that records cost is a retry you pay for twice and see once.
spendgraph makes one design choice about this: the retry wraps the whole recorded call, not just the model request inside it.
retry inside the call retry around the call
┌ recorded call ──────────┐ ┌ recorded call ┐ attempt 1 $0.005
│ model ✗ │ └───────────────┘
│ model ✓ │ ┌ recorded call ┐ attempt 2 $0.006
└─────────────────────────┘ └───────────────┘
one row: $0.006 two rows: $0.011
On the left, the retry is smaller and cheaper to build, and your accounting is wrong. On the right, every attempt is its own priced record, so the total adds up.
Two other choices follow from it:
A stage is one prompt, one schema, and one priced reply. You get the answer as soon as it is ready, and the price when it resolves:
import { runStage } from "@spendgraph/stage";
const outcome = await runStage(prompts, llm, "classify-email", SCHEMA, {
question: "Is this email spam or clean?",
}, {
attempts: 3,
emit: (event) => {
if (event.type === "stage:failed") {
console.warn(`attempt ${event.attempt} failed: ${event.error.message}`);
}
},
});
console.log(outcome.data);
const micros = await outcome.pricing;
A few things worth knowing:
prompts is your prompt store and llm is your model client. The pricing is a promise, and the answer does not wait for it. A billing lookup has no business on the path of a user waiting for a reply.undefined when no price could be found. That is deliberate: an unknown price is not the same as a zero one, and it should not look like one.stage:failed as it happens, and shows up as its own priced record, so the extra spend is on the bill and not only in your logs.
If you only want spend tracking on the client you already use, without stages, the SDK wraps it in one line:
import OpenAI from "openai";
import { SpendGraph } from "@spendgraph/sdk";
const meter = new SpendGraph({
apiKey: process.env.SPENDGRAPH_API_KEY,
baseUrl: process.env.SPENDGRAPH_BASE_URL,
});
const openai = meter.wrap(new OpenAI());
Use openai exactly as before. Each call's usage is read off the reply and reported in the background, and the meter never throws into your code.
If a retry happens where your cost tracking cannot see it, your cost tracking is wrong, and wrong in the direction that makes things look cheaper than they are.
Count every attempt. Look at which steps fail most. The prompt that fails half the time is usually a cheaper fix than a cheaper model.