Stop Paying for Invisible Retries: How to Measure What an AI Agent Really Costs

A developer has released SpendGraph, an SDK and stage-based framework that prices every retry attempt of an AI agent call rather than only the successful one, arguing that success-only cost tracking understates real spend. In an illustrative two-step job, the dashboard showed $0.034 while the provider billed $0.066, leaving 48% of the spend invisible. The library records each attempt as its own priced row, emits stage:failed events, and returns pricing as a promise so billing lookups stay off the user-facing response path.

Imagine you hire a researcher to answer a question. They hand you a neat one-page report and say, "That took me 10 minutes." What they don't tell you: That is how most AI agents report their cost today. An agent step rarely succeeds on the first call. The model answers in prose when you asked for JSON, so the step asks again. A reviewer rejects the answer, so the review runs again. Your provider bills every one of those calls. Most tracking only keeps the call whose answer you ended up using. The failed attempts are spent, billed, and missing from your own numbers. Here is one small job, two steps, with every call on the receipt: step attempt result cost extract 1 failed: prose instead of JSON $0.005 extract 2 ok $0.006 review 1 failed: answer rejected $0.027 review 2 ok $0.028 ────── what you paid $0.066 what success-only shows $0.034 The dashboard says $0.034. The provider charges $0.066. In this run, 48% of the spend is invisible, and nothing about the answer tells you it happened. The numbers are illustrative, but the shape is not. Any retry that happens below the layer that records cost is a retry you pay for twice and see once. spendgraph https://spendgraph.locusgraph.com/docs/stage/overview makes one design choice about this: the retry wraps the whole recorded call, not just the model request inside it. retry inside the call retry around the call ┌ recorded call ──────────┐ ┌ recorded call ┐ attempt 1 $0.005 │ model ✗ │ └───────────────┘ │ model ✓ │ ┌ recorded call ┐ attempt 2 $0.006 └─────────────────────────┘ └───────────────┘ one row: $0.006 two rows: $0.011 On the left, the retry is smaller and cheaper to build, and your accounting is wrong. On the right, every attempt is its own priced record, so the total adds up. Two other choices follow from it: A stage is one prompt, one schema, and one priced reply. You get the answer as soon as it is ready, and the price when it resolves: js import { runStage } from "@spendgraph/stage"; const outcome = await runStage prompts, llm, "classify-email", SCHEMA, { question: "Is this email spam or clean?", }, { attempts: 3, emit: event = { if event.type === "stage:failed" { console.warn attempt ${event.attempt} failed: ${event.error.message} ; } }, } ; console.log outcome.data ; const micros = await outcome.pricing; A few things worth knowing: prompts is your prompt store and llm is your model client. The pricing is a promise, and the answer does not wait for it. A billing lookup has no business on the path of a user waiting for a reply. undefined when no price could be found. That is deliberate: an unknown price is not the same as a zero one, and it should not look like one. stage:failed as it happens, and shows up as its own priced record, so the extra spend is on the bill and not only in your logs. If you only want spend tracking on the client you already use, without stages, the SDK wraps it in one line: python import OpenAI from "openai"; import { SpendGraph } from "@spendgraph/sdk"; const meter = new SpendGraph { apiKey: process.env.SPENDGRAPH API KEY, baseUrl: process.env.SPENDGRAPH BASE URL, } ; const openai = meter.wrap new OpenAI ; Use openai exactly as before. Each call's usage is read off the reply and reported in the background, and the meter never throws into your code. If a retry happens where your cost tracking cannot see it, your cost tracking is wrong, and wrong in the direction that makes things look cheaper than they are. Count every attempt. Look at which steps fail most. The prompt that fails half the time is usually a cheaper fix than a cheaper model.