Member-only story
For a while I assumed the model choice was the big lever on our API bill. Switch to something cheaper, save money — simple. Then I actually sat down and ran the numbers on our prompts themselves, and the model was barely the story. The prompt was. Here’s the math, worked out on a realistic example, because the gap surprised me enough that I wanted to see it in black and white.
The formula is dead simple. Cost per request is just (input tokens × input price) + (output tokens × output price)
. The part people forget: output tokens usually cost about 5x more per token than input tokens. And cached tokens cost roughly a tenth of the normal input price on a hit. Those two facts alone explain most of what follows.
I’m using Claude Sonnet 5 pricing for this — $3 per million input tokens, $15 per million output — but the method transfers to whatever model you’re actually running. Token counts below are ballparked at about 1.3 tokens per English word, which is close enough for illustration; for your real numbers, run your actual prompts through the model’s token counter instead of eyeballing it.
Exhibit A: the prompt nobody’s touched since launch. This is the classic first-draft system prompt — the one that grew by accretion, a sentence patched in every time…