Most LLM bills I look at waste money in the same four places. If you run a product on OpenAI, Anthropic or Gemini, check these first.
A long system prompt, tool definitions or retrieved docs that are identical on every request. Cached input is billed at roughly a tenth of the normal input price on Anthropic, and both OpenAI and Gemini discount it too. If the fixed part of your prompt is large and you aren't caching it, this is usually the biggest single saving.
On my own 7,407 coding-agent calls, caching was the difference between $6,301 and $566 at list prices. Write-up here.
Classification, short rewrites, routing and "is this valid?" checks rarely need the top model. Send them to a small model and escalate only when it isn't confident.
Cost grows with conversation length times the number of turns. Summarise or trim older turns, and cache the stable part.
Repeat questions, the same URL, the same image. Hash the input and serve a saved answer.
I'm building up case studies. If you send me token counts per request for a few days (no prompt text, no user data), I'll send back where the money goes and the three biggest savings with a monthly dollar figure for each, within about 48 hours. If you want the changes implemented afterwards, that's a separate paid job, priced as a share of what you save.
Comment below or email krishalivaibhav@gmail.com with roughly what you spend a month and which models you use.
Drafted with help from Claude.