The Hidden Cost of Reasoning Tokens #
The biggest blind spot right now is "reasoning tokens." Many modern models generate an internal chain of thought to solve complex problems. You are billed for every single one of these tokens, even if the API doesn't return them to you or if they are hidden in a separate field.
I saw this play out during a model evaluation where a "cheaper" candidate model looked 30% less expensive on paper. However, a single real-world request revealed the truth:
Prompt tokens: 274Completion tokens: 132Actual answer tokens: 9Reasoning tokens: 123
The provider billed for all 132 completion tokens. Because only 9 were actually useful, the effective price per useful token jumped by nearly 15x. The "cheaper" model was actually an order of magnitude more expensive for the actual result.
The "Empty Response" Budget Trap #
Reasoning tokens don't just cost money; they eat your output limit. If you set a tight max_tokens
limit, the model might use that entire budget on internal reasoning and leave zero room for the actual answer. You'll get an empty response and assume the model is "dumb" or incapable, when in reality, it just ran out of room to speak.
How to Actually Price a Model #
To get a real-world cost analysis, you need to move past the marketing tables. Here is a practical tutorial on how to verify actual costs:
-
Send a representative set of real-world requests to the model.
-
Inspect the response metadata specifically for
completion_tokens_details.reasoning_tokens
.
- Calculate your cost based on useful output tokens, not total completion tokens.
If a provider doesn't expose the reasoning token count, you effectively cannot price the model accurately.
Accuracy vs. Reliability #
Another mistake is assuming models exist on a linear scale from "dumb" to "smart." Two models can have nearly identical accuracy scores but fail in completely different ways.
In tool-calling tasks, a cheap model might have 93% accuracy compared to a frontier model's 96%. That 3% gap seems negligible until you look at "trap" requests (where the model should decline to use a tool). A frontier model might handle 100% of traps, while the cheap model fails 10% of the time. In a production deployment, that 10% failure rate—where the AI confidently picks the wrong tool—is far more expensive in terms of engineering overhead and user frustration than the few cents saved on tokens.
Next SigNoz LLM Observability: A Deployment Guide →