For a while we leaned on a specialist decision model to make quick judgement calls inside our products. Recently I swapped it for a small general model, and in my experience the general one is simply the better judge.
- A specialist System One style model is fast and well calibrated, but it matches the narrow pattern it was trained on
- A small general model, Claude Haiku 5.5 at low effort, reads the whole context and handles odd cases that a specialist can get confidently wrong
- New rules can be written in plain words, with no retraining or recalibration
- Fewer outside dependencies in a critical path means fewer ways for a safety check to quietly stop working
- The trade-offs are real: it is slower, and its confidence scores are more extreme, so thresholds need rechecking
- We have not measured accuracy or cost, so this is my experience and not a benchmark
The idea in one sentence #
If your product needs a fast yes, no or ranking from a model, and not a paragraph of text, you do not necessarily need a specialist for it. A small general model can often do that job better.
What a specialist is good at #
Specialist decision models earn their place. They are quick, they give you probabilities you can calibrate, and they do one narrow thing without any fuss. For a long time that was exactly what I wanted, and I want to be fair to the one we used: it did its job well for what it is.
The limit of a specialist is that it matches the shape of what it was trained on. That works until the case in front of it is slightly odd, and then it can be confidently wrong.
Why a small general model judges better #
I moved to Claude Haiku 5.5 at low effort, and the difference came down to a few things. It reads the whole context, so it notices the odd detail and weighs it. When I need a new rule, I can write it in a sentence instead of retraining or recalibrating anything. And it removed one outside dependency from a path where a silent failure matters. Every extra vendor in a critical path is another place a check can quietly stop checking.
Anthropic launches Claude Haiku 5.5 with prices cut up to 90 percent Anthropic cut Haiku's API price by up to 90% below 100,000 tokens and gave it a one-million-token context window. The move matches OpenAI's GPT-6 Luna pricing on its cheapest tier, intensifying competition for high-volume, cost-sensitive developers. - claude haiku 5.5 pricing and cost comparison - anthropic launches haiku 5.5 with lower prices
My view is that general models have now caught up with narrow specialists at this kind of work, and in my experience they have passed them. That is a judgement from using it, not a measured result, and I would not ask you to take it on faith.
The trade-offs #
It is slower. A call takes several seconds where the specialist took about one, which is fine for checks that run before something goes out and not fine for anything a person is waiting on.
It is also more emphatic. Its confidence scores are more extreme than the carefully calibrated ones from a specialist, so any threshold you tuned before needs rechecking against real use before you trust it.
And I have not measured any of this. I have no accuracy percentage or cost figure to quote, and I will not invent one. If the comparison matters to you, run both on your own questions and count the results.
What I would take from it #
Keep the questions you ask a model simple and typed, so the model behind them can be swapped without touching anything else. Make sure your checks fail in a way you notice. And do not assume a specialist beats a general model just because it was built for the job. Test it. I expected to be settling for something simpler, and instead I got a better judge.
This article is posted in AI News, check it out for more related stories.
Join the discussion #
Open in the community → Almost there. Sign in and your reply posts straight away.