I’ve been trying to understand how well relatively small, domain-specific LLMs perform in production.
A common claim I keep hearing is that a 7B–14B model, when combined with techniques such as SFT, LoRA, distillation, and RAG, can achieve performance close to much larger models for specialized domains like finance or legal.
Interestingly, when I ask frontier LLMs (ChatGPT, Claude, Gemini, etc.), they all give essentially the same answer: that this approach is already widely adopted in industry and works well in practice.
However, I’ve also spoken with a few engineers who have actually deployed LLMs or run local models extensively. Their opinions seem much more cautious. Some of them mentioned that even models around the size of Qwen3.6-27B still struggle with reasoning, consistency, and reliability in real-world applications.
This left me wondering where the reality actually lies.
For those of you who have hands-on experience building or deploying these systems: I’m not looking for benchmark results as much as practical experience and lessons learned from real deployments.
I’d really appreciate hearing your experiences or any case studies you can share.