There is a lot of discussion around whether production LLM applications should rely on RAG, fine-tuning, prompt engineering, or increasingly capable base models.
But the answer becomes less obvious when dealing with real business data.
For example: RAG can provide current and private information but introduces retrieval quality, chunking, ranking, and latency challenges.
Fine-tuning can improve specialized behavior and domain performance, but maintaining updated knowledge can become difficult.
Prompt engineering is faster to implement but may become fragile as applications grow.
And increasingly, smaller specialized models can sometimes perform surprisingly well at a fraction of the inference cost.
For people building production systems: Where do you currently draw the line between RAG and fine-tuning?
I’m especially interested in experiences involving:
What has actually worked for you beyond the prototype stage?