Building Production-Ready LLM Applications: What Usually Breaks After the Prototype?

Many LLM and Generative AI projects perform well during the prototype stage but become much harder to manage once they move toward production.

Some of the challenges I frequently see include:

I’m interested in learning how others in the Hugging Face community approach this transition.

For production LLM or conversational AI applications: What has been your biggest challenge moving from a working prototype to a reliable production system?

Also, which approach has worked best for you: fine-tuning, RAG, prompt optimization, smaller specialized models, or a combination of these?

Would be great to hear practical experiences, architectures, evaluation methods, or Hugging Face tools/models that have worked well.