# Taking an LLM app from prototype to production: an ai engineer interview answer > Source: > Published: 2026-10-27 08:04:41+00:00 # Taking an LLM app from prototype to production: an ai engineer interview answer Here is the direct answer for an ai engineer interview: the interviewer is not asking how to make a demo prettier. The question is how you would put a working chatbot in front of real users, and a strong answer starts from the failure modes a demo hides. Open with why the prototype is not most of the work. Moving generative AI products from proof of concept to production has proven hard, largely because teams treat them as extensions of traditional transactional or analytical systems. That habit hides fresh headaches: invented content, data access without clear bounds, and outputs that vary from run to run. Then describe direct prompting, the baseline to move away from. Users type straight to an off-the-shelf model with no intermediate steps, and it has three weaknesses. First, stale knowledge: the model only knows its training data, so it knows nothing after training and nothing outside that set. Second, manipulation: it can be tricked by malicious prompts into revealing confidential information. Third, confident errors: it tends to sound confident when its knowledge is weak and to invent plausible but nonsensical answers. A useful image is a junior researcher who is articulate and well-read, but overconfident and unfamiliar with your details. What follows is about equipping that researcher with the right documents, reviewing the work, and blocking statements that must not be made. That is what evals, retrieval, guardrails and continuous evaluation after launch are for. Evals come before any architectural choice. Ordinary software gets checked with tests, yet an LLM setup is non-deterministic: the same input can yield different outputs. Evals are the replacement: evaluations of model responses for a specific task, usually across a range of scenarios rather than a single output. A scorer gives each output a numerical score on metrics such as relevance, coherence, factuality, or semantic similarity to an expected answer. Who judges matters, and there are three options with different trade-offs. Self-evaluation is risky, because a flawed self-assessment can reinforce its own errors. Using a different model as judge helps, because two models are unlikely to share the same errors. Human evaluation checks tone and intent by hand, but it is hard to scale. Name all three, and explain why self-evaluation alone is not enough. Close with order: fine-tuning comes last and only when the evals call for it. Failure modes first, measurement second, retrieval and guardrails third, fine-tuning last. That order shows production thinking, not model selection followed by damage control. [Next LangGraph quickstart for production agents: what is ai engineering in practice →](https://promptcube3.com/en/threads/10132/) ## All Replies （0） Want a live back-and-forth? [Join the global AI chat room](https://promptcube3.com/en/chat/) — login to talk. No replies yet — be the first!