21 Days of LLMs, Day 3: Prompt Engineering That Actually Works in Production

A senior AI engineer shares production-tested prompt engineering patterns for LLMs, including a five-part skeleton (role, context, task, format, constraints), few-shot examples over verbose instructions, and selective use of chain-of-thought reasoning to avoid unnecessary costs. The post emphasizes that prompts must be reliable at scale, not just in a playground, and provides concrete templates for fintech data analysis and customer feedback classification.

Forget “be creative” and “think step by step.” Here are the patterns that hold up when real users hit your app. Most prompt engineering advice on the internet is useless for production. “Be specific.” “Give context.” “Use examples.” That’s not engineering. That’s common sense wrapped in a blog post. When I started building LLM-powered features for real users, I learned fast that the gap between a prompt that works in a playground and one that works at scale is enormous. A prompt that gives you a great answer once means nothing. A prompt that gives you a reliable answer ten thousand times is everything. That’s what today’s post is about. Every production prompt I write follows the same skeleton. Role, context, task, format, constraints. In that order. Not because it’s clever. Because it works consistently. system prompt = """You are a senior data analyst at a fintech company.CONTEXT:You have access to transaction data for the past 90 days.Users ask questions in plain English about their spending.TASK:Analyze the user's question and respond with a clear,specific answer based on the data provided.FORMAT:- Start with a one-sentence summary- Follow with supporting numbers- End with one actionable recommendationCONSTRAINTS:- Never invent data that isn't provided- If the data is insufficient, say so explicitly- Keep responses under 150 words""" This isn’t theory. This is the exact pattern I use every time. The role tells the model who it is. The context gives it the world it operates in. The task tells it what to do. The format tells it how to structure the output. The constraints set the guardrails. Remove any one of these and your outputs start drifting. Here’s something I learned the hard way. You can write a 500-word instruction explaining how to classify customer feedback. Or you can show three examples. The examples win almost every time. few shot prompt = """Classify the following customer message into exactly one category: billing, technical, feature request, praise.Examples:Message: "I was charged twice this month"Category: billingMessage: "The app crashes when I upload a CSV"Category: technicalMessage: "Can you add dark mode?"Category: feature requestNow classify this:Message: "{user message}"Category:""" Research backs this up. Three to five high-quality examples consistently outperform zero-shot prompting for classification, extraction, and formatting tasks. The key word is high-quality. Three perfect examples beat ten sloppy ones. Pick examples that cover your edge cases, not just the happy path. Chain-of-thought prompting means asking the model to show its reasoning before giving an answer. Adding “Let’s think through this step by step” to a math problem can improve accuracy by 40% or more. But here’s what nobody tells you. Chain of thought is expensive. Every reasoning token the model generates counts toward your output bill. And output tokens cost 3x to 5x more than input tokens. For a simple classification task, CoT is overkill. You’re paying for the model to “think” about something it can answer in one word. Use CoT for multi-step reasoning, math, logic, and complex analysis. Skip it for classification, extraction, reformatting, and simple Q&A. That one rule saved me roughly $400 per month on a customer support bot. Six months ago I was demoing a document summarization tool to my team. The prompt said: “Summarize the following document. Be thorough but concise. Include all important details but keep it short.” Read that again. I was telling the model to be thorough AND concise. To include everything AND keep it short. The model did its best, but every summary was a different length. Some were 50 words. Some were 400. The output was completely unpredictable. The fix was embarrassingly simple. I replaced the vague instruction with: “Summarize in exactly 3 bullet points, each under 20 words.” Consistent output, every single time. Vague instructions produce vague results. Specific constraints produce specific results. That’s the entire lesson. Before shipping any prompt to production, I run through four questions. Does it have explicit output format instructions? Does it include at least two examples? Does it have constraints that prevent the model from improvising? Have I tested it with at least 20 different inputs, including adversarial ones? If any answer is no, the prompt isn’t ready. Ship it anyway and you’ll spend the next week firefighting weird edge cases that a 10-minute review would have caught. Tomorrow in Day 4, we’ll cover temperature, top-p, and the other parameters that silently control your model’s behavior. Most developers never touch them. That’s a mistake. What’s the worst prompt failure you’ve experienced? The more embarrassing, the better. Comments are open. 🔥 This is Day 3 of my “21 Days of Building with LLMs” series. If you’re learning along with me, follow so you don’t miss the next one. 👉 Follow me here for daily posts on AI, Python, Data Engineering, and SQL. Missed a day? Start from Day 1 → link — Eswar 21 Days of LLMs, Day 3: Prompt Engineering That Actually Works in Production https://blog.stackademic.com/21-days-of-llms-day-3-prompt-engineering-that-actually-works-in-production-631f2ecb03db was originally published in Stackademic https://blog.stackademic.com on Medium, where people are continuing the conversation by highlighting and responding to this story.