Building Production Prompts That Actually Work (And Why Most Fail)

A developer argues that most LLM-based features fail because prompts are treated like documentation rather than precise instructions. The post provides concrete examples showing how adding constraints, structure, and explicit rules transforms vague prompts into reliable production components. The key insight is that shipping an LLM feature means shipping a prompt, which should be version-controlled and tested like code.

So you've got an LLM API key and grand plans. Then you ship it and... users get garbage. Sound familiar? The problem isn't the model. It's that we treat prompting like writing documentation—verbose, formal, generic. But prompts are more like instructions to a coworker. Be specific. Be weird. Get the tone right. Here's what actually works. Bad prompt: Summarize this text. That could mean 3 sentences or 30. Could be bullet points or prose. The model has to guess. Better: Summarize this in 2-3 sentences. Use simple English. Answer: what happened and why it matters? Even better for production : Summarize in exactly 2-3 sentences using words a 12-year-old knows. Format: What happened Why it matters If you can't summarize it, say "unclear." The extra details aren't padding—they're constraints that make outputs predictable. That's what production needs. Let's say you're building an IDE plugin that explains errors. Here's what bad looks like: Explain this error: {error message} Here's production: You are a debugging assistant for junior developers. A developer got this error: {error message} Respond with: 1. What went wrong 1 sentence 2. Why it happened 2-3 sentences 3. How to fix it step-by-step, numbered 4. One code example showing the fix Keep it simple. Avoid jargon. If you're unsure, say "I'm not sure about this one." See the difference? You're not just asking for an explanation—you're defining: This turns an LLM into a reliable component, not a lucky roll. Naive approach: Is this comment appropriate for a work chat? {comment} You'll get yes/no, but why is it inappropriate? What's the threshold? Better: You're a content moderator for a professional Slack workspace. Is this message appropriate to post? Respond with JSON: { "ok": true/false, "reason": "brief explanation if not ok", "confidence": 0.95 } Guidelines: - Profanity is sometimes fine context matters - Sarcasm is okay - Off-topic rants are not okay - Politics/religion are not okay - Venting about work is okay if not naming individuals Message to review: {comment} Now you get structured output, consistency, and explicit rules the model follows. Three reasons: Spend 30 minutes prompt engineering with your actual data , not toy examples. Fix those specific things. Then ship. When you ship an LLM feature, you're not shipping a model—you're shipping a prompt. The model is a blackbox; your prompt is the API surface. Treat it like code: prompts/v1.0.txt , prompts/v1.1.txt test cases.json with expected outputs If you're using Claude or another API, the principles are the same. But check the docs for platform-specific tricks: <thinking tags for reasoning-heavy tasks <examples blocks for in-context learningYou'll be shocked how much better it gets. Production-ready prompts aren't magic. They're just specific, constrained, tested. Like any good code. Ship it.