{"slug": "generative-ai-development-is-20-prompting-and-80-everything-else", "title": "Generative AI Development Is 20% Prompting and 80% Everything Else", "summary": "A developer who built a working generative AI support assistant in a weekend reports that shipping it to production took three more months, with the prompt itself barely changing. The remaining work went into knowledge-base cleanup, building an evaluation set, handling hallucinations, controlling token costs and latency, and integrating the feature with existing systems. The developer estimates prompting accounts for roughly 20% of generative AI development, with engineering, data and operations making up the other 80%.", "body_md": "Last spring I built a working generative AI feature in a weekend: a support assistant that read a customer's question, pulled context from our help docs, and wrote a friendly, accurate answer. The demo went great. Everyone nodded.\n\nShipping it took three more months.\n\nNot because the prompt was wrong. The prompt barely changed. The time went into everything around it: cleaning the knowledge base, building an evaluation set, handling the cases where the model confidently made things up, keeping costs from spiraling, and wiring it into systems that had never heard of an LLM.\n\nI've watched several teams go through the same arc, and I'm convinced of a rough rule: **prompting is about 20% of generative AI development.** The other 80% is engineering, data, and operations work that never shows up in a demo. Here's what that 80% looks like.\n\nA demo is a best-case path. You pick the inputs, you know what the model handles well, and nobody asks a question you didn't anticipate. Production is the opposite. Real users type half sentences, paste entire emails, switch languages, and probe edge cases whether they mean to or not.\n\nThe gap between demo and production isn't a prompt-quality gap. It's a reliability gap. A model that's right 90% of the time looks magical on stage and unacceptable in a customer-facing workflow, where the other 10% becomes support tickets and refunds. Closing that gap is the job, and almost none of it is prompting.\n\nMost useful generative AI features sit on top of your own data: documents, tickets, product catalogs, policies. In our case, the help docs contained contradictory articles and three versions of the refund policy. The model happily retrieved all three and blended them into an answer that matched none of them.\n\nSo the real work looked like this:\n\nRetrieval quality sets the ceiling on answer quality. When the output is bad, look at what went into the context window before you touch the prompt.\n\nThe biggest shift in how I work was treating evaluation as a first-class deliverable. Without an eval set, every change is a vibe check. You tweak the prompt, try five examples, and ship. Two weeks later you find that your improvement broke an entire category of questions, and you can't say when.\n\nA basic setup doesn't need to be fancy:\n\nUse automated checks where you can, such as valid JSON, required fields, or a citation present, and model-based or human grading where you can't. Model graders have their own biases, so spot-check them against human judgment.\n\nThe payoff is confidence. You can swap models or change retrieval strategy and know within minutes whether things improved or worsened.\n\nEvery generative AI system will eventually produce something wrong, off-brand, or unsafe. What matters is what happens next. Plan for failure at several layers:\n\nThe most important design decision is often what the system does when it doesn't know. A feature that admits uncertainty earns more trust than one that's fluent and wrong.\n\nIn a demo, nobody checks the bill. In production, tokens are a line item, and latency is a UX metric. Our first month's API spend was roughly triple our estimate, thanks to bloated context windows, retries on timeouts, and using the biggest model for tasks a smaller one handled just as well.\n\nWhat moved the needle:\n\nA feature that's accurate but too expensive to run is still a failed feature.\n\nTraditional monitoring tells you whether a service is up. It doesn't tell you whether an answer was good. Log prompts, retrieved context, outputs, model versions, latency, and cost for every request, with sensible privacy handling. Add feedback signals like thumbs up and down, escalations, and follow-up questions that suggest what the first answer missed.\n\nThen read a sample of real conversations every week. It's the fastest way to find failure patterns your eval set doesn't cover yet, and each one should flow back into the eval set. That loop of observe, add to evals, improve, redeploy is the real engine of quality.\n\nA model call is one API request. The system around it is authentication, permissions, rate limits, queues, existing databases, and compliance. Small questions can turn into weeks. Which user's data can the model see? Where are conversations stored, and for how long? What happens during a provider outage?\n\nThis is also where teams decide whether to build everything themselves or bring in specialists. If your workflow is unusual, your data is sensitive, or the feature must plug into several internal systems, [custom Generative AI development services](https://wpwebinfotech.com/ai/development/generative-ai/) can save months, because a team that has already built retrieval pipelines, eval harnesses, and guardrails won't have to learn those lessons on your budget. \n\nFor standard use cases, an off-the-shelf tool may be plenty. The honest answer depends on how specific your requirements are and how much of the 80% you want to own.\n\nThe cheapest mistake to fix is the one made on paper. Many failed projects share a root cause: nobody defined success before the first line of code. Before building, answer these:\n\nThis is where [AI consulting](https://www.groovyweb.co/blog/ai-consulting-rates-2026) earns its keep when done well. A good consultant's most valuable output is sometimes a smaller scope, a different approach, or a clear \"not yet.\" Whether you bring in outside help or run it internally, the discipline matters more than who runs it.\n\n[Prompting](https://learn.chatgpt.com/docs/prompting) is the visible part of generative AI development, so it gets the attention. But the teams that ship reliable features are rarely the ones with the cleverest prompts. They treat the model as one component in a system that needs data quality, testing, safety nets, cost control, and monitoring.\n\nThe good news is that this is familiar territory. It's software engineering applied to a probabilistic component. If you've shipped production systems before, you already have most of the skills.\n\n**What does your 80% look like?** Which part surprised your team most: evals, data, cost, or something I didn't cover? Tell me in the comments.\n\nGenerative AI development is the building of AI apps that create content, answer questions, and perform tasks using AI models, data, integrations, and evaluation systems.\n\nPrompting guides the model's behavior, but production-ready AI also requires data preparation, evaluation, guardrails, cost optimization, monitoring, security, and integration with existing systems.\n\nImprove accuracy by cleaning the source data, using robust retrieval methods, building strong eval datasets, validating outputs, testing edge cases, and routing uncertain responses to a human.\n\nThe key issues are unreliable outputs, poor data quality, prompt injection, high API costs, slow response times, privacy issues, and integration with existing business systems.\n\nBusinesses can improve reliability by defining clear, measurable success criteria, testing every change against an evaluation dataset, implementing guardrails and fallbacks, monitoring real conversations, and starting with a very specific use case.", "url": "https://wpnews.pro/news/generative-ai-development-is-20-prompting-and-80-everything-else", "canonical_source": "https://dev.to/wpwebinfotech/generative-ai-development-is-20-prompting-and-80-everything-else-59jb", "published_at": "2026-10-28 09:24:19+00:00", "updated_at": "2026-10-28 09:41:29.672840+00:00", "lang": "en", "topics": ["generative-ai", "ai-products", "mlops", "ai-tools"], "entities": [], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/generative-ai-development-is-20-prompting-and-80-everything-else", "markdown": "https://wpnews.pro/news/generative-ai-development-is-20-prompting-and-80-everything-else.md", "text": "https://wpnews.pro/news/generative-ai-development-is-20-prompting-and-80-everything-else.txt", "jsonld": "https://wpnews.pro/news/generative-ai-development-is-20-prompting-and-80-everything-else.jsonld"}}