Prompt Templates and Prompt Chaining: How to Build AI Workflows That Actually Scale — Part 32

A developer discovered that maintaining 23 scattered prompt strings across a codebase, including three conflicting versions of a legal-summary prompt, is an engineering problem rather than an AI problem. Prompt Templates and Prompt Chaining are presented as patterns that bring structured, maintainable, and composable workflows to AI applications, analogous to SQL prepared statements. The article is Part 32 of a generative AI series and demonstrates Spring AI's PromptTemplate class for reusable prompt structures.

Six months into building AI applications, I made an embarrassing discovery. I was maintaining 23 different prompt strings scattered across my codebase. Learning Generative AI From Scratch: The Complete Roadmap 120+ Articles https://medium.com/codetodeploy/learning-generative-ai-from-scratch-the-complete-roadmap-120-articles-c3bff1f0aa10 Some were hardcoded directly inside service methods. Some were buried in utility classes. A few were duplicated with small variations I couldn’t even explain anymore. One critical prompt — the one that determined how my AI summarized legal documents — existed in three slightly different versions, and I had no idea which one was actually running in production. When a stakeholder asked me to change the tone of the AI’s responses from formal to conversational, I spent four hours tracking down every prompt string in the codebase. That’s when I realized I had an engineering problem, not an AI problem. I was treating prompts like one-off strings — something to write once, make work, and never think about again. But prompts are actually the core logic of an AI application. They deserve the same engineering discipline as any other critical piece of code. Prompt Templates and Prompt Chaining are the two patterns that changed how I build AI applications — from scattered string concatenation to structured, maintainable, composable AI workflows. This is Part 32 of Learning Generative AI From Scratch. In Part 31 we covered memory — how AI applications remember across conversations. Now we’re going deeper into the layer that controls what the AI actually does: the prompts themselves. Memory in AI Applications: How to Make AI Remember Beyond a Single Conversation — Part 31 https://sumanthpoola.medium.com/memory-in-ai-applications-how-to-make-ai-remember-beyond-a-single-conversation-part-31-e7c5820d1e10 A Prompt Template is a reusable prompt structure with placeholders for dynamic values. Instead of building prompt strings manually every time: // The problem — scattered, unmaintainable prompt stringspublic String summarizeDocument String document, String audience { String prompt = "Summarize the following document for a " + audience + " audience. Keep it under 200 words.\n\nDocument:\n" + document; return chatClient.prompt .user prompt .call .content ;} You define the structure once and inject values: // The solution — a reusable templateString templateText = """ Summarize the following document for a {audience} audience. Keep it under {maxWords} words. Focus on {focus}. Document: {document} """; Simple idea. Significant impact on maintainability. The analogy that made this click for me: SQL prepared statements. Instead of concatenating SQL strings vulnerable, unreadable, unmaintainable , you define the query structure and inject parameters separately. Prompt templates do the same thing for AI instructions. Spring AI has first-class support for prompt templates through its PromptTemplate class: python import org.springframework.ai.chat.prompt.PromptTemplate;import org.springframework.ai.chat.prompt.Prompt;@Servicepublic class DocumentSummaryService { private final ChatClient chatClient; public DocumentSummaryService ChatClient.Builder builder { this.chatClient = builder.build ; } public String summarize String document, String audience, int maxWords, String focus { PromptTemplate template = new PromptTemplate """ Summarize the following document for a {audience} audience. Keep it under {maxWords} words. Focus specifically on {focus}. Use clear, direct language appropriate for {audience}. Document: {document} """ ; Map<String, Object variables = Map.of "audience", audience, "maxWords", maxWords, "focus", focus, "document", document ; Prompt prompt = template.create variables ; return chatClient.prompt prompt .call .content ; }} Clean, readable, and the template structure is visible in one place. But the real power comes when you store templates externally rather than hardcoding them in Java source files. Storing prompts in your Java source means every prompt change requires a code change, a build, and a deployment. For applications where prompts are actively tuned — which is most of them — this creates unnecessary friction. Spring AI supports loading templates from the classpath: src/main/resources/prompts/summarize.st StringTemplate format : You are a professional document summarizer.Your task is to summarize the document below for a {audience} audience.Requirements:- Maximum {maxWords} words- Focus on {focus}- Tone: {tone}- Include key decisions and action items if presentDocument:{document}Summary: Loading and using it: @Servicepublic class TemplateService { private final ChatClient chatClient; @Value "classpath:prompts/summarize.st" private Resource summarizeTemplate; @Value "classpath:prompts/classify.st" private Resource classifyTemplate; @Value "classpath:prompts/extract.st" private Resource extractTemplate; public String summarize Map<String, Object variables { try { String templateText = summarizeTemplate .getContentAsString StandardCharsets.UTF 8 ; PromptTemplate template = new PromptTemplate templateText ; return chatClient.prompt template.create variables .call .content ; } catch IOException e { throw new TemplateLoadException "Failed to load summarize template", e ; } }} Now prompt changes are file changes — no recompilation required. In some setups, you can even store prompts in a database and update them without any deployment at all. A pattern I use in almost every production application: separate system-level instructions from user-level content. java @Servicepublic class StructuredPromptService { private final ChatClient chatClient; public String chat String userContext, String userMessage { // System template — defines AI behavior, persona, constraints PromptTemplate systemTemplate = new PromptTemplate """ You are a helpful Java backend development assistant. You specialize in: - Spring Boot applications - Microservices architecture - Java best practices - Performance optimization User context: {userContext} Guidelines: - Always provide working code examples - Explain the "why" not just the "what" - Flag potential production issues - Keep responses practical and specific """ ; // User template — structures the actual request PromptTemplate userTemplate = new PromptTemplate """ {message} Please include: 1. A direct answer 2. A code example if applicable 3. One potential gotcha to watch for """ ; String systemPrompt = systemTemplate.render Map.of "userContext", userContext ; String userPrompt = userTemplate.render Map.of "message", userMessage ; return chatClient.prompt .system systemPrompt .user userPrompt .call .content ; }} Why separate them: The system template defines who the AI is and how it behaves — this changes rarely. The user template defines what’s being asked — this changes with every request. Keeping them separate makes each easier to maintain and tune independently. Prompt Chaining is connecting multiple AI calls together — where the output of one prompt becomes the input to the next. The simplest mental model: an assembly line. Station 1 does one thing. Passes its output to Station 2. Station 2 does one thing. Passes to Station 3. Each station is focused, specialized, and testable independently. Without chaining: // One massive prompt trying to do everythingString prompt = """ Read this customer support ticket. Determine if it's urgent. If urgent, draft an empathetic response. Also extract the product name mentioned. Also suggest three possible solutions. Also check if this matches any known bug reports. Ticket: {ticket} """; The problem with “do everything in one prompt”: the model splits its attention across too many tasks. Complex multi-step reasoning in a single prompt produces worse results than breaking the same reasoning into focused sequential steps. With chaining: Step 1: Classify the ticket urgent/normal/low Step 2: Extract product and issue typeStep 3: Search for matching known issues using Step 2 output Step 4: Draft response using Step 1 + Step 2 + Step 3 outputs Each step is focused. Each step’s output is verifiable. If Step 2 extracts the wrong product, you catch it before it propagates through the rest of the chain. Let’s build a real example — a customer support ticket processing pipeline: @Servicepublic class SupportTicketPipeline { private final ChatClient chatClient; public SupportTicketPipeline ChatClient.Builder builder { this.chatClient = builder.build ; } // Main pipeline method — orchestrates all steps public TicketProcessingResult processTicket String ticketContent { // Step 1: Classify urgency UrgencyClassification urgency = classifyUrgency ticketContent ; // Step 2: Extract key information TicketInfo ticketInfo = extractTicketInfo ticketContent ; // Step 3: Find relevant solutions uses Step 2 output List<String suggestedSolutions = findSolutions ticketInfo.productName , ticketInfo.issueType ; // Step 4: Draft response uses all previous outputs String draftResponse = draftResponse ticketContent, urgency, ticketInfo, suggestedSolutions ; // Step 5: Quality check the draft uses Step 4 output String finalResponse = qualityCheck draftResponse, urgency ; return new TicketProcessingResult urgency, ticketInfo, suggestedSolutions, finalResponse ; } // Step 1: Classify urgency private UrgencyClassification classifyUrgency String ticket { PromptTemplate template = new PromptTemplate """ Classify the urgency of this support ticket. Respond with ONLY one of these exact values: CRITICAL - system down, data loss, security breach HIGH - major feature broken, significant business impact NORMAL - feature issue, workaround available LOW - question, minor inconvenience, feature request Ticket: {ticket} Urgency level: """ ; String result = chatClient.prompt template.create Map.of "ticket", ticket .call .content .trim ; return UrgencyClassification.valueOf result ; } // Step 2: Extract structured information private TicketInfo extractTicketInfo String ticket { PromptTemplate template = new PromptTemplate """ Extract information from this support ticket. Respond in this exact JSON format: { "productName": "name of product mentioned or UNKNOWN", "issueType": "one of: authentication, performance, data, integration, other", "customerName": "customer name if mentioned or UNKNOWN", "errorCode": "error code if mentioned or NONE" } Respond with ONLY the JSON. No explanation. Ticket: {ticket} """ ; String jsonResult = chatClient.prompt template.create Map.of "ticket", ticket .call .content ; return parseTicketInfo jsonResult ; } // Step 3: Find solutions based on extracted info private List<String findSolutions String productName, String issueType { PromptTemplate template = new PromptTemplate """ Suggest three practical solutions for this issue. Product: {productName} Issue type: {issueType} Format each solution as a single actionable sentence. Number them 1, 2, 3. Be specific and practical, not generic. """ ; String result = chatClient.prompt template.create Map.of "productName", productName, "issueType", issueType .call .content ; return Arrays.asList result.split "\n" .stream .filter s - s.matches "^ 123 . " .collect Collectors.toList ; } // Step 4: Draft the actual response private String draftResponse String originalTicket, UrgencyClassification urgency, TicketInfo ticketInfo, List<String solutions { PromptTemplate template = new PromptTemplate """ Draft a customer support response for this ticket. Original ticket: {ticket} Context: - Urgency: {urgency} - Issue type: {issueType} - Customer name: {customerName} Suggested solutions to incorporate: {solutions} Requirements: - Open with empathy appropriate to {urgency} urgency - Address the customer by name if known - Present the most likely solution first - Keep it under 150 words - End with next steps - Professional but warm tone """ ; return chatClient.prompt template.create Map.of "ticket", originalTicket, "urgency", urgency.name , "issueType", ticketInfo.issueType , "customerName", ticketInfo.customerName , "solutions", String.join "\n", solutions .call .content ; } // Step 5: Quality check before sending private String qualityCheck String draft, UrgencyClassification urgency { PromptTemplate template = new PromptTemplate """ Review this customer support response draft. Urgency level: {urgency} Check for: 1. Is the tone appropriate for {urgency} urgency? 2. Are there any promises we can't keep? 3. Is any sensitive information accidentally included? 4. Is it under 150 words? If the draft is acceptable, return it unchanged. If it needs fixes, return the corrected version. Return ONLY the response text — no explanation. Draft: {draft} """ ; return chatClient.prompt template.create Map.of "urgency", urgency.name , "draft", draft .call .content ; } // Supporting records public enum UrgencyClassification { CRITICAL, HIGH, NORMAL, LOW } public record TicketInfo String productName, String issueType, String customerName, String errorCode {} public record TicketProcessingResult UrgencyClassification urgency, TicketInfo ticketInfo, List<String suggestedSolutions, String finalResponse {}} Five focused prompts. Each does one thing. Each output is verifiable. The whole pipeline produces better results than any single “do everything” prompt could. Not every chain is linear. Real workflows branch based on intermediate results: @Servicepublic class ConditionalChainService { private final ChatClient chatClient; private final SupportTicketPipeline ticketPipeline; public String handleRequest String userInput { // Step 1: Route — decide which path to take String intent = classifyIntent userInput ; // Step 2: Branch based on intent return switch intent { case "SUPPORT TICKET" - { TicketProcessingResult result = ticketPipeline.processTicket userInput ; yield result.finalResponse ; } case "PRODUCT QUESTION" - handleProductQuestion userInput ; case "BILLING ISSUE" - handleBillingIssue userInput ; default - handleGeneral userInput ; }; } private String classifyIntent String input { PromptTemplate template = new PromptTemplate """ Classify this user input into exactly one category. Categories: SUPPORT TICKET - reporting a bug or technical problem PRODUCT QUESTION - asking how something works BILLING ISSUE - payment or subscription related GENERAL - anything else Respond with ONLY the category name. Input: {input} """ ; return chatClient.prompt template.create Map.of "input", input .call .content .trim ; }} This pattern — classify first, then route to specialized chains — is the foundation of most production AI workflow systems. Some tasks don’t need to run sequentially. When steps are independent, run them in parallel: @Servicepublic class ParallelChainService { private final ChatClient chatClient; public DocumentAnalysisResult analyzeDocument String document throws InterruptedException, ExecutionException { ExecutorService executor = Executors.newFixedThreadPool 3 ; // Run three independent analyses in parallel Future<String summaryFuture = executor.submit - summarize document ; Future<List<String keyPointsFuture = executor.submit - extractKeyPoints document ; Future<String sentimentFuture = executor.submit - analyzeSentiment document ; // Wait for all to complete String summary = summaryFuture.get ; List<String keyPoints = keyPointsFuture.get ; String sentiment = sentimentFuture.get ; executor.shutdown ; // Combine results in a final synthesis step return synthesizeResults summary, keyPoints, sentiment ; } private DocumentAnalysisResult synthesizeResults String summary, List<String keyPoints, String sentiment { PromptTemplate template = new PromptTemplate """ Create a final executive briefing from these analyses. Summary: {summary} Key Points: {keyPoints} Overall Sentiment: {sentiment} Produce a cohesive 3-paragraph briefing that integrates all three analyses naturally. """ ; String briefing = chatClient.prompt template.create Map.of "summary", summary, "keyPoints", String.join ", ", keyPoints , "sentiment", sentiment .call .content ; return new DocumentAnalysisResult summary, keyPoints, sentiment, briefing ; }} Three independent analyses run simultaneously, then a final synthesis step combines them. Total time is the maximum of the three parallel steps, not their sum. Chains fail at individual steps. Handle it gracefully: @Servicepublic class ResilientChainService { private final ChatClient chatClient; public ProcessingResult processWithFallback String input { try { // Attempt the full chain return runFullChain input ; } catch ChainStepException e { log.warn "Chain step {} failed: {}", e.getStep , e.getMessage ; // Fallback: skip failed step, continue with defaults return runFallbackChain input, e.getStep ; } } private ProcessingResult runFallbackChain String input, String failedStep { // If classification failed, default to NORMAL if "classification".equals failedStep { return runChainFromStep input, "extraction", UrgencyClassification.NORMAL ; } // If extraction failed, use minimal defaults if "extraction".equals failedStep { return generateGenericResponse input ; } // If drafting failed, return a safe template response return safeTemplateResponse ; } private ProcessingResult safeTemplateResponse { return new ProcessingResult "Thank you for contacting us. A team member will " + "review your request and respond within 24 hours.", true // flagged for human review ; }} Rule I follow: every chain step should have a defined fallback behavior. A chain that fails completely and returns an error to the user is worse than a chain that degrades gracefully. Once your application goes live, prompts need to evolve. Users give feedback. Edge cases appear. The model gets updated. Without version control for prompts, you lose the ability to roll back a bad change: @Entity@Table name = "prompt templates" public class PromptTemplate { @Id @GeneratedValue strategy = GenerationType.IDENTITY private Long id; private String templateName; private String version; @Column columnDefinition = "TEXT" private String content; private boolean active; private LocalDateTime createdAt; private String createdBy; private String changeNote;}@Repositorypublic interface PromptTemplateRepository extends JpaRepository<PromptTemplate, Long { Optional<PromptTemplate findByTemplateNameAndActive String templateName, boolean active ; List<PromptTemplate findByTemplateNameOrderByCreatedAtDesc String templateName ;}@Servicepublic class VersionedTemplateService { private final PromptTemplateRepository repository; private final ChatClient chatClient; public String renderWithActiveTemplate String templateName, Map<String, Object variables { PromptTemplate template = repository .findByTemplateNameAndActive templateName, true .orElseThrow - new TemplateNotFoundException templateName ; org.springframework.ai.chat.prompt.PromptTemplate springTemplate = new org.springframework.ai.chat.prompt.PromptTemplate template.getContent ; return chatClient.prompt springTemplate.create variables .call .content ; } public void activateVersion String templateName, String version { // Deactivate current active version repository.findByTemplateNameAndActive templateName, true .ifPresent t - { t.setActive false ; repository.save t ; } ; // Activate new version repository.findByTemplateNameAndVersion templateName, version .ifPresent t - { t.setActive true ; repository.save t ; } ; }} Now prompt changes are database changes — tracked, auditable, and reversible without a deployment. When I cleaned up those 23 scattered prompt strings and refactored them into versioned, externalized templates feeding into proper chains, something unexpected happened. The AI’s output quality improved — not because I changed the instructions, just because I made them consistent. What I realized: prompt inconsistency is one of the sneakiest sources of AI application quality problems. When the same logical operation runs on slightly different prompt strings depending on which code path executed, you get subtly different outputs that are very hard to debug. You end up chasing what feels like model inconsistency when it’s actually prompt inconsistency. Templates force consistency. Chains force single-responsibility. And both together make AI applications significantly easier to test, debug, and improve — the same way software engineering principles improve any complex codebase. The lesson: AI applications need software engineering. Not instead of AI thinking, but in addition to it. Build a three-step content pipeline: Test with the same input paragraph going through all three steps. Observe how each step builds on the previous one’s output. Then try breaking it by giving Step 2 a wrong classification — see how the error propagates through the chain. That propagation behavior is the most important thing to understand about chaining — errors compound downstream, which is why Step 1 accuracy matters so much. Prompt Templates bring software engineering discipline to your AI instructions — making them reusable, maintainable, testable, and versionable. Prompt Chaining breaks complex AI tasks into focused, verifiable steps — producing better results than any single “do everything” prompt, and making failures easier to detect and handle. Together they transform AI development from string manipulation into structured application architecture. In Part 33, we’ll cover Function Calling Explained — how AI models can invoke specific functions in your application, extending beyond text generation into real actions. The application layer is almost complete. Each part adds another layer of production capability. Prompt Templates and Prompt Chaining: How to Build AI Workflows That Actually Scale — Part 32 https://blog.stackademic.com/prompt-templates-and-prompt-chaining-how-to-build-ai-workflows-that-actually-scale-part-32-c20e381a4eba was originally published in Stackademic https://blog.stackademic.com on Medium, where people are continuing the conversation by highlighting and responding to this story.