Six months into building AI applications, I made an embarrassing discovery.
I was maintaining 23 different prompt strings scattered across my codebase.
Learning Generative AI From Scratch: The Complete Roadmap (120+ Articles)
Some were hardcoded directly inside service methods. Some were buried in utility classes. A few were duplicated with small variations I couldn’t even explain anymore. One critical prompt — the one that determined how my AI summarized legal documents — existed in three slightly different versions, and I had no idea which one was actually running in production.
When a stakeholder asked me to change the tone of the AI’s responses from formal to conversational, I spent four hours tracking down every prompt string in the codebase.
That’s when I realized I had an engineering problem, not an AI problem.
I was treating prompts like one-off strings — something to write once, make work, and never think about again. But prompts are actually the core logic of an AI application. They deserve the same engineering discipline as any other critical piece of code.
Prompt Templates and Prompt Chaining are the two patterns that changed how I build AI applications — from scattered string concatenation to structured, maintainable, composable AI workflows.
This is Part 32 of Learning Generative AI From Scratch. In Part 31 we covered memory — how AI applications remember across conversations. Now we’re going deeper into the layer that controls what the AI actually does: the prompts themselves.
Memory in AI Applications: How to Make AI Remember Beyond a Single Conversation — Part 31
A Prompt Template is a reusable prompt structure with placeholders for dynamic values.
Instead of building prompt strings manually every time:
// The problem — scattered, unmaintainable prompt stringspublic String summarizeDocument(String document, String audience) { String prompt = "Summarize the following document for a " + audience + " audience. Keep it under 200 words.\n\nDocument:\n" + document; return chatClient.prompt().user(prompt).call().content();}
You define the structure once and inject values:
// The solution — a reusable templateString templateText = """ Summarize the following document for a {audience} audience. Keep it under {maxWords} words. Focus on {focus}. Document: {document} """;
Simple idea. Significant impact on maintainability.
The analogy that made this click for me: SQL prepared statements. Instead of concatenating SQL strings (vulnerable, unreadable, unmaintainable), you define the query structure and inject parameters separately. Prompt templates do the same thing for AI instructions.
Spring AI has first-class support for prompt templates through its PromptTemplate class:
import org.springframework.ai.chat.prompt.PromptTemplate;import org.springframework.ai.chat.prompt.Prompt;@Servicepublic class DocumentSummaryService { private final ChatClient chatClient; public DocumentSummaryService(ChatClient.Builder builder) { this.chatClient = builder.build(); } public String summarize(String document, String audience, int maxWords, String focus) { PromptTemplate template = new PromptTemplate(""" Summarize the following document for a {audience} audience. Keep it under {maxWords} words. Focus specifically on {focus}. Use clear, direct language appropriate for {audience}. Document: {document} """); Map<String, Object> variables = Map.of( "audience", audience, "maxWords", maxWords, "focus", focus, "document", document ); Prompt prompt = template.create(variables); return chatClient.prompt(prompt) .call() .content(); }}
Clean, readable, and the template structure is visible in one place.
But the real power comes when you store templates externally rather than hardcoding them in Java source files.
Storing prompts in your Java source means every prompt change requires a code change, a build, and a deployment. For applications where prompts are actively tuned — which is most of them — this creates unnecessary friction.
Spring AI supports templates from the classpath:
src/main/resources/prompts/summarize.st (StringTemplate format):
You are a professional document summarizer.Your task is to summarize the document below for a {audience} audience.Requirements:- Maximum {maxWords} words- Focus on {focus}- Tone: {tone}- Include key decisions and action items if presentDocument:{document}Summary:
and using it:
@Servicepublic class TemplateService { private final ChatClient chatClient; @Value("classpath:prompts/summarize.st") private Resource summarizeTemplate; @Value("classpath:prompts/classify.st") private Resource classifyTemplate; @Value("classpath:prompts/extract.st") private Resource extractTemplate; public String summarize(Map<String, Object> variables) { try { String templateText = summarizeTemplate .getContentAsString(StandardCharsets.UTF_8); PromptTemplate template = new PromptTemplate(templateText); return chatClient.prompt(template.create(variables)) .call() .content(); } catch (IOException e) { throw new TemplateLoadException("Failed to load summarize template", e); } }}
Now prompt changes are file changes — no recompilation required. In some setups, you can even store prompts in a database and update them without any deployment at all.
A pattern I use in almost every production application: separate system-level instructions from user-level content.
java
@Servicepublic class StructuredPromptService { private final ChatClient chatClient; public String chat(String userContext, String userMessage) { // System template — defines AI behavior, persona, constraints PromptTemplate systemTemplate = new PromptTemplate(""" You are a helpful Java backend development assistant. You specialize in: - Spring Boot applications - Microservices architecture - Java best practices - Performance optimization User context: {userContext} Guidelines: - Always provide working code examples - Explain the "why" not just the "what" - Flag potential production issues - Keep responses practical and specific """); // User template — structures the actual request PromptTemplate userTemplate = new PromptTemplate(""" {message} Please include: 1. A direct answer 2. A code example if applicable 3. One potential gotcha to watch for """); String systemPrompt = systemTemplate.render( Map.of("userContext", userContext) ); String userPrompt = userTemplate.render( Map.of("message", userMessage) ); return chatClient.prompt() .system(systemPrompt) .user(userPrompt) .call() .content(); }}
Why separate them: The system template defines who the AI is and how it behaves — this changes rarely. The user template defines what’s being asked — this changes with every request. Keeping them separate makes each easier to maintain and tune independently.
Prompt Chaining is connecting multiple AI calls together — where the output of one prompt becomes the input to the next.
The simplest mental model: an assembly line.
Station 1 does one thing. Passes its output to Station 2. Station 2 does one thing. Passes to Station 3. Each station is focused, specialized, and testable independently.
Without chaining:
// One massive prompt trying to do everythingString prompt = """ Read this customer support ticket. Determine if it's urgent. If urgent, draft an empathetic response. Also extract the product name mentioned. Also suggest three possible solutions. Also check if this matches any known bug reports. Ticket: {ticket} """;
The problem with “do everything in one prompt”: the model splits its attention across too many tasks. Complex multi-step reasoning in a single prompt produces worse results than breaking the same reasoning into focused sequential steps.
With chaining:
Step 1: Classify the ticket (urgent/normal/low)Step 2: Extract product and issue typeStep 3: Search for matching known issues (using Step 2 output)Step 4: Draft response (using Step 1 + Step 2 + Step 3 outputs)
Each step is focused. Each step’s output is verifiable. If Step 2 extracts the wrong product, you catch it before it propagates through the rest of the chain.
Let’s build a real example — a customer support ticket processing pipeline:
@Servicepublic class SupportTicketPipeline { private final ChatClient chatClient; public SupportTicketPipeline(ChatClient.Builder builder) { this.chatClient = builder.build(); } // Main pipeline method — orchestrates all steps public TicketProcessingResult processTicket(String ticketContent) { // Step 1: Classify urgency UrgencyClassification urgency = classifyUrgency(ticketContent); // Step 2: Extract key information TicketInfo ticketInfo = extractTicketInfo(ticketContent); // Step 3: Find relevant solutions (uses Step 2 output) List<String> suggestedSolutions = findSolutions(ticketInfo.productName(), ticketInfo.issueType()); // Step 4: Draft response (uses all previous outputs) String draftResponse = draftResponse( ticketContent, urgency, ticketInfo, suggestedSolutions ); // Step 5: Quality check the draft (uses Step 4 output) String finalResponse = qualityCheck(draftResponse, urgency); return new TicketProcessingResult( urgency, ticketInfo, suggestedSolutions, finalResponse ); } // Step 1: Classify urgency private UrgencyClassification classifyUrgency(String ticket) { PromptTemplate template = new PromptTemplate(""" Classify the urgency of this support ticket. Respond with ONLY one of these exact values: CRITICAL - system down, data loss, security breach HIGH - major feature broken, significant business impact NORMAL - feature issue, workaround available LOW - question, minor inconvenience, feature request Ticket: {ticket} Urgency level: """); String result = chatClient.prompt( template.create(Map.of("ticket", ticket))) .call() .content() .trim(); return UrgencyClassification.valueOf(result); } // Step 2: Extract structured information private TicketInfo extractTicketInfo(String ticket) { PromptTemplate template = new PromptTemplate(""" Extract information from this support ticket. Respond in this exact JSON format: { "productName": "name of product mentioned or UNKNOWN", "issueType": "one of: authentication, performance, data, integration, other", "customerName": "customer name if mentioned or UNKNOWN", "errorCode": "error code if mentioned or NONE" } Respond with ONLY the JSON. No explanation. Ticket: {ticket} """); String jsonResult = chatClient.prompt( template.create(Map.of("ticket", ticket))) .call() .content(); return parseTicketInfo(jsonResult); } // Step 3: Find solutions based on extracted info private List<String> findSolutions(String productName, String issueType) { PromptTemplate template = new PromptTemplate(""" Suggest three practical solutions for this issue. Product: {productName} Issue type: {issueType} Format each solution as a single actionable sentence. Number them 1, 2, 3. Be specific and practical, not generic. """); String result = chatClient.prompt( template.create(Map.of( "productName", productName, "issueType", issueType ))) .call() .content(); return Arrays.asList(result.split("\n")) .stream() .filter(s -> s.matches("^[123].*")) .collect(Collectors.toList()); } // Step 4: Draft the actual response private String draftResponse(String originalTicket, UrgencyClassification urgency, TicketInfo ticketInfo, List<String> solutions) { PromptTemplate template = new PromptTemplate(""" Draft a customer support response for this ticket. Original ticket: {ticket} Context: - Urgency: {urgency} - Issue type: {issueType} - Customer name: {customerName} Suggested solutions to incorporate: {solutions} Requirements: - Open with empathy appropriate to {urgency} urgency - Address the customer by name if known - Present the most likely solution first - Keep it under 150 words - End with next steps - Professional but warm tone """); return chatClient.prompt( template.create(Map.of( "ticket", originalTicket, "urgency", urgency.name(), "issueType", ticketInfo.issueType(), "customerName", ticketInfo.customerName(), "solutions", String.join("\n", solutions) ))) .call() .content(); } // Step 5: Quality check before sending private String qualityCheck(String draft, UrgencyClassification urgency) { PromptTemplate template = new PromptTemplate(""" Review this customer support response draft. Urgency level: {urgency} Check for: 1. Is the tone appropriate for {urgency} urgency? 2. Are there any promises we can't keep? 3. Is any sensitive information accidentally included? 4. Is it under 150 words? If the draft is acceptable, return it unchanged. If it needs fixes, return the corrected version. Return ONLY the response text — no explanation. Draft: {draft} """); return chatClient.prompt( template.create(Map.of( "urgency", urgency.name(), "draft", draft ))) .call() .content(); } // Supporting records public enum UrgencyClassification { CRITICAL, HIGH, NORMAL, LOW } public record TicketInfo( String productName, String issueType, String customerName, String errorCode ) {} public record TicketProcessingResult( UrgencyClassification urgency, TicketInfo ticketInfo, List<String> suggestedSolutions, String finalResponse ) {}}
Five focused prompts. Each does one thing. Each output is verifiable. The whole pipeline produces better results than any single “do everything” prompt could.
Not every chain is linear. Real workflows branch based on intermediate results:
@Servicepublic class ConditionalChainService { private final ChatClient chatClient; private final SupportTicketPipeline ticketPipeline; public String handleRequest(String userInput) { // Step 1: Route — decide which path to take String intent = classifyIntent(userInput); // Step 2: Branch based on intent return switch (intent) { case "SUPPORT_TICKET" -> { TicketProcessingResult result = ticketPipeline.processTicket(userInput); yield result.finalResponse(); } case "PRODUCT_QUESTION" -> handleProductQuestion(userInput); case "BILLING_ISSUE" -> handleBillingIssue(userInput); default -> handleGeneral(userInput); }; } private String classifyIntent(String input) { PromptTemplate template = new PromptTemplate(""" Classify this user input into exactly one category. Categories: SUPPORT_TICKET - reporting a bug or technical problem PRODUCT_QUESTION - asking how something works BILLING_ISSUE - payment or subscription related GENERAL - anything else Respond with ONLY the category name. Input: {input} """); return chatClient.prompt( template.create(Map.of("input", input))) .call() .content() .trim(); }}
This pattern — classify first, then route to specialized chains — is the foundation of most production AI workflow systems.
Some tasks don’t need to run sequentially. When steps are independent, run them in parallel:
@Servicepublic class ParallelChainService { private final ChatClient chatClient; public DocumentAnalysisResult analyzeDocument(String document) throws InterruptedException, ExecutionException { ExecutorService executor = Executors.newFixedThreadPool(3); // Run three independent analyses in parallel Future<String> summaryFuture = executor.submit( () -> summarize(document) ); Future<List<String>> keyPointsFuture = executor.submit( () -> extractKeyPoints(document) ); Future<String> sentimentFuture = executor.submit( () -> analyzeSentiment(document) ); // Wait for all to complete String summary = summaryFuture.get(); List<String> keyPoints = keyPointsFuture.get(); String sentiment = sentimentFuture.get(); executor.shutdown(); // Combine results in a final synthesis step return synthesizeResults(summary, keyPoints, sentiment); } private DocumentAnalysisResult synthesizeResults( String summary, List<String> keyPoints, String sentiment) { PromptTemplate template = new PromptTemplate(""" Create a final executive briefing from these analyses. Summary: {summary} Key Points: {keyPoints} Overall Sentiment: {sentiment} Produce a cohesive 3-paragraph briefing that integrates all three analyses naturally. """); String briefing = chatClient.prompt( template.create(Map.of( "summary", summary, "keyPoints", String.join(", ", keyPoints), "sentiment", sentiment ))) .call() .content(); return new DocumentAnalysisResult( summary, keyPoints, sentiment, briefing ); }}
Three independent analyses run simultaneously, then a final synthesis step combines them. Total time is the maximum of the three parallel steps, not their sum.
Chains fail at individual steps. Handle it gracefully:
@Servicepublic class ResilientChainService { private final ChatClient chatClient; public ProcessingResult processWithFallback(String input) { try { // Attempt the full chain return runFullChain(input); } catch (ChainStepException e) { log.warn("Chain step {} failed: {}", e.getStep(), e.getMessage()); // Fallback: skip failed step, continue with defaults return runFallbackChain(input, e.getStep()); } } private ProcessingResult runFallbackChain(String input, String failedStep) { // If classification failed, default to NORMAL if ("classification".equals(failedStep)) { return runChainFromStep(input, "extraction", UrgencyClassification.NORMAL); } // If extraction failed, use minimal defaults if ("extraction".equals(failedStep)) { return generateGenericResponse(input); } // If drafting failed, return a safe template response return safeTemplateResponse(); } private ProcessingResult safeTemplateResponse() { return new ProcessingResult( "Thank you for contacting us. A team member will " + "review your request and respond within 24 hours.", true // flagged for human review ); }}
Rule I follow: every chain step should have a defined fallback behavior. A chain that fails completely and returns an error to the user is worse than a chain that degrades gracefully.
Once your application goes live, prompts need to evolve. Users give feedback. Edge cases appear. The model gets updated.
Without version control for prompts, you lose the ability to roll back a bad change:
@Entity@Table(name = "prompt_templates")public class PromptTemplate { @Id @GeneratedValue(strategy = GenerationType.IDENTITY) private Long id; private String templateName; private String version; @Column(columnDefinition = "TEXT") private String content; private boolean active; private LocalDateTime createdAt; private String createdBy; private String changeNote;}@Repositorypublic interface PromptTemplateRepository extends JpaRepository<PromptTemplate, Long> { Optional<PromptTemplate> findByTemplateNameAndActive( String templateName, boolean active); List<PromptTemplate> findByTemplateNameOrderByCreatedAtDesc( String templateName);}@Servicepublic class VersionedTemplateService { private final PromptTemplateRepository repository; private final ChatClient chatClient; public String renderWithActiveTemplate(String templateName, Map<String, Object> variables) { PromptTemplate template = repository .findByTemplateNameAndActive(templateName, true) .orElseThrow(() -> new TemplateNotFoundException(templateName)); org.springframework.ai.chat.prompt.PromptTemplate springTemplate = new org.springframework.ai.chat.prompt.PromptTemplate( template.getContent() ); return chatClient.prompt(springTemplate.create(variables)) .call() .content(); } public void activateVersion(String templateName, String version) { // Deactivate current active version repository.findByTemplateNameAndActive(templateName, true) .ifPresent(t -> { t.setActive(false); repository.save(t); }); // Activate new version repository.findByTemplateNameAndVersion(templateName, version) .ifPresent(t -> { t.setActive(true); repository.save(t); }); }}
Now prompt changes are database changes — tracked, auditable, and reversible without a deployment.
When I cleaned up those 23 scattered prompt strings and refactored them into versioned, externalized templates feeding into proper chains, something unexpected happened.
The AI’s output quality improved — not because I changed the instructions, just because I made them consistent.
What I realized: prompt inconsistency is one of the sneakiest sources of AI application quality problems. When the same logical operation runs on slightly different prompt strings depending on which code path executed, you get subtly different outputs that are very hard to debug. You end up chasing what feels like model inconsistency when it’s actually prompt inconsistency.
Templates force consistency. Chains force single-responsibility. And both together make AI applications significantly easier to test, debug, and improve — the same way software engineering principles improve any complex codebase.
The lesson: AI applications need software engineering. Not instead of AI thinking, but in addition to it.
Build a three-step content pipeline:
Test with the same input paragraph going through all three steps. Observe how each step builds on the previous one’s output. Then try breaking it by giving Step 2 a wrong classification — see how the error propagates through the chain.
That propagation behavior is the most important thing to understand about chaining — errors compound downstream, which is why Step 1 accuracy matters so much.
Prompt Templates bring software engineering discipline to your AI instructions — making them reusable, maintainable, testable, and versionable.
Prompt Chaining breaks complex AI tasks into focused, verifiable steps — producing better results than any single “do everything” prompt, and making failures easier to detect and handle.
Together they transform AI development from string manipulation into structured application architecture.
In Part 33, we’ll cover Function Calling Explained — how AI models can invoke specific functions in your application, extending beyond text generation into real actions.
The application layer is almost complete. Each part adds another layer of production capability.
Prompt Templates and Prompt Chaining: How to Build AI Workflows That Actually Scale — Part 32 was originally published in Stackademic on Medium, where people are continuing the conversation by highlighting and responding to this story.