Building AI Agents with Spring AI — Tool Calling, Memory, and Autonomous Workflows

A developer demonstrates how to build AI agents using Spring AI, focusing on tool calling, memory, and autonomous workflows. The post explains how LLMs can be extended beyond text generation to interact with enterprise systems through tools, distinguishing this approach from RAG and illustrating the architecture with examples like order tracking and customer support.

Large Language Models are excellent at generating text. But generation alone isn't enough to build truly useful AI applications. Imagine asking an AI assistant: What's the status of my order? A normal LLM can explain how order tracking works. But it cannot magically access your order database. Or suppose you ask: Cancel my order ORD-10291. The model can tell you how to cancel an order. But it cannot actually cancel anything unless your application gives it the ability to perform that action. This is where tool calling and AI agents come in. Instead of simply generating an answer, an AI application can: Understand the request ↓ Decide what action is required ↓ Select a tool ↓ Execute the tool ↓ Observe the result ↓ Continue reasoning ↓ Generate the final response In this article, we'll explore how to build this architecture using Spring AI . An AI agent is an application where an LLM can decide what actions need to be performed and use available tools to accomplish a goal. A traditional LLM application looks like: User ↓ Prompt ↓ LLM ↓ Response An agent-based application looks more like: User ↓ LLM ↓ Decision ↓ Tool ↓ Result ↓ LLM ↓ Decision ↓ Another Tool ↓ Result ↓ Final Answer The important difference is: The LLM is no longer limited to generating text. It can interact with the application through controlled capabilities. Tool calling allows an LLM to request the execution of a function exposed by your application. For example, imagine our application provides: getOrderStatus cancelOrder getCustomer createSupportTicket The user asks: Where is my order? The model might determine that it needs: getOrderStatus The application executes the function and returns: Order 10291 Status: Shipped Expected delivery: September 10 The LLM can then generate: Your order has been shipped and is expected to arrive on September 10. The LLM didn't directly access the database. Instead: LLM ↓ Tool Request ↓ Application ↓ Database ↓ Tool Result ↓ LLM ↓ Answer This distinction is extremely important for enterprise applications. Without tools: LLM ↓ Text With tools: LLM ↓ Tools ├── Database ├── REST APIs ├── Search ├── Payment systems ├── CRM ├── Internal services └── Business workflows This turns the LLM from a text-generation component into an interface for interacting with your application. For example, an AI sales assistant could have: getCustomer getCustomerOrders createLead updateLead sendEmail scheduleMeeting A support agent could have: searchKnowledgeBase getCustomerAccount getOrder createTicket updateTicket An internal developer assistant could have: searchDocumentation searchGitRepository getBuildStatus createIssue The possibilities are much broader than simple question answering. At this point, it is useful to distinguish RAG from tool calling. RAG is primarily about retrieving information . Tool calling is about performing actions or retrieving live data through application capabilities . For example: RAG ↓ Retrieve company documentation ↓ Answer question Tool calling: LLM ↓ Call order API ↓ Get live order status ↓ Answer They can also be combined. User ↓ AI Agent ├── RAG → Search company policies │ ├── Tool → Get customer account │ └── Tool → Check order status ↓ LLM ↓ Answer This combination is extremely powerful. Spring AI provides abstractions that make it easier to expose application capabilities to language models. A simplified architecture looks like: Spring Boot │ ├── ChatModel │ ├── Tools │ ├── Advisors │ ├── Chat Memory │ └── Vector Store The application controls which tools are available. The model decides whether a tool is needed. This separation is important. The model should not have unrestricted access to your application. Instead, the application exposes specific capabilities. Imagine we have an order service. @Service public class OrderService { public OrderStatus getOrderStatus String orderId { // Fetch order from database return orderRepository.findStatus orderId ; } } We can expose a controlled method as an AI tool. Conceptually: @Tool description = "Get the current status of an order" public OrderStatus getOrderStatus String orderId { return orderService.getOrderStatus orderId ; } The description is important. The model uses the tool description to understand: What does this tool do? When should I use it? What parameters does it require? Tool: getOrderStatus Description: Returns the current shipping and delivery status for a customer order. Input: orderId The model can then determine whether this tool is appropriate. A tool can be thought of as: Tool Name + Description + Input Schema + Execution Logic getOrderStatus orderId: String The model might produce a tool request conceptually like: { "name": "getOrderStatus", "arguments": { "orderId": "ORD-10291" } } The application receives the request and executes the corresponding Java method. A typical interaction looks like this: User │ │ "What's the status of ORD-10291?" ↓ LLM │ │ Tool Request ↓ getOrderStatus "ORD-10291" │ ↓ Order Service │ ↓ Database │ ↓ Tool Result │ ↓ LLM │ ↓ Final Answer Notice something important. The LLM doesn't execute Java code itself. The application remains responsible for execution. The model only requests the action. Spring AI's ChatClient provides a convenient API for interacting with chat models. ChatClient chatClient; String response = chatClient.prompt .user "What's the status of order ORD-10291?" .tools orderTools .call .content ; The exact APIs may vary depending on the Spring AI version you're using, but the architecture remains the same: ChatClient ↓ ChatModel ↓ Tool Selection ↓ Tool Execution ↓ Tool Result ↓ Final Response A real agent usually has more than one tool. CustomerAgentTools ├── getCustomer ├── getCustomerOrders ├── getOrderStatus ├── createSupportTicket └── updateCustomer Now consider this question: My order is late. Please check the status and create a support ticket if necessary. The model might determine: 1. getOrderStatus 2. Analyze result 3. createSupportTicket 4. Return final response The application executes each requested operation. This is where the concept of an agent starts becoming much more interesting. A simple agent loop can be represented as: ┌───────────────┐ │ User │ └───────┬───────┘ ↓ ┌───────────┐ │ LLM │ └─────┬─────┘ ↓ Need a Tool? / \ No Yes ↓ ↓ Final Answer Tool Call ↓ Tool Execution ↓ Tool Result ↓ LLM ↓ Need Another Tool? The model can repeatedly interact with tools until it has enough information to produce the final response. This distinction is important. People often hear: AI Agent and immediately think: Give AI access to everything ↓ Let AI do whatever it wants That is not how production systems should be designed. A production agent should operate inside clear boundaries. Allowed Tools ↓ Authorization ↓ Validation ↓ Execution ↓ Audit The application remains in control. The model should not be trusted with unrestricted capabilities. Imagine we expose: @Tool public void deleteCustomer String customerId { ... } This is potentially dangerous. An LLM should not automatically receive unrestricted permission to perform destructive operations. Instead, sensitive tools should have additional controls. User ↓ Authentication ↓ Authorization ↓ Agent ↓ Tool Request ↓ Permission Check ↓ Confirmation ↓ Execution For destructive operations, you may require explicit user confirmation. Example: AI: I found customer account C-19291. Deleting this account is irreversible. Do you want me to continue? User: Yes. AI: Executing deletion... The AI should assist with the decision process, not bypass your security model. A useful production architecture is to classify tools. READ getCustomer getOrder searchDocuments getInvoice Then: WRITE createTicket updateCustomer createLead And: DESTRUCTIVE deleteCustomer cancelSubscription refundPayment Different permission levels can then be applied. READ → Automatically allowed WRITE → Role-based authorization DESTRUCTIVE → Authorization + confirmation This makes agent behavior much safer. Tool calling solves one problem. But another problem appears quickly: What does the agent remember? Consider this conversation: User: My order is late. AI: What's your order number? User: ORD-10291. AI: Let me check it. Now the next message is: Can you create a support ticket for it? The AI needs to understand that: "it" refers to: ORD-10291 This requires conversational context. That's where chat memory becomes important. A simple conversation can be represented as: User: My order is late. Assistant: What's your order number? User: ORD-10291. Assistant: Let me check that order. The application maintains the conversation history. Conversation ID ↓ Chat Memory ↓ Previous Messages ↓ Current Prompt ↓ LLM Spring AI provides abstractions for managing chat memory. It is useful to distinguish two concepts. Conversation context. User: My order is late. User: It's order 10291. User: Can you check it? The system remembers the current conversation. Persistent information about the user. Customer: Ayush Preferences: Preferred language = English Preferred notification = Email Long-term memory usually requires persistence in a database or another storage system. A production architecture might look like: Conversation ↓ Chat Memory Store ↓ PostgreSQL / Redis The exact storage mechanism depends on the application. Now we can combine: User ↓ Agent ↓ Memory ↓ LLM ↓ Tools ↓ Tool Results ↓ Memory ↓ LLM ↓ Answer This enables more natural multi-turn interactions. Another important Spring AI concept is the Advisor . Advisors can intercept and influence the interaction between the application and the model. They can be used for concerns such as: Conversation memory RAG Logging Security Prompt modification Context injection Observability User ↓ ChatClient ↓ Advisor ↓ ChatModel ↓ Advisor ↓ Response This allows cross-cutting AI behavior to be separated from business logic. Now things become much more powerful. Imagine an enterprise support agent. It has: RAG ↓ Company documentation Tools: getCustomer getOrder createTicket Memory: Conversation history The architecture becomes: User ↓ AI Agent ↓ ┌───────┼────────┐ ↓ ↓ ↓ RAG Tools Memory ↓ ↓ ↓ Knowledge APIs Conversation │ │ │ └───────┼────────┘ ↓ LLM ↓ Response This is much closer to a production AI application. Consider the request: My payment failed for order ORD-19291. Can you check what happened and tell me what I should do? The agent could perform: 1. getOrder "ORD-19291" 2. getPaymentStatus "ORD-19291" 3. searchKnowledgeBase "payment failure" 4. Generate explanation The final answer could be: Your payment attempt failed because the transaction was declined by the payment provider. According to the payment policy, you can retry the payment using another payment method. Would you like me to create a support ticket? The model combined: Live application data + Knowledge base + Conversation context This is significantly more useful than a standalone chatbot. Agents can also perform multi-step workflows. User: Find my overdue invoices and send reminders. The agent could reason through: getCustomer ↓ getInvoices ↓ Filter overdue invoices ↓ sendReminder ↓ Return summary The workflow becomes: Goal ↓ Plan ↓ Tool ↓ Observe ↓ Next Decision ↓ Tool ↓ Observe ↓ Final Result This pattern is often called an agent loop . A simplified conceptual implementation looks like: while completed { AgentDecision decision = llm.decide context ; if decision.requiresTool { ToolResult result = toolExecutor.execute decision.toolCall ; context.add result ; } else { return decision.finalAnswer ; } } In real applications, frameworks handle much of this interaction. But understanding the underlying loop is important. An important engineering lesson: Not every AI feature needs an agent. If your workflow is deterministic: Validate request ↓ Call API ↓ Save result ↓ Return response you probably don't need an autonomous agent. A normal service workflow may be better. Agents become more useful when: The next step depends on the current result. Check order ↓ If delayed ↓ Check refund policy ↓ If eligible ↓ Ask for confirmation ↓ Create refund request The dynamic decision-making is where agents become valuable. A → B → C → D Everything is predetermined. A ↓ LLM decides ├── B ├── C └── D ↓ Observe result ↓ Decide again Agents provide flexibility. Traditional workflows provide predictability. Production systems often use both. A practical Spring Boot architecture might look like: ┌───────────────┐ │ Frontend │ └───────┬───────┘ ↓ ┌───────────────┐ │ Spring Boot │ │ API │ └───────┬───────┘ ↓ ┌────────────┐ │ ChatClient │ └─────┬──────┘ ↓ ┌──────────────┐ │ Agent │ └──────┬───────┘ ↓ ┌────────────┼────────────┐ ↓ ↓ ↓ Memory RAG Tools ↓ ↓ ↓ PostgreSQL pgvector APIs ↓ Microservices This architecture fits naturally into existing Spring Boot applications. Agent systems can become difficult to debug. Imagine an agent performs: Tool 1 Tool 2 Tool 3 Tool 4 and the final response is incorrect. You need to know: What did the model decide? Which tools were selected? What arguments were sent? How long did each tool take? What did each tool return? How many model calls happened? How many tokens were consumed? Therefore, observability is critical. Track: LLM latency Tool latency Retrieval latency Token usage Tool calls Tool failures Model responses Agent iterations Errors An agent can potentially continue calling tools indefinitely. LLM ↓ Tool ↓ LLM ↓ Tool ↓ LLM ↓ Tool ↓ ... Production systems should enforce limits. Maximum iterations = 10 Maximum tool calls = 20 Maximum execution time = 30 seconds You should also define clear failure behavior. Agent limit reached ↓ Stop execution ↓ Return safe response ↓ Log failure Never blindly trust model-generated tool arguments. Suppose the model requests: { "orderId": "ORD-999999999" } Your application should still validate: Does the order exist? Does the user own the order? Is the user authorized? Is the order accessible to this tenant? The architecture should be: LLM ↓ Tool Request ↓ Schema Validation ↓ Authorization ↓ Business Validation ↓ Tool Execution The LLM is not your security boundary. Your application is. This becomes especially important in SaaS applications. Imagine: Tenant A ├── Customers ├── Orders └── Documents Tenant B ├── Customers ├── Orders └── Documents An AI agent must never retrieve Tenant B's information while processing a Tenant A request. Every tool and retrieval operation should carry tenant context. tenant id user id roles permissions User ↓ Authentication ↓ Tenant Context ↓ Agent ↓ Tool ↓ Authorization ↓ Tenant-scoped Data The same principle applies to RAG. Vector Search + tenant id filter should ensure that retrieved documents belong to the correct tenant. Production agents should have explicit guardrails. Examples: Input validation Output validation Tool authorization Rate limiting Token limits Iteration limits PII protection Audit logging Human approval For high-risk actions: Agent ↓ Tool Request ↓ Risk Evaluation ↓ Human Approval ↓ Execution This creates a human-in-the-loop architecture. Not every decision should be fully automated. Refund amount < $50 ↓ Automatic Refund amount $50 ↓ Human approval Or: Create support ticket ↓ Automatic Delete account ↓ Confirmation required This gives us a practical balance: AI Automation + Business Rules + Human Oversight At this point, we can combine everything we've discussed. User ↓ Spring Boot ↓ ChatClient ↓ AI Agent ↓ ┌───────────┼───────────┐ ↓ ↓ ↓ Memory RAG Tools ↓ ↓ ↓ PostgreSQL pgvector REST APIs ↓ Business Services ↓ Database This is a strong foundation for enterprise AI applications. Imagine a sales assistant. Show me the latest opportunities for Acme and tell me which ones are likely to close this month. The agent could: 1. getCustomer "Acme" 2. getOpportunities "Acme" 3. retrieve sales documentation 4. analyze opportunity information 5. generate summary Now the user says: Create a follow-up task for the highest priority opportunity. The agent can: 1. Identify opportunity 2. createFollowUpTask 3. Return task details This is where AI starts becoming an application interface rather than simply a chatbot. Think about the responsibilities this way: LLM = Reasoning + Language RAG = Knowledge Retrieval Tools = Actions + Live Data Memory = Conversation Context Spring Boot = Application + Security + Business Logic Together: LLM + RAG + Tools + Memory + Business Logic = AI Application Spring AI provides abstractions that allow Java developers to work with AI capabilities using familiar Spring patterns. Important building blocks include: ChatClient ChatModel EmbeddingModel VectorStore Document Advisors Chat Memory Tools This means an enterprise Java team can integrate AI into an existing Spring Boot architecture instead of creating an entirely separate AI stack. Existing Spring Boot Application ↓ Spring AI Layer ↓ Model + RAG + Tools ↓ Existing Microservices This makes AI integration much more practical for Java teams. A more complete production system might eventually look like: ┌───────────────┐ │ User │ └───────┬───────┘ ↓ API Gateway ↓ Authentication ↓ Spring Boot API ↓ AI Agent ↓ ┌──────────────────────┼──────────────────────┐ ↓ ↓ ↓ Memory RAG Tools ↓ ↓ ↓ PostgreSQL pgvector Microservices ↓ Business Database ↓ LLM Provider ↓ Response And around the entire system: Security Observability Rate Limiting Audit Logging Guardrails Evaluation These are not optional concerns in serious enterprise deployments. It is useful to understand the difference. User ↓ LLM ↓ Answer User ↓ Retrieve Knowledge ↓ LLM ↓ Answer User ↓ LLM ↓ Tool ↓ Result ↓ Answer User ↓ Agent ↓ Reason ↓ Tool ↓ Observe ↓ Reason ↓ Tool ↓ Observe ↓ Final Answer The complexity increases at every stage. We can now think about the evolution of an AI application: Level 1 LLM ↓ Text Generation Level 2 LLM + RAG ↓ Knowledge Retrieval Level 3 LLM + Tools ↓ Actions Level 4 LLM + Tools + Memory ↓ Contextual Assistant Level 5 LLM + RAG + Tools + Memory ↓ Agent Level 6 Multiple Agents + Workflows ↓ Agentic System This progression is useful when deciding how much complexity your application actually needs. The evolution from traditional AI applications to agentic applications can be summarized as: LLM ↓ Generate Text RAG ↓ Retrieve Knowledge Tool Calling ↓ Take Actions Memory ↓ Remember Context Agents ↓ Make Decisions Workflows ↓ Coordinate Multiple Steps Spring AI provides Java developers with abstractions for building many of these capabilities inside the Spring ecosystem. The most important engineering principle is: Let the model decide, but let your application control. The LLM can decide which tool may be useful. Your application should decide whether that tool is actually allowed to execute. That separation gives us a much safer architecture for enterprise AI. We've now covered three major capabilities: LLM ↓ Generate RAG ↓ Retrieve Tools ↓ Act But there is another challenge. What happens when a system has: Multiple agents ↓ Multiple tools ↓ Multiple services ↓ Multiple AI models How do these agents communicate? How do we standardize tool discovery? How can an AI agent securely interact with external tools and services? This leads us toward another important concept in modern AI engineering: Model Context Protocol — MCP. In the next article, we'll explore: Building MCP Clients and Tool-Based AI Applications with Spring AI. The future of enterprise AI isn't just: LLM → Answer It's increasingly: LLM ↓ Reason ↓ Retrieve ↓ Call Tools ↓ Observe ↓ Act ↓ Remember ↓ Complete the Goal And that's where AI agents with Spring AI become truly interesting.