Why Your AI Chatbot Forgets Everything — And How to Fix It

A developer demonstrated how to give a stateless LLM chatbot persistent memory by storing per-session conversation history in a Java ConcurrentHashMap and resending the full message list on every Gemini API call, then outlined the approach's limits — data loss on server restart, shared-state concurrency issues, and unbounded token growth — and proposed PostgreSQL-backed persistent storage with session isolation as the production fix.

In the last article we built a working chat endpoint. Send a message, get a reply. It felt like magic. Then I tried to have an actual conversation. Me: "My name is Sham." AI: "Hi Sham How can I help you?" Me: "What's my name?" AI: "I don't have access to personal information about you." The model had completely forgotten who I was. Not because it was broken — because of something fundamental about how LLMs work. Every API call is completely independent. The model has no memory between calls. If you want it to remember anything, that's your problem to solve. This article shows how — starting from the simplest possible solution, hitting its limits, then building the real one. When you call the Gemini API, you send a list of messages. The model reads them, generates a reply, and the call ends. The next call starts completely fresh — the model has no idea the previous call ever happened. So when the user sends message 5, the model only sees message 5. It has no knowledge of messages 1 through 4. The fix is simple in concept: include all previous messages in every call. Send the full conversation history every time, so the model always has context. Let's build that. The simplest fix: a Map where the key is a session ID and the value is the list of messages for that session. @RestController @RequestMapping "/api/chat" public class ChatController { private final ChatClient chatClient; // session ID → list of messages for that session private final Map<String, List<Message sessions = new ConcurrentHashMap< ; public ChatController ChatClient.Builder builder { this.chatClient = builder .defaultSystem "You are a helpful assistant." .build ; } @PostMapping "/session" public Map<String, String startSession { String sessionId = UUID.randomUUID .toString ; sessions.put sessionId, new ArrayList< ; return Map.of "sessionId", sessionId ; } @PostMapping public String chat @RequestBody ChatRequest request { List<Message history = sessions.getOrDefault request.sessionId , new ArrayList< ; // Add user message to history history.add new UserMessage request.message ; // Send full history to the model String reply = chatClient.prompt .messages history .call .content ; // Add model reply to history history.add new AssistantMessage reply ; sessions.put request.sessionId , history ; return reply; } record ChatRequest String sessionId, String message {} } Every user gets their own session ID. Their messages are stored in their own list. Every API call sends the full history for that session — so the model has context. Now try the conversation: Me: "My name is Sham." AI: "Hi Sham How can I help you?" Me: "What's my name?" AI: "Your name is Sham." It works. Two different users with two different session IDs — completely separate conversations. The code is simple. Every Java developer knows what a Map and a List are. No framework magic, just plain Java. This works great — until you restart the server. All history is gone. Everyone's conversations, gone. There's another problem: this is a single ArrayList shared across all users. User A and User B are in the same conversation. Not great. And there's the token problem: a long conversation becomes thousands of tokens on every call, whether those old messages are relevant or not. In-memory works for a quick demo. For anything real, you need persistent storage with session isolation. Before writing any code, you need a database. The easiest free option is Neon https://neon.tech — serverless PostgreSQL, free tier, no credit card required. postgresql://username:password@ep-xxx.us-east-1.aws.neon.tech/dbname?sslmode=require DATABASE URL=postgresql://username:password@... That's it. Free, no setup, no local PostgreSQL installation needed. Spring AI has a built-in JdbcChatMemoryRepository that stores conversation history in a database. Each conversation gets a unique ID — so different users are completely isolated from each other. Step 1 — Add the dependency <dependency <groupId org.springframework.ai</groupId <artifactId spring-ai-starter-model-chat-memory-repository-jdbc</artifactId </dependency <dependency <groupId org.postgresql</groupId <artifactId postgresql</artifactId <scope runtime</scope </dependency Step 2 — Create the table Create src/main/resources/chat-memory-schema.sql : CREATE TABLE IF NOT EXISTS chat history conversation id VARCHAR 256 NOT NULL, content TEXT NOT NULL, type VARCHAR 64 NOT NULL, timestamp TIMESTAMP NOT NULL ; Step 3 — Tell Spring AI to use your table By default Spring AI uses a table called SPRING AI CHAT MEMORY . To use your own table name, implement JdbcChatMemoryRepositoryDialect : public class ChatHistoryDialect implements JdbcChatMemoryRepositoryDialect { private static final String TABLE = "chat history"; @Override public String getSelectMessagesSql { return "SELECT content, type FROM " + TABLE + " WHERE conversation id = ? ORDER BY timestamp"; } @Override public String getInsertMessageSql { return "INSERT INTO " + TABLE + " conversation id, content, type, timestamp VALUES ?, ?, ?, ? "; } @Override public String getSelectConversationIdsSql { return "SELECT DISTINCT conversation id FROM " + TABLE; } @Override public String getDeleteMessagesSql { return "DELETE FROM " + TABLE + " WHERE conversation id = ?"; } } Step 4 — Wire it up in the controller @RestController @RequestMapping "/api/chat-ai" public class SpringAiChatController { private final ChatClient chatClient; private final JdbcChatMemoryRepository memoryRepository; public SpringAiChatController ChatClient.Builder builder, JdbcTemplate jdbcTemplate { this.memoryRepository = JdbcChatMemoryRepository.builder .jdbcTemplate jdbcTemplate .dialect new ChatHistoryDialect .build ; // Keep last 20 messages — older ones are evicted automatically MessageWindowChatMemory memory = MessageWindowChatMemory.builder .chatMemoryRepository memoryRepository .maxMessages 20 .build ; this.chatClient = builder .defaultSystem "You are a helpful assistant." .defaultAdvisors MessageChatMemoryAdvisor.builder memory .build .build ; } // Create a new session — returns a unique conversation ID @PostMapping "/session" public Map<String, String startSession { String conversationId = UUID.randomUUID .toString ; return Map.of "conversationId", conversationId ; } // Chat — pass the conversation ID with every message @PostMapping "/chat" public String chat @RequestBody ChatRequest request { return chatClient.prompt .user request.message .advisors a - a.param "chat memory conversation id", request.conversationId .call .content ; } // Delete a conversation @DeleteMapping "/session/{conversationId}" public Map<String, String deleteSession @PathVariable String conversationId { memoryRepository.deleteByConversationId conversationId ; return Map.of "status", "deleted", "conversationId", conversationId ; } record ChatRequest String conversationId, String message {} } Step 5 — Configure application.properties spring.datasource.url=${DATABASE URL} spring.datasource.username=${DATABASE USERNAME} spring.datasource.password=${DATABASE PASSWORD} spring.sql.init.schema-locations=classpath:chat-memory-schema.sql spring.sql.init.mode=always mode=always forces the schema script to run on every startup. Without it, PostgreSQL being a non-embedded database won't run the script at all. Starting a conversation: POST /api/chat-ai/session → { "conversationId": "a3f9b2c1-..." } Sending messages — pass the ID every time: POST /api/chat-ai/chat { "conversationId": "a3f9b2c1-...", "message": "My name is Sham." } → "Hi Sham How can I help you?" POST /api/chat-ai/chat { "conversationId": "a3f9b2c1-...", "message": "What's my name?" } → "Your name is Sham." Spring AI fetches the last 20 messages for that conversation ID from PostgreSQL, includes them in the API call, saves the new exchange, and returns the response. You wrote none of that logic yourself. Two users, two different conversation IDs — completely isolated. Server restarts — history survives. Long conversation — only the last 20 messages are sent, keeping tokens under control. | | Spring AI In-Memory | Spring AI PostgreSQL | |---|---|---| | Setup | Zero | Database + dependency | | Survives restart | No | Yes | | Multi-user | Yes — isolated by session ID | Yes — isolated by session ID | | Token control | Automatic maxMessages | Automatic maxMessages | | Good for | Local dev, quick demos | Production, anything real | Start with in-memory locally. Switch to PostgreSQL before you deploy. The chat app has memory now. Next up: deploying it to the cloud — Render, Docker, environment variables. Because a chat app that only runs on your laptop isn't a chat app, it's a script. Have you hit the "it forgot everything" problem before understanding why? Drop it in the comments. Sham Prakash K — Backend Engineer, 4+ years in Java, Spring Boot, and distributed systems. Building AI backend infrastructure. Writing about what I actually learned, mistakes included.