{"slug": "why-your-ai-chatbot-forgets-everything-and-how-to-fix-it", "title": "Why Your AI Chatbot Forgets Everything — And How to Fix It", "summary": "A developer demonstrated how to give a stateless LLM chatbot persistent memory by storing per-session conversation history in a Java ConcurrentHashMap and resending the full message list on every Gemini API call, then outlined the approach's limits — data loss on server restart, shared-state concurrency issues, and unbounded token growth — and proposed PostgreSQL-backed persistent storage with session isolation as the production fix.", "body_md": "In the last article we built a working chat endpoint. Send a message, get a reply. It felt like magic.\n\nThen I tried to have an actual conversation.\n\nMe: \"My name is Sham.\"\n\nAI: \"Hi Sham! How can I help you?\"\n\nMe: \"What's my name?\"\n\nAI: \"I don't have access to personal information about you.\"\n\nThe model had completely forgotten who I was. Not because it was broken — because of something fundamental about how LLMs work. Every API call is completely independent. The model has no memory between calls.\n\nIf you want it to remember anything, that's your problem to solve.\n\nThis article shows how — starting from the simplest possible solution, hitting its limits, then building the real one.\n\nWhen you call the Gemini API, you send a list of messages. The model reads them, generates a reply, and the call ends. The next call starts completely fresh — the model has no idea the previous call ever happened.\n\nSo when the user sends message 5, the model only sees message 5. It has no knowledge of messages 1 through 4.\n\nThe fix is simple in concept: **include all previous messages in every call.** Send the full conversation history every time, so the model always has context.\n\nLet's build that.\n\nThe simplest fix: a `Map` where the key is a session ID and the value is the list of messages for that session.\n\n```\n@RestController\n@RequestMapping(\"/api/chat\")\npublic class ChatController {\n\n    private final ChatClient chatClient;\n\n    // session ID → list of messages for that session\n    private final Map<String, List<Message>> sessions = new ConcurrentHashMap<>();\n\n    public ChatController(ChatClient.Builder builder) {\n        this.chatClient = builder\n            .defaultSystem(\"You are a helpful assistant.\")\n            .build();\n    }\n\n    @PostMapping(\"/session\")\n    public Map<String, String> startSession() {\n        String sessionId = UUID.randomUUID().toString();\n        sessions.put(sessionId, new ArrayList<>());\n        return Map.of(\"sessionId\", sessionId);\n    }\n\n    @PostMapping\n    public String chat(@RequestBody ChatRequest request) {\n        List<Message> history = sessions.getOrDefault(\n            request.sessionId(), new ArrayList<>());\n\n        // Add user message to history\n        history.add(new UserMessage(request.message()));\n\n        // Send full history to the model\n        String reply = chatClient.prompt()\n            .messages(history)\n            .call()\n            .content();\n\n        // Add model reply to history\n        history.add(new AssistantMessage(reply));\n        sessions.put(request.sessionId(), history);\n\n        return reply;\n    }\n\n    record ChatRequest(String sessionId, String message) {}\n}\n```\n\nEvery user gets their own session ID. Their messages are stored in their own list. Every API call sends the full history for that session — so the model has context.\n\nNow try the conversation:\n\nMe: \"My name is Sham.\"\n\nAI: \"Hi Sham! How can I help you?\"\n\nMe: \"What's my name?\"\n\nAI: \"Your name is Sham.\"\n\nIt works. Two different users with two different session IDs — completely separate conversations.\n\nThe code is simple. Every Java developer knows what a `Map` and a `List` are. No framework magic, just plain Java.\n\nThis works great — until you restart the server. All history is gone. Everyone's conversations, gone.\n\nThere's another problem: this is a single `ArrayList` shared across all users. User A and User B are in the same conversation. Not great.\n\nAnd there's the token problem: a long conversation becomes thousands of tokens on every call, whether those old messages are relevant or not.\n\nIn-memory works for a quick demo. For anything real, you need persistent storage with session isolation.\n\nBefore writing any code, you need a database. The easiest free option is [Neon](https://neon.tech) — serverless PostgreSQL, free tier, no credit card required.\n\n```\npostgresql://username:password@ep-xxx.us-east-1.aws.neon.tech/dbname?sslmode=require\nDATABASE_URL=postgresql://username:password@...\n```\n\nThat's it. Free, no setup, no local PostgreSQL installation needed.\n\nSpring AI has a built-in `JdbcChatMemoryRepository` that stores conversation history in a database. Each conversation gets a unique ID — so different users are completely isolated from each other.\n\n**Step 1 — Add the dependency**\n\n```\n<dependency>\n    <groupId>org.springframework.ai</groupId>\n    <artifactId>spring-ai-starter-model-chat-memory-repository-jdbc</artifactId>\n</dependency>\n<dependency>\n    <groupId>org.postgresql</groupId>\n    <artifactId>postgresql</artifactId>\n    <scope>runtime</scope>\n</dependency>\n```\n\n**Step 2 — Create the table**\n\nCreate `src/main/resources/chat-memory-schema.sql`:\n\n```\nCREATE TABLE IF NOT EXISTS chat_history (\n    conversation_id VARCHAR(256) NOT NULL,\n    content         TEXT         NOT NULL,\n    type            VARCHAR(64)  NOT NULL,\n    timestamp       TIMESTAMP    NOT NULL\n);\n```\n\n**Step 3 — Tell Spring AI to use your table**\n\nBy default Spring AI uses a table called `SPRING_AI_CHAT_MEMORY`. To use your own table name, implement `JdbcChatMemoryRepositoryDialect`:\n\n```\npublic class ChatHistoryDialect implements JdbcChatMemoryRepositoryDialect {\n\n    private static final String TABLE = \"chat_history\";\n\n    @Override\n    public String getSelectMessagesSql() {\n        return \"SELECT content, type FROM \" + TABLE +\n               \" WHERE conversation_id = ? ORDER BY timestamp\";\n    }\n\n    @Override\n    public String getInsertMessageSql() {\n        return \"INSERT INTO \" + TABLE +\n               \" (conversation_id, content, type, timestamp) VALUES (?, ?, ?, ?)\";\n    }\n\n    @Override\n    public String getSelectConversationIdsSql() {\n        return \"SELECT DISTINCT conversation_id FROM \" + TABLE;\n    }\n\n    @Override\n    public String getDeleteMessagesSql() {\n        return \"DELETE FROM \" + TABLE + \" WHERE conversation_id = ?\";\n    }\n}\n```\n\n**Step 4 — Wire it up in the controller**\n\n```\n@RestController\n@RequestMapping(\"/api/chat-ai\")\npublic class SpringAiChatController {\n\n    private final ChatClient chatClient;\n    private final JdbcChatMemoryRepository memoryRepository;\n\n    public SpringAiChatController(ChatClient.Builder builder, JdbcTemplate jdbcTemplate) {\n\n        this.memoryRepository = JdbcChatMemoryRepository.builder()\n            .jdbcTemplate(jdbcTemplate)\n            .dialect(new ChatHistoryDialect())\n            .build();\n\n        // Keep last 20 messages — older ones are evicted automatically\n        MessageWindowChatMemory memory = MessageWindowChatMemory.builder()\n            .chatMemoryRepository(memoryRepository)\n            .maxMessages(20)\n            .build();\n\n        this.chatClient = builder\n            .defaultSystem(\"You are a helpful assistant.\")\n            .defaultAdvisors(MessageChatMemoryAdvisor.builder(memory).build())\n            .build();\n    }\n\n    // Create a new session — returns a unique conversation ID\n    @PostMapping(\"/session\")\n    public Map<String, String> startSession() {\n        String conversationId = UUID.randomUUID().toString();\n        return Map.of(\"conversationId\", conversationId);\n    }\n\n    // Chat — pass the conversation ID with every message\n    @PostMapping(\"/chat\")\n    public String chat(@RequestBody ChatRequest request) {\n        return chatClient.prompt()\n            .user(request.message())\n            .advisors(a -> a.param(\"chat_memory_conversation_id\", request.conversationId()))\n            .call()\n            .content();\n    }\n\n    // Delete a conversation\n    @DeleteMapping(\"/session/{conversationId}\")\n    public Map<String, String> deleteSession(@PathVariable String conversationId) {\n        memoryRepository.deleteByConversationId(conversationId);\n        return Map.of(\"status\", \"deleted\", \"conversationId\", conversationId);\n    }\n\n    record ChatRequest(String conversationId, String message) {}\n}\n```\n\n**Step 5 — Configure `application.properties`**\n\n```\nspring.datasource.url=${DATABASE_URL}\nspring.datasource.username=${DATABASE_USERNAME}\nspring.datasource.password=${DATABASE_PASSWORD}\n\nspring.sql.init.schema-locations=classpath:chat-memory-schema.sql\nspring.sql.init.mode=always\n```\n\n`mode=always` forces the schema script to run on every startup. Without it, PostgreSQL (being a non-embedded database) won't run the script at all.\n\n**Starting a conversation:**\n\n```\nPOST /api/chat-ai/session\n→ { \"conversationId\": \"a3f9b2c1-...\" }\n```\n\n**Sending messages — pass the ID every time:**\n\n```\nPOST /api/chat-ai/chat\n{ \"conversationId\": \"a3f9b2c1-...\", \"message\": \"My name is Sham.\" }\n→ \"Hi Sham! How can I help you?\"\n\nPOST /api/chat-ai/chat\n{ \"conversationId\": \"a3f9b2c1-...\", \"message\": \"What's my name?\" }\n→ \"Your name is Sham.\"\n```\n\nSpring AI fetches the last 20 messages for that conversation ID from PostgreSQL, includes them in the API call, saves the new exchange, and returns the response. You wrote none of that logic yourself.\n\nTwo users, two different conversation IDs — completely isolated. Server restarts — history survives. Long conversation — only the last 20 messages are sent, keeping tokens under control.\n\n|  | Spring AI In-Memory | Spring AI PostgreSQL | \n|---|---|---|\n| **Setup** | Zero | Database + dependency | \n| **Survives restart** | No | Yes | \n| **Multi-user** | Yes — isolated by session ID | Yes — isolated by session ID | \n| **Token control** | Automatic (maxMessages) | Automatic (maxMessages) | \n| **Good for** | Local dev, quick demos | Production, anything real | \n\nStart with in-memory locally. Switch to PostgreSQL before you deploy.\n\nThe chat app has memory now. Next up: deploying it to the cloud — Render, Docker, environment variables. Because a chat app that only runs on your laptop isn't a chat app, it's a script.\n\nHave you hit the \"it forgot everything\" problem before understanding why? Drop it in the comments.\n\n*Sham Prakash K — Backend Engineer, 4+ years in Java, Spring Boot, and distributed systems. Building AI backend infrastructure. Writing about what I actually learned, mistakes included.*", "url": "https://wpnews.pro/news/why-your-ai-chatbot-forgets-everything-and-how-to-fix-it", "canonical_source": "https://dev.to/shamprakash2000/why-your-ai-chatbot-forgets-everything-and-how-to-fix-it-26je", "published_at": "2026-09-23 14:30:00+00:00", "updated_at": "2026-09-23 14:58:54.086993+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "developer-tools", "ai-products"], "entities": ["Gemini", "Neon", "PostgreSQL", "Java"], "alternates": {"html": "https://wpnews.pro/news/why-your-ai-chatbot-forgets-everything-and-how-to-fix-it", "markdown": "https://wpnews.pro/news/why-your-ai-chatbot-forgets-everything-and-how-to-fix-it.md", "text": "https://wpnews.pro/news/why-your-ai-chatbot-forgets-everything-and-how-to-fix-it.txt", "jsonld": "https://wpnews.pro/news/why-your-ai-chatbot-forgets-everything-and-how-to-fix-it.jsonld"}}