Why Your AI Chatbot Forgets Everything — And How to Fix It

In the last article we built a working chat endpoint. Send a message, get a reply. It felt like magic.

Then I tried to have an actual conversation.

Me: "My name is Sham."

AI: "Hi Sham! How can I help you?"

Me: "What's my name?"

AI: "I don't have access to personal information about you."

The model had completely forgotten who I was. Not because it was broken — because of something fundamental about how LLMs work. Every API call is completely independent. The model has no memory between calls.

If you want it to remember anything, that's your problem to solve.

This article shows how — starting from the simplest possible solution, hitting its limits, then building the real one.

When you call the Gemini API, you send a list of messages. The model reads them, generates a reply, and the call ends. The next call starts completely fresh — the model has no idea the previous call ever happened.

So when the user sends message 5, the model only sees message 5. It has no knowledge of messages 1 through 4.

The fix is simple in concept: include all previous messages in every call. Send the full conversation history every time, so the model always has context.

Let's build that.

The simplest fix: a Map where the key is a session ID and the value is the list of messages for that session.

@RestController
@RequestMapping("/api/chat")
public class ChatController {

    private final ChatClient chatClient;

    // session ID → list of messages for that session
    private final Map<String, List<Message>> sessions = new ConcurrentHashMap<>();

    public ChatController(ChatClient.Builder builder) {
        this.chatClient = builder
            .defaultSystem("You are a helpful assistant.")
            .build();
    }

    @PostMapping("/session")
    public Map<String, String> startSession() {
        String sessionId = UUID.randomUUID().toString();
        sessions.put(sessionId, new ArrayList<>());
        return Map.of("sessionId", sessionId);
    }

    @PostMapping
    public String chat(@RequestBody ChatRequest request) {
        List<Message> history = sessions.getOrDefault(
            request.sessionId(), new ArrayList<>());

        // Add user message to history
        history.add(new UserMessage(request.message()));

        // Send full history to the model
        String reply = chatClient.prompt()
            .messages(history)
            .call()
            .content();

        // Add model reply to history
        history.add(new AssistantMessage(reply));
        sessions.put(request.sessionId(), history);

        return reply;
    }

    record ChatRequest(String sessionId, String message) {}
}

Every user gets their own session ID. Their messages are stored in their own list. Every API call sends the full history for that session — so the model has context.

Now try the conversation:

Me: "My name is Sham."

AI: "Hi Sham! How can I help you?"

Me: "What's my name?"

AI: "Your name is Sham."

It works. Two different users with two different session IDs — completely separate conversations.

The code is simple. Every Java developer knows what a Map and a List are. No framework magic, just plain Java.

This works great — until you restart the server. All history is gone. Everyone's conversations, gone.

There's another problem: this is a single ArrayList shared across all users. User A and User B are in the same conversation. Not great.

And there's the token problem: a long conversation becomes thousands of tokens on every call, whether those old messages are relevant or not.

In-memory works for a quick demo. For anything real, you need persistent storage with session isolation.

Before writing any code, you need a database. The easiest free option is Neon — serverless PostgreSQL, free tier, no credit card required.

postgresql://username:password@ep-xxx.us-east-1.aws.neon.tech/dbname?sslmode=require
DATABASE_URL=postgresql://username:password@...

That's it. Free, no setup, no local PostgreSQL installation needed.

Spring AI has a built-in JdbcChatMemoryRepository that stores conversation history in a database. Each conversation gets a unique ID — so different users are completely isolated from each other.

Step 1 — Add the dependency

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-chat-memory-repository-jdbc</artifactId>
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
</dependency>

Step 2 — Create the table

Create src/main/resources/chat-memory-schema.sql:

CREATE TABLE IF NOT EXISTS chat_history (
    conversation_id VARCHAR(256) NOT NULL,
    content         TEXT         NOT NULL,
    type            VARCHAR(64)  NOT NULL,
    timestamp       TIMESTAMP    NOT NULL
);

Step 3 — Tell Spring AI to use your table

By default Spring AI uses a table called SPRING_AI_CHAT_MEMORY. To use your own table name, implement JdbcChatMemoryRepositoryDialect:

public class ChatHistoryDialect implements JdbcChatMemoryRepositoryDialect {

    private static final String TABLE = "chat_history";

    @Override
    public String getSelectMessagesSql() {
        return "SELECT content, type FROM " + TABLE +
               " WHERE conversation_id = ? ORDER BY timestamp";
    }

    @Override
    public String getInsertMessageSql() {
        return "INSERT INTO " + TABLE +
               " (conversation_id, content, type, timestamp) VALUES (?, ?, ?, ?)";
    }

    @Override
    public String getSelectConversationIdsSql() {
        return "SELECT DISTINCT conversation_id FROM " + TABLE;
    }

    @Override
    public String getDeleteMessagesSql() {
        return "DELETE FROM " + TABLE + " WHERE conversation_id = ?";
    }
}

Step 4 — Wire it up in the controller

@RestController
@RequestMapping("/api/chat-ai")
public class SpringAiChatController {

    private final ChatClient chatClient;
    private final JdbcChatMemoryRepository memoryRepository;

    public SpringAiChatController(ChatClient.Builder builder, JdbcTemplate jdbcTemplate) {

        this.memoryRepository = JdbcChatMemoryRepository.builder()
            .jdbcTemplate(jdbcTemplate)
            .dialect(new ChatHistoryDialect())
            .build();

        // Keep last 20 messages — older ones are evicted automatically
        MessageWindowChatMemory memory = MessageWindowChatMemory.builder()
            .chatMemoryRepository(memoryRepository)
            .maxMessages(20)
            .build();

        this.chatClient = builder
            .defaultSystem("You are a helpful assistant.")
            .defaultAdvisors(MessageChatMemoryAdvisor.builder(memory).build())
            .build();
    }

    // Create a new session — returns a unique conversation ID
    @PostMapping("/session")
    public Map<String, String> startSession() {
        String conversationId = UUID.randomUUID().toString();
        return Map.of("conversationId", conversationId);
    }

    // Chat — pass the conversation ID with every message
    @PostMapping("/chat")
    public String chat(@RequestBody ChatRequest request) {
        return chatClient.prompt()
            .user(request.message())
            .advisors(a -> a.param("chat_memory_conversation_id", request.conversationId()))
            .call()
            .content();
    }

    // Delete a conversation
    @DeleteMapping("/session/{conversationId}")
    public Map<String, String> deleteSession(@PathVariable String conversationId) {
        memoryRepository.deleteByConversationId(conversationId);
        return Map.of("status", "deleted", "conversationId", conversationId);
    }

    record ChatRequest(String conversationId, String message) {}
}

Step 5 — Configure application.properties

spring.datasource.url=${DATABASE_URL}
spring.datasource.username=${DATABASE_USERNAME}
spring.datasource.password=${DATABASE_PASSWORD}

spring.sql.init.schema-locations=classpath:chat-memory-schema.sql
spring.sql.init.mode=always

mode=always forces the schema script to run on every startup. Without it, PostgreSQL (being a non-embedded database) won't run the script at all.

Starting a conversation:

POST /api/chat-ai/session
→ { "conversationId": "a3f9b2c1-..." }

Sending messages — pass the ID every time:

POST /api/chat-ai/chat
{ "conversationId": "a3f9b2c1-...", "message": "My name is Sham." }
→ "Hi Sham! How can I help you?"

POST /api/chat-ai/chat
{ "conversationId": "a3f9b2c1-...", "message": "What's my name?" }
→ "Your name is Sham."

Spring AI fetches the last 20 messages for that conversation ID from PostgreSQL, includes them in the API call, saves the new exchange, and returns the response. You wrote none of that logic yourself.

Two users, two different conversation IDs — completely isolated. Server restarts — history survives. Long conversation — only the last 20 messages are sent, keeping tokens under control.

Start with in-memory locally. Switch to PostgreSQL before you deploy.

The chat app has memory now. Next up: deploying it to the cloud — Render, Docker, environment variables. Because a chat app that only runs on your laptop isn't a chat app, it's a script.

Have you hit the "it forgot everything" problem before understanding why? Drop it in the comments.

Sham Prakash K — Backend Engineer, 4+ years in Java, Spring Boot, and distributed systems. Building AI backend infrastructure. Writing about what I actually learned, mistakes included.