Same chatbot, same question, I asked twice: "What did I say I bought?"
First run, it answers immediately. Headphones, last week. Second run, it has no idea and politely asks me which product I mean.
Nothing changed on the AWS side between those two runs. Same model, same region, same prompt. I changed two lines of my own Python, and that is the entire subject of this post, because those two lines are how every chatbot you have ever used "remembers" you.
I'll show you a support bot for my test shop BrightCart, which is a made-up online electronics store I use across this AWS series so I don't have to keep inventing products. It runs in my terminal, it talks to Amazon Bedrock, and it does three things a single API call doesn't: follows a system prompt, so it knows who it is and how long its answers should be, remembers the conversation and streams the answer as it's written instead of leaving you looking at an empty line.
Model is Amazon Nova 2 Lite. Cheap, fast, and for a support bot answering short questions all day that's the whole shopping list.
Before you start
Python, boto3, an AWS account with Amazon Bedrock available in us-east-1, and credentials that work in your terminal. I use aws login. If none of that is set up yet, the earlier post walks through it: Building GenAI apps and AI agents on AWS: a hands-on path
Code lives in the repo https://github.com/Foxfix/agenticai_mini_course.git
The system prompt
Instructions that sit above the whole conversation. The customer never sees them, the model reads them before every single answer it writes.
Ours is one sentence: you are BrightCart's friendly support assistant, answer in three short sentences or less. That length limit is doing real work, not decoration. Support answers should be short because people are annoyed when they're reading them, and short answers also cost less, since you pay for every token the model produces.
In the Converse API, which is the one request format every Bedrock chat model accepts, the system prompt gets its own field called system, sitting right next to messages. You could paste the instructions onto the front of the first user message instead and it would mostly work. I wouldn't do it. Separate field tells the model "this is policy". Glued onto a user message, it tells the model "a customer said this", and eventually a customer will say something that argues with it.
Memory is a list. That's the whole trick.
Here's the surprise, and it catches almost everyone: the model remembers nothing. Every request starts from zero. There is no session on the Bedrock side quietly holding your conversation, there is nothing to attach to or expire.
So how does any chatbot remember what you said five messages ago? Your code keeps a list: each turn you add the customer’s message, send the whole list back, and add the model’s answer to it, and that last step is the one people drop.
If you skip it, the list only ever contains what the customer said. The bot can see every question it was asked and none of the answers it gave, which produces a very specific and very confusing kind of broken: it repeats itself, it re-introduces itself, it offers you the same three options it already offered. It looks like the model has gone stupid. The model is fine. It's reading a transcript with half the dialogue missing.
Next turn the list is longer, the model sees everything that was said, and that is the memory. All of it. There is no second mechanism.
One side effect to know now rather than later. You resend the entire conversation on every request, so input tokens climb with every message, and a long chat gets more expensive as it goes. Real products trim old messages or summarise them for exactly this reason. For a terminal demo it's irrelevant. The day you put a bot in front of actual customers it stops being irrelevant.
Streaming
A normal converse call makes you wait until the whole answer exists, then hands it over in one piece. converse_stream sends the answer in fragments while the model is still writing, and your code prints each fragment the moment it lands.
Total time is roughly the same. It just feels enormously faster, because the first words appear almost instantly instead of after two seconds of nothing. For anything a human is sitting in front of, that difference is worth the extra six lines of code.
"""A03 - a support chatbot in the terminal: system prompt, memory of the conversation, streaming."""
import sys
sys.path.insert(0, __file__.rsplit("/", 2)[0])
import boto3
from config import REGION, MODEL_ID
bedrock = boto3.client("bedrock-runtime", region_name=REGION)
SYSTEM = [{"text": "You are the friendly support assistant of BrightCart, an online electronics store. Answer in 3 short sentences or less."}]
history = []
print("BrightCart support (type 'quit' to exit)")
while True:
question = input("\nYou: ").strip()
if question.lower() in ("quit", "exit"):
break
history.append({"role": "user", "content": [{"text": question}]})
stream = bedrock.converse_stream(modelId=MODEL_ID, system=SYSTEM, messages=history,
inferenceConfig={"maxTokens": 400, "temperature": 0.3})
print("Bot: ", end="", flush=True)
answer = ""
for event in stream["stream"]:
if "contentBlockDelta" in event:
piece = event["contentBlockDelta"]["delta"].get("text", "")
answer += piece
print(piece, end="", flush=True)
print()
history.append({"role": "assistant", "content": [{"text": answer}]})
REGION and MODEL_ID come out of a small config.py in the repo. MODEL_ID defaults to Nova 2 Lite and you can override it with an environment variable, which is handy when you want to try another model without editing anything.
SYSTEM is a list of content
blocks, same shape as message content. And history = [] is the bot's entire memory. One empty list. That's the thing the title of this post is about.
Inside the loop the message goes into history, converse_stream gets the system prompt and the whole list, and each contentBlockDelta prints the moment it arrives. The last line appends the answer back into history, and we’re about to delete it.
I ran it, told it about the headphones, then asked what I’d said I bought, and it answered correctly, because the list remembered.
python genai/a03_chat.py
Now let's break it
Claiming the memory lives in that list is easy. Proving it takes two edits and about thirty seconds, and it's the most useful thirty seconds in this whole post, because a thing you have broken on purpose is a thing you understand.
Two changes. Comment out the last line, the one that appends the bot's answer. And instead of appending the customer's message to the history, replace the history with that one message:
history = [{"role": "user", "content": [{"text": question}]}]
...
Now every request carries exactly one message and nothing else. The rest of the code remained completely untouched.
Run it again and paste the same two messages. Then the test question, and it has no clue. Same model, same settings, no memory, because the memory was never on the AWS side at all, it was that list.
Then I put it back.
The bot forgets. One of those two lines. Print len(history) in the loop and watch whether it grows.
Streaming doesn't stream. Missing flush=True.
Empty chunks. Not every delta carries text, that's what .get("text", "") is for.
Two answers at once. You pasted two lines.
AccessDeniedException. Expired credentials (aws login lasts 12 hours) or the wrong region.
Fractions of a cent on Nova 2 Lite. The number that climbs is input tokens, since every turn resends the whole conversation, print event["metadata"]["usage"] if you want to see it. Prices move, check the Amazon Bedrock pricing page for your region.
Full code: github.com/Foxfix/agenticai_mini_course
I publish hands-on material on AWS, AI and deployment at @CodeAiFlow. I'm also an AWS Community Builder.