← Blog3 min by Nishant Joshi
This title started you thinking about either of 2 things: a red apple or an iPhone. But I am here to talk about neither. I am rather interested in talking about what just happened. When you opened this you were thinking that you would get either a biological fun fact about apples or some hot and spicy news about Apple and their devices. But because I am not going to talk about either, you might be disappointed or even feel deceived.
Fun fact: that's how the memory systems of these AI agents that you use all the time work. Imagine having a list of things pre-loaded before receiving a single message. That's why, when you are talking about something, if it even slightly touches one of those aforementioned topics, that agent derails. It starts reading through memories and everything you say will be received with respect to what the memories say. Which would be a problem. Just look at this blog. The title was misleading and it set expectations for something this blog isn't about, before you read a line.
What's this blog about? #
I was doing some tests with a new GitHub connector for importing skills in a new project that I am working on. Basically, it pulls all my skills that are public on my GitHub and makes them accessible through one of my MCPs that I wrote. And I wanted to test, so I did what any sane person would do: I popped open a Claude session and I was like: "can you pull this branch and start the server, let's test this locally before we push this to staging"
For what it's worth, I have 3 Neon branches for the Postgres DB I am using: dev, staging, and prod. And Claude knows this, but you won't believe what happened: it took almost 6 minutes to start the server for me to test, while emitting enough characters to write the whole One Piece, twice. I obviously ignored them (that was the blunder, honestly), but what happened was, it had started a local Postgres server on Docker, ran the migrations, and was proudly sitting for me to test. And when I opened the app, none of my accounts or existing test data was there. I was like, classic Claude, it deleted everything. But that was not the case. I started interrogating it, and while that happened, it told me its rationale: the reason why it did this was because somewhere in the memory, it was written that we should do all testing locally, duh! But that's not what I meant. I think I'll stop the rant here and get to the point. I was partially to blame for not auditing its work or making sure it didn't mishear me. But as a tool, it should work around that, shouldn't it?
The same night, I was thinking about how memory works in general: my memory, like the human one :) (idk why I am overexplaining this, lol)
I don't go into conversations with context of what this person is going to say exactly, I would like to but I can't. I don't have these notes and pointers in my head when I talk to people. It's like, when people talk to me (for the sake of this argument, when people prompt me), context appears: "instinctual context," information that's surfaced by my instincts right after someone says something. And after that, I would think about remembering something, aka do a read tool call, or pull out my notes or my phone.
What was that? What's this instinct that I am talking about? Does this happen to you? I would love to know. But that's how I have been thinking about memory recently: it shouldn't be loaded before the conversation, and it shouldn't just be something that the model will call on its own. It's instinct; it's more primal. It appears once you're in the conversation. This is the exact shape that I had in mind, but maybe I am over-anthropomorphizing this.
But when you enter a prompt into a model, there should be a system that goes through all the previous conversations that you've had with the agent, ever, and figures out the relevant bits. No generalizing or Amalgamating progressive disclosure. Finding the exact segments and attaching them, summarized, just in time, with threads to draw upon.
Almost like a just in time memory.