Around three weeks ago, I made the switch from OpenClaw. The idea of a self-learning AI agent (as Nous Research advertises it) hooked me. It also matches how I think about AI agents: I want them to grow with me. I installed the Hermes Agent and gave it a try.
I ran it on my Raspberry Pi 4 with Telegram as the messenger to interact with it. From the outside, I had pretty much the same setup running as before. I even reused my Telegram bot token.
In general, the Hermes Agent feels like a polished version of OpenClaw. It comes with more default options such as TTS or STT, and I never had to tell it to stop sending me tables in the Telegram chat (which OpenClaw loved to do). But I didn’t switch for that. I came for the self-learning aspect.
At the heart of it sits the persistent memory, which consists of two parts. One part, a local SQLite database called state.db, contains the application state, including all messages you ever exchanged with the LLM. The agent can query it through a tool called session_search to look up past conversations. The other part consists of a MEMORY.md and a USER.md file. Both get injected into the system prompt when a new session starts. Using the memory tool, the LLM or a background process can add new information to either file. The catch: both have a character limit. If a new piece of information exceeds that limit, the agent has to remove or merge existing entries to make room.
While that all sounds great, I did a little digging through the state.db. Actually, I didn’t dig anything. I copied the database from my Pi and asked Claude to run some queries for me. First, I wanted to know how often the agent used session_search, and the answer shocked me: 9 times on its own initiative, across the 1,990 messages I sent it. Then again, I ran the whole thing in only 5 sessions, one of them spanning more than a week. Of course it didn’t need to consult its long-term episodic memory. Everything important still sat in the context anyway.
Why only 5 sessions? Because I never really ended one. Every evening I shut down the Pi and restart it in the morning. When I then send a message, the Hermes Agent simply picks up the last session again. Honestly, a super convenient feature. It also means my sessions grew absurdly long and expensive.
The memory files tell the same story. The agent almost never removed entries from MEMORY.md or USER.md. It almost always compacted them instead, squashing existing entries into ever shorter abbreviations to make room for new ones.
And then it clicked: I used Hermes wrong. The persistent memory assumes a world with many session boundaries: MEMORY.md and USER.md carry the essentials into the next session, and session_search fills the gaps once the old context disappears. My world had almost no boundaries. In 18 days, the cycle of forgetting and looking things up only got to run a handful of times. The agent had no reason to recall anything, because I never let it forget.