# I think I used the Hermes Agent wrong

> Source: <https://heyjonny.dev/posts/i-think-i-used-the-hermes-agent-wrong/>
> Published: 2026-09-09 16:03:40+00:00

# I think I used the Hermes Agent wrong

Around three weeks ago, I made the switch from OpenClaw. The idea of a self-learning AI agent (as Nous Research advertises it) hooked me. It also matches how I think about AI agents: I want them to grow with me. I installed the Hermes Agent and gave it a try.

I ran it on my Raspberry Pi 4 with Telegram as the messenger to interact with it. From the outside, I had pretty much the same setup running as before. I even reused my Telegram bot token.

In general, the Hermes Agent feels like a polished version of OpenClaw. It comes with more default options such as TTS or STT, and I never had to tell it to stop sending me tables in the Telegram chat (which OpenClaw loved to do). But I didn’t switch for that. I came for the self-learning aspect.

At the heart of it sits the persistent memory, which consists of two parts. One part, a local SQLite database called `state.db`, contains the application state, including all messages you ever exchanged with the LLM. The agent can query it through a tool called `session_search` to look up past conversations. The other part consists of a `MEMORY.md` and a `USER.md` file. Both get injected into the system prompt when a new session starts. Using the memory tool, the LLM or a background process can add new information to either file. The catch: both have a character limit. If a new piece of information exceeds that limit, the agent has to remove or merge existing entries to make room.

While that all sounds great, I did a little digging through the `state.db`. Actually, I didn’t dig anything. I copied the database from my Pi and asked Claude to run some queries for me. First, I wanted to know how often the agent used `session_search`, and the answer shocked me: 9 times on its own initiative, across the 1,990 messages I sent it. Then again, I ran the whole thing in only 5 sessions, one of them spanning more than a week. Of course it didn’t need to consult its long-term episodic memory. Everything important still sat in the context anyway.

Why only 5 sessions? Because I never really ended one. Every evening I shut down the Pi and restart it in the morning. When I then send a message, the Hermes Agent simply picks up the last session again. Honestly, a super convenient feature. It also means my sessions grew absurdly long and expensive.

The memory files tell the same story. The agent almost never removed entries from `MEMORY.md` or `USER.md`. It almost always compacted them instead, squashing existing entries into ever shorter abbreviations to make room for new ones.

And then it clicked: I used Hermes wrong. The persistent memory assumes a world with many session boundaries: `MEMORY.md` and `USER.md` carry the essentials into the next session, and `session_search` fills the gaps once the old context disappears. My world had almost no boundaries. In 18 days, the cycle of forgetting and looking things up only got to run a handful of times. The agent had no reason to recall anything, because I never let it forget.
