How Much Memory Does Your Agent Need? — A Practical Memory Store Selection Guide

The Pain: You search GitHub and find everyone using different memory stores — ChromaDB, PostgreSQL, plain Markdown files, "SQLite is good enough for a decade." Which is right?

The Answer: All of them. And none of them.Choosing without understanding your scenario is like buying a car without checking the road.

When I was building a university admissions data scraper (91 universities), I made the classic mistake: I equipped the agent with a full ChromaDB + vector retrieval stack, spent three days tuning it, and then discovered — 95% of the agent's time was just reading "which page did I get to last time."

A boolean would have sufficed. I built a vector database capable of semantic search.

An engineer friend at ByteDance told me their internal agent platform found, after six months, that vector retrieval accounted for only 3.7% of all Memory Store requests. The remaining 96.3%? Key-value lookups, state reads/writes, error dedup. The vector search you spent two weeks integrating might serve less than 4% of your queries.

So before discussing "which storage," we must answer a more fundamental question: what does your agent actually need to remember?

I categorize memory into four types:

See the pattern? If your agent mainly does multi-step automation (data scraping, report generation, CI/CD pipelines), the first three types are the real needs — and none of them require a vector database.

Core thesis: memory selection is about finding the layer that is "just enough." One layer too many is waste; one layer too few is disaster.

I've used each of these across three projects of different scales. The "crash moments" are recorded in the table:

Crash moments (real experience):

Three layers complement each other: L1 fast but small / L2 structured / L3 dedupable.

Here's an architecture validated across three projects. No new dependencies — just a combination that covers "session state + domain knowledge + error history."

┌──────────────────────────────────────────┐
│        Three-Layer Memory Architecture    │
│                                           │
│  L1: memory()      ← 4KB KV, preferences │
│  L2: STATE.md      ← file, progress      │
│  L3: ERROR_LOG.md  ← file, pitfalls      │
│                                           │
│  L1 fast but small / L2 structured /      │
│  L3 dedupable                              │
└──────────────────────────────────────────┘

Best for: user preferences ("reply in Chinese"), project conventions ("outputs go to /outputs/"), environment facts ("Python 3.11").

Fatal trap: the 4,000-char hard limit. The solution isn't to avoid it — treat L1 as a cache and push critical data down to L2.

The backbone of three-layer memory. A single Markdown file, read at session start, updated after task completion. Core code is under 80 lines:

"""Agent's workbench — read each session, update after each task"""
from pathlib import Path
from datetime import datetime
import re

STATE_FILE = Path("./STATE.md")

def load_state() -> str:
    """Call at session start: read STATE.md, inject into agent context."""
    if STATE_FILE.exists():
        content = STATE_FILE.read_text()
        now = datetime.now().strftime("%Y-%m-%d %H:%M")
        return (
            f"> Project current state (last updated: agent-maintained)\n"
            f"> Time: {now}\n\n{content}\n\n"
            f"---\nAbove is project state. Continue based on this.\n"
        )
    return """# Project State File
> Maintained by agent. Read at each session start.

## Active Tasks
(none)

## Blockers
(none)

## Attempted & Failed
(none)

## Project Overview
| Metric | Value | Note |
|--------|-------|------|
| Status | OK | - |
"""

def update_state(section: str, content: str):
    """Call when agent completes a task or finds a blocker."""
    current = STATE_FILE.read_text() if STATE_FILE.exists() else load_state()
    marker = f"## {section}"

    if marker in current:
        parts = current.split(marker, 1)
        before = parts[0]
        after_parts = parts[1].split("\n## ", 1)
        rest = "\n## " + after_parts[1] if len(after_parts) > 1 else ""
        new_content = before + f"{marker}\n{content}\n" + rest
    else:
        new_content = current.rstrip() + f"\n\n{marker}\n{content}\n"

    STATE_FILE.write_text(new_content)
    print(f"[state_manager] Updated section '{section}'")

Append-only log of errors with dedup. The key mechanism: each error gets a fingerprint (e.g., hash of the error message). If the same fingerprint appears, skip it — no duplicate entries.

from pathlib import Path
import hashlib
from datetime import datetime

ERROR_FILE = Path("./ERROR_LOG.md")

def log_error(scene: str, message: str, solution: str = ""):
    """Log an error, dedup by message hash."""
    fp = hashlib.md5(message.encode()).hexdigest()[:8]
    current = ERROR_FILE.read_text() if ERROR_FILE.exists() else ""
    if f"[{fp}]" in current:
        return  # already logged — skip
    entry = (
        f"\n## [{fp}] {scene} @ {datetime.now().strftime('%Y-%m-%d %H:%M')}\n"
        f"- Error: {message}\n"
        f"- Solution: {solution or 'TBD'}\n"
    )
    with ERROR_FILE.open("a") as f:
        f.write(entry)

Three layers complement each other: L1 fast but small / L2 structured & readable / L3 dedupable & queryable.

Need to store...
├── User preference / project convention?  → L1 memory()
├── Task progress / state?                 → L2 STATE.md
├── Error / pitfall?                       → L3 ERROR_LOG.md
├── Structured task data (queries needed)? → SQLite
├── Semantic search?                       → ChromaDB (only if 4% case)
└── Multi-tenant production?               → PostgreSQL

Rule of thumb: start with the simplest layer that works. Add complexity only when you hit a concrete limit.

You're no longer the developer who installs a vector database because "it's what the cool kids use." You're becoming an architect who matches storage to the actual memory pattern of your agent.

Most agents don't need vector search. They need a sticky note, a navigation map, and a pitfall record. The three-layer architecture gives you all three with zero new dependencies.

Next: Quality isn't accidental — Maker/Checker separation and automated validation.

About the author: Wu Ji (无记) — AI & digitalization practitioner focused on Agent engineering, Loop Engineering, and digital transformation. Practical, hands-on tutorials — follow along and it just works.