{"slug": "stop-stuffing-your-llm-agent-s-context-window-structured-memory-categories-with", "title": "Stop Stuffing Your LLM Agent's Context Window: Structured Memory Categories with Mem0", "summary": "A developer proposes using typed memory categories with Mem0 to improve LLM agent memory management, addressing failures from treating all facts equally. By storing memories as 'user', 'feedback', 'project', or 'reference' types with metadata, agents can retrieve and expire context appropriately, avoiding stale data and context window overload.", "body_md": "Most tutorials on giving an LLM agent \"memory\" show you the same three lines:\n\n```\nm = Memory()\nm.add(\"User likes dark mode\", user_id=\"alice\")\nm.search(\"What does the user prefer?\", user_id=\"alice\")\n```\n\nThis works in a demo. It falls apart in a real agent that runs for weeks, because it treats every fact as equally important and equally permanent. In practice, an agent accumulates at least four *different kinds* of memory that decay, get retrieved, and get invalidated in completely different ways. If you store them all the same way, you get one of two failure modes: the agent re-reads stale project state as if it were still true, or it drowns its context window in low-value trivia every time it calls `search()`\n\n.\n\nThis article walks through a typed memory schema on top of Mem0 that fixes both problems, with working code.\n\nSay your agent is a coding assistant working across sessions on the same repo. Over a few weeks it will learn things like:\n\n`INGEST`\n\nproject in Linear.\"These look similar as text, but they behave completely differently:\n\nA flat `memory.add(text)`\n\ncall has no way to express this. When you later call `search()`\n\n, Mem0's relevance ranking will happily surface a three-week-old \"ETA next Thursday\" note alongside a permanent user preference, because both score similarly on semantic similarity to your query.\n\nMem0's `add()`\n\naccepts arbitrary `metadata`\n\n, and `search()`\n\n/`get_all()`\n\nsupport filtering on it. That's enough to build a lightweight type system without touching Mem0's internals.\n\n``` python\nfrom mem0 import Memory\n\nm = Memory()\n\ndef remember(text, user_id, kind, **extra):\n    \"\"\"kind: 'user' | 'feedback' | 'project' | 'reference'\"\"\"\n    m.add(\n        [{\"role\": \"user\", \"content\": text}],\n        user_id=user_id,\n        metadata={\"kind\": kind, **extra},\n    )\n\nremember(\n    \"User is a backend engineer, new to the React side of this repo.\",\n    user_id=\"alice\",\n    kind=\"user\",\n)\n\nremember(\n    \"Don't mock the database in integration tests — a mocked/prod \"\n    \"divergence masked a broken migration last quarter.\",\n    user_id=\"alice\",\n    kind=\"feedback\",\n    scope=\"testing\",\n)\n\nremember(\n    \"Auth middleware rewrite is blocked on legal review of session \"\n    \"token storage. Target: 2026-08-06.\",\n    user_id=\"alice\",\n    kind=\"project\",\n    expires=\"2026-08-13\",\n)\n\nremember(\n    \"Bug reports live in Linear project 'INGEST'.\",\n    user_id=\"alice\",\n    kind=\"reference\",\n)\n```\n\nRetrieval now becomes a two-step process instead of one blind semantic search: pull relevant memories by kind, then let the LLM decide how to use each type.\n\n``` python\ndef load_context(user_id, query):\n    facts = m.search(query, user_id=user_id, filters={\"kind\": \"user\"})\n    rules = m.get_all(user_id=user_id, filters={\"kind\": \"feedback\"})\n    state = m.search(query, user_id=user_id, filters={\"kind\": \"project\"})\n    return {\n        \"facts\": [r[\"memory\"] for r in facts[\"results\"]],\n        \"rules\": [r[\"memory\"] for r in rules[\"results\"]],\n        \"state\": [r[\"memory\"] for r in state[\"results\"]],\n    }\n```\n\nYou now compose the system prompt from three distinct sections instead of one undifferentiated memory dump — \"here's who the user is,\" \"here are standing rules you must not violate,\" \"here's what's currently in flight.\" Feedback-kind memories, in particular, should be injected *unconditionally* near the top of the system prompt rather than semantically retrieved — a correction like \"don't skip pre-commit hooks\" needs to apply even when the current query has no lexical overlap with \"hooks.\"\n\nProject-state memories are the ones that cause real bugs when stale. Mem0 doesn't auto-expire memories, so build expiry into your read path, not just your write path:\n\n``` python\nfrom datetime import date\n\ndef load_project_state(user_id, query):\n    results = m.search(query, user_id=user_id, filters={\"kind\": \"project\"})\n    today = date.today().isoformat()\n    fresh = []\n    for r in results[\"results\"]:\n        expires = r.get(\"metadata\", {}).get(\"expires\")\n        if expires and expires < today:\n            m.delete(r[\"id\"])   # prune, don't just skip\n            continue\n        fresh.append(r[\"memory\"])\n    return fresh\n```\n\nPruning on read (rather than a separate cron job) keeps the store self-cleaning without extra infrastructure, and it means your token budget for the \"project state\" section of the prompt never grows unbounded.\n\nThe single biggest inefficiency I've seen in Mem0 integrations is calling `search()`\n\nonce per turn with the raw user message as the query, at `top_k`\n\ndefaults, for every memory kind. That's 3-4 vector searches and several KB of retrieved text per turn, most of which is irrelevant to a short follow-up question like \"did that work?\"\n\nTwo cheap fixes:\n\n`feedback`\n\n-kind memories in-process`project`\n\n-kind search entirelySplitting memory by kind instead of using one flat store did three concrete things in a long-running coding agent I maintain: it stopped stale \"in progress\" notes from being read back as current fact, it let behavioral corrections apply consistently instead of depending on semantic luck, and it cut average per-turn retrieved-memory tokens by roughly 40% by making expiry and caching possible in the first place. None of this requires anything beyond what `mem0ai`\n\nalready exposes — `metadata`\n\nand `filters`\n\nare enough to build a real type system on top of a memory store that, out of the box, treats every fact the same.\n\nIf you're integrating Mem0 into an agent that's meant to run for more than a single session, the schema is the part worth designing deliberately — the SDK calls themselves are the easy part.", "url": "https://wpnews.pro/news/stop-stuffing-your-llm-agent-s-context-window-structured-memory-categories-with", "canonical_source": "https://dev.to/mukesh_13/stop-stuffing-your-llm-agents-context-window-structured-memory-categories-with-mem0-4l8i", "published_at": "2026-07-29 04:55:59+00:00", "updated_at": "2026-07-29 05:00:36.062636+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "developer-tools"], "entities": ["Mem0", "Linear"], "alternates": {"html": "https://wpnews.pro/news/stop-stuffing-your-llm-agent-s-context-window-structured-memory-categories-with", "markdown": "https://wpnews.pro/news/stop-stuffing-your-llm-agent-s-context-window-structured-memory-categories-with.md", "text": "https://wpnews.pro/news/stop-stuffing-your-llm-agent-s-context-window-structured-memory-categories-with.txt", "jsonld": "https://wpnews.pro/news/stop-stuffing-your-llm-agent-s-context-window-structured-memory-categories-with.jsonld"}}