TokenMizer: Giving LLMs a Memory That Doesn't Forget Between Sessions

A developer has built TokenMizer, a graph-memory proxy that sits between applications and OpenAI-compatible APIs, giving LLMs persistent memory across sessions. Instead of storing flat chat logs, it extracts entities and relationships into a graph database, keeping retrieval fast as history grows. The tool includes a visual Graph Explorer for debugging and supports MCP for integration with editors like Cursor.

How a graph-memory proxy sits between your app and the OpenAI API, quietly remembering everything your LLM would otherwise lose. Every conversation with a large language model starts from zero. Close the tab, start a new session, and the model has no idea who you are, what you discussed yesterday, or what you decided last week. Most tools work around this by stuffing more and more chat history into the context window — which is expensive, slow, and eventually hits a hard limit. I built TokenMizer to solve this differently: instead of remembering by re-reading everything, it remembers by building a graph . TokenMizer sits as a proxy in front of any OpenAI-compatible API. Your application doesn't change how it calls the model — it just points its API base URL at TokenMizer instead of directly at OpenAI. Every request and response passes through, gets analyzed, and gets stored before continuing on to the real model. This design choice matters more than it looks. It means TokenMizer works with any framework or app that already speaks the OpenAI API format, with no SDK changes and no rewritten integration code. You add memory to an existing app by changing one URL. Two systems do the heavy lifting. File Intelligence watches what code, documents, or files are referenced during a conversation and builds context around them — so if you're debugging the same file across three sessions, TokenMizer already knows its history in the conversation, not just its current contents. Graph Memory is the more interesting part. Instead of storing conversation history as a flat log, TokenMizer extracts entities and relationships — people, projects, decisions, dependencies — and stores them as nodes and edges in a graph, backed by SQLite. When a new message comes in, TokenMizer doesn't search through old transcripts; it queries the graph for relevant nodes and pulls in only what's connected to the current topic. The practical difference: a flat-log memory system gets slower and more expensive as history grows, because it has to search more text. A graph memory system stays fast, because it's traversing relationships, not scanning transcripts. Memory systems that live entirely inside a database are hard to trust, because you can't see what they're doing. TokenMizer ships with a Graph Explorer built on D3.js — a visual, interactive map of every entity and connection the system has learned. You can watch new nodes appear as a conversation progresses, or trace why the model suddenly "remembered" something from three sessions ago by following the edge back to its source. This turned out to be more useful for debugging than for demos. When memory retrieval pulls in the wrong context, the graph view shows exactly why — which node matched, and through which relationship. TokenMizer is distributed as a pip-installable library, so it drops into an existing Python environment directly. For day-to-day use there's a CLI to inspect, query, and manage the memory graph without writing code. The second integration path is MCP Model Context Protocol server support, which lets TokenMizer's memory be used as a tool by MCP-compatible clients — including editors like Cursor. This was a deliberate design decision: memory shouldn't be locked to one app. Whether you're calling the API directly, scripting through the CLI, or working inside an AI-assisted editor, the same graph is behind all of it. Building a proxy that intercepts every API call raises the stakes on reliability — if TokenMizer fails, your app's LLM calls fail with it. An internal audit surfaced real problems worth naming honestly: All nine verified issues were fixed and tested. The installer bug in particular changed how I think about setup scripts generally: anything that touches a user's existing data on install needs to default to the safest possible behavior, even if that means asking one more confirmation question. Existing memory layers for LLMs mostly work — but they usually mean adopting their SDK and their storage model. TokenMizer's proxy-first design means it can sit under tools you already use without asking you to restructure how you call the model. The graph-based retrieval is also a deliberate bet: as conversations and codebases grow, relationship-based lookup scales in a way flat retrieval doesn't. It's early — this is a project built in the open, not a finished product — but the architecture is stable enough now to be worth explaining properly, and worth other developers trying against their own workflows. TokenMizer is open source and pip-installable. Code, CLI docs, and the Graph Explorer are on GitHub: Shweta-Mishra-ai/tokenmizer. I write about building developer tools and AI infrastructure at TechNova World. If you need someone who can build a system like this and explain it clearly to your team or your users, let's talk.