{"slug": "same-agent-memory-accuracy-with-8x-fewer-tokens-by-changing-the-embedding-model", "title": "Same agent-memory accuracy with 8x fewer tokens by changing the embedding model", "summary": "Turnbook 0.10, a local agent-memory tool from the goyohan0611-png GitHub project, scored 91.8% on all 500 LongMemEval_S questions while cutting the text the answer model reads per question from 26.3k tokens to 3.1k tokens, an 8x reduction achieved by changing the embedding model. Turnbook stores every conversation turn word for word in a single SQLite file with zero model tokens spent saving memory, versus agentmemory's ~16.5k tokens per question and 80.4% score under the same gpt-5-mini answer model and official gpt-4o judge. The tool connects to Claude Code, Codex, Cursor, Copilot CLI, Gemini CLI and OpenCode through MCP, and with its 0.9.47 search model still places the answer among the first 30 results for 79.7% of questions in a 205,000-turn memory (82.8% at 6,000 turns), with searches taking about 0.4 seconds.", "body_md": "### Your AI forgets everything when the session ends. This one remembers, and it never calls an LLM to do it.\n\n**91.8% on LongMemEval from 3.1k tokens of memory per question ([0.10.0](#does-it-work)). Zero model tokens spent saving memory. Runs on your own PC.**\n\n*Decide something with one agent on Monday. Ask another on Wednesday. It knows.*\n\n|  | tokens spent **saving** memory | text the answer model reads per question | LongMemEval | \n|---|---|---|---|\n| **Turnbook 0.10** | **0** | **3.1k tokens** | **91.8%** | \n| Turnbook, previous search model | 0 | 26.3k tokens | 91.8% | \n| agentmemory | — | ~16.5k tokens | 80.4% | \n\n<sub>All 500 LongMemEval_S questions (agentmemory: 499), same answer model (gpt-5-mini), prompt and official gpt-4o judge; only the memory differs. Tools like claude-mem and Mem0 run an LLM over your conversations to save them; Turnbook stores them word for word. [Details](#does-it-work)</sub>\n\n**English** · [한국어](https://github.com/goyohan0611-png/turnbook/blob/main/README.ko.md)\n\n<sub>Recorded live on one PC: Claude Code (Opus 5.5, effort high) and Codex (GPT-6.1-Sol, effort high) sharing one Turnbook memory. Model thinking time is fast-forwarded.</sub>\n\nWindows: download [Turnbook-Setup.cmd](https://raw.githubusercontent.com/goyohan0611-png/turnbook/main/scripts/Turnbook-Setup.cmd)\nand double-click it, or in PowerShell:\n\n```\nirm https://raw.githubusercontent.com/goyohan0611-png/turnbook/main/scripts/install.ps1 | iex\n```\n\nmacOS / Linux:\n\n```\ncurl -LsSf https://raw.githubusercontent.com/goyohan0611-png/turnbook/main/scripts/install.sh | sh\n```\n\nNo Python needed; nothing else to set up. Restart your AI apps afterwards. Already have Python?\n`pip install turnbook && turnbook-install`. More: [docs/install.md](https://github.com/goyohan0611-png/turnbook/blob/main/docs/install.md).\n\n## Connecting an MCP client by hand\n\nAfter `pip install turnbook`:\n\n```\n{\n  \"mcpServers\": {\n    \"turnbook\": {\n      \"command\": \"python\",\n      \"args\": [\"-m\", \"graph_mind.mcp_server\"]\n    }\n  }\n}\n```\n\nClaude Code: `claude mcp add turnbook -- python -m graph_mind.mcp_server`. This gives the\ntools only; `turnbook-install` also saves every conversation automatically.\n\nWork as usual for a while, then open a new session, another app, or another model and ask:\n\n- *\"Where did we leave off yesterday?\"*\n- *\"Which database did we pick for the billing service, and why?\"*\n- *\"What did you change in the auth middleware last week?\"*\n- *\"What's my dog's name?\"* (any fact you mentioned once, in passing)\n\nNothing to tag or save by hand: every turn is already stored, and the agent pulls back the few that answer the question.\n\n- **Every agent, one memory.** Claude Code, Codex, Cursor, Copilot CLI, Gemini CLI, OpenCode and\nany MCP app read and write the same memory. Decide with one, ask another; switch models, the\nmemory stays.\n- **The right memory in every message.** In Claude Code, Codex, Copilot CLI, Gemini CLI and\nOpenCode, Turnbook looks up what your message needs from the past and adds it before the model\nanswers. Nothing to ask for.\n- **Free to save, on your PC.** Every turn is stored word for word in one SQLite file: no model\ncall, no tokens, no account, no cloud.\n\n## Everything else\n\n- **Made for orchestration.** When an orchestrator runs several agents at once, they all write\nto the same memory as they work, so what one agent decided is there for the next. Every\nmemory keeps which app and session it came from.\n- **Holds up as memory piles up.** With 0.9.47's search model, in a memory of 205,000 turns the\nanswer is still among the first 30 found for 79.7% of questions (82.8% at 6,000), and a search takes about 0.4 s.[As the memory grows](#as-the-memory-grows)\n- **Any language, across languages.** The search model reads Korean almost as well as English:\nask in Korean about something said in English, or the other way round.[In Korean](#in-korean)\n- **Your project's documents too.**`turnbook docs add FOLDER` keeps its Markdown and text files\nsearchable beside the conversations, section by section, and up to date as they change.\n- **One memory across your PCs.** Run`turnbook folder share` , then`turnbook folder <code>` on\nthe other PC (the AI tells you the command; it never moves your memory by itself, so a web page\ncannot either). Office network or[Tailscale](https://tailscale.com) , over TLS, every line\nencrypted before it leaves a PC.\n- **Hand a memory to someone else as a file.**`turnbook export FILE` writes all of it;`--project FOLDER` writes only the conversations held in that folder. They run`turnbook import FILE` .\n- **Yours.** No account, no cloud of its own, no telemetry. Secrets are masked before anything is\nstored, and*\"delete the password I typed earlier\"* deletes it everywhere.[docs/privacy.md](https://github.com/goyohan0611-png/turnbook/blob/main/docs/privacy.md)\n- **Teams, if you want them.** Share a project's decisions and progress with teammates' AIs, never\nyour personal memory; files stay in Slack and GitHub.[docs/teams.md](https://github.com/goyohan0611-png/turnbook/blob/main/docs/teams.md)\n\nThe installer finds these and connects them; run it again after installing a new one.\n\n| app | conversations saved | how | \n|---|---|---|\n| Claude Code | every turn | reads its transcripts; a hook adds relevant memory to each message | \n| Codex CLI | every turn | reads its transcripts; a hook adds relevant memory to each message (trust it once in `/hooks` ) | \n| Cursor | every turn | reads its transcripts | \n| Copilot CLI | every turn | reads its transcripts; a hook adds relevant memory to each message | \n| Claude Desktop (Cowork) | every turn | reads its session files | \n| Gemini CLI | every turn | hooks; also adds relevant memory before each answer | \n| OpenCode | every turn | plugin; also adds relevant memory before each answer, for any model | \n| Claude Desktop chat, Windsurf, VS Code Copilot, Cline, LM Studio | what the model saves | MCP tools | \n\nAny other MCP client works through the tools: see the hand-written config above.\n\n```\nflowchart LR\n    A[\"Claude Code · Codex · Cursor<br/>Gemini CLI · OpenCode · ...\"] -- \"transcripts, hooks, plugin\" --> C[\"capture service<br/>(runs in the background)\"]\n    C -- \"verbatim turns,<br/>secrets masked\" --> S[(\"your memory<br/>on this PC\")]\n    S --> I[\"local search<br/>words + meaning + time\"]\n    A -- \"a question that needs the past\" --> I\n    I -- \"the few turns that answer it<br/>(a few thousand tokens)\" --> A\n```\n\n- **Saving costs nothing.** Turns are stored as they were said. No model summarises or extracts\nanything at write time, so nothing is lost or invented on the way in, and no tokens are spent.\n- **Search runs on your PC.** Word search, a local embedding model (Google's EmbeddingGemma,\n4-bit; ONNX, no GPU) and when things were said are combined, then ordered by a small learned\nscore; the agent gets a short packet with the source turns, not a dump of history.\n- **The model you already use does the thinking.** Turnbook hands it evidence; it answers.\nWhen models get better, the answers get better without changing the memory.\n\nUnderneath, Turnbook is RAG over chat history with everything before the answer built in. A typical RAG stack is assembled from parts; Turnbook runs every step up to the answer on its own and leaves only the answer to the model you already use:\n\n| Step | Typical RAG stack | Turnbook | \n|---|---|---|\n| Collect | an ingestion script you write | every turn saved automatically from your agents | \n| Chunk | splitting rules to tune | one turn per entry, word for word | \n| Embed | an embedding API, paid per token | a local model, free | \n| Store | a vector database to run | one SQLite file | \n| Retrieve | search code to write | words, meaning and time, re-ranked by a small learned score | \n| Hand over | prompt-assembly code | a bounded evidence packet over MCP | \n| Answer | your LLM | your LLM (Claude, Codex, ...) | \n\nThe same engine could serve a chatbot that remembers each of its users, a support bot that recalls past tickets, or a team's searchable decision log. Today it ships as a personal, local memory; a Python API and multi-user server mode are not built yet. Interested? Open an issue.\n\n[LongMemEval](https://github.com/xiaowu0162/LongMemEval) is the public benchmark for long-term\nchat memory: 500 questions over about 50 past sessions each. Every system below got the same\nanswer model (gpt-5-mini), the same prompt and the official gpt-4o judge; only the memory differs.\n\n|  | accuracy | model tokens spent **saving** memory | text the answer model reads | \n|---|---|---|---|\n| **Turnbook 0.10.0** | **91.8%** (all 500) | **0** | **3.1k tokens** | \n| Turnbook 0.9.9 | 88.8% (all 500) | 0 | 3.4k tokens | \n| agentmemory | 80.4% (same 499) | — | ~16.5k tokens | \n\nagentmemory was measured against 0.9.9, which scored 89.0% on agentmemory's 499. Retrieval alone, on agentmemory's own measure (an answer session among the first 5 retrieved): Turnbook 0.9.9 98.5%, agentmemory 95.2%. A question the answer model leaves unanswered (it spends its whole output limit thinking) counts as wrong, as in every Turnbook score here.\n\n0.10.0 changes the search model for new installs from mpnet to EmbeddingGemma. The same 500 questions, the same reader and judge; only the search model and the packet size differ:\n\n| search model | text the answer model reads (median) | accuracy | \n|---|---|---|\n| mpnet | 3.4k tokens | 86.6% | \n| mpnet | 6.7k tokens | 89.6% | \n| mpnet | 13.2k tokens | 90.4% | \n| mpnet | 26.3k tokens | 91.8% | \n| **EmbeddingGemma** | **3.1k tokens** | **91.8%** | \n\nAgainst mpnet at the same size, EmbeddingGemma answers 33 questions mpnet misses and misses 7 it\nanswers (p = 0.00004). Questions that need several past sessions gain most: 73.6% → 88.4%. The\nre-ranker weights for each model were trained on LoCoMo and LongMemEval together. These packets\nheld up to 15,000 characters; since 0.10.0 `brain_context` sends up to 30,000 by default\n(`max_chars` sets it). Its retrieval on LoCoMo, scored on conversations its weights never saw\n(answer turn among the first 10 found): 83.7%, mpnet 70.2%.\n[research/runs/graph-v9.14-full500-mpnet8-gemma](https://github.com/goyohan0611-png/turnbook/blob/main/research/runs/graph-v9.14-full500-mpnet8-gemma)\n\nThe charts below were measured with 0.9.47 and its search model (mpnet). With 0.10's model, at 6,000 turns the answer turn is among the first 30 found for 92.5% of questions (0.9.47: 82.8%); the larger memories are still being measured.\n\nRecall gets harder as memory piles up: more past conversations look like the one you mean. Since 0.9.46 Turnbook orders what it finds with a small learned score instead of a fixed blend. On LoCoMo (1,536 questions), with each person's memory grown to 10× by storing other conversations beside it, the share of questions whose answer reaches the model fell 5.6 points before (82.5% → 76.9%) and 3.1 points now (83.5% → 80.4%). Each conversation was scored with weights learned on the other nine; the added conversations are other people's, likely easier to tell apart than one person's own history. On LongMemEval the same change took the answer reaching the model from 97.5% to 98.3%.\n\n0.9.47 adds more to that score (how rare the shared words are, how much of one past conversation was found) and looks at 100 candidates per search instead of 50. With LoCoMo inside a memory grown to 205,000 turns, the answer turn is among the first 30 found for 79.7% of questions, against 73.0% with the fixed blend of 0.9.45; a search takes about 0.4 s there.\n\nFour LoCoMo conversations translated into Korean (582 questions). Search by meaning alone, share of questions whose answer turn is among the first 30 found:\n\n| search model | English | Korean | Korean question, English memory | \n|---|---|---|---|\n| mpnet | 75.6% | 67.5% | 70.6% | \n| **EmbeddingGemma** (0.10.0 default) | **90.2%** | **86.9%** | **87.8%** | \n\nWith all of search (words, meaning, re-ranker) on mpnet, 0.9.47 found the answer turn among the\nfirst 30 for 77.5% of Korean questions (84.7% in English); word search finds \"콩이\" in \"콩이를\"\n(Korean particles). [research/runs/graph-v9.21-korean](https://github.com/goyohan0611-png/turnbook/blob/main/research/runs/graph-v9.21-korean)\n\nEvery question, answer and verdict is in [`research/runs/`](https://github.com/goyohan0611-png/turnbook/blob/main/research/runs); method, failures and\nhow to rerun it: [docs/benchmarks.md](https://github.com/goyohan0611-png/turnbook/blob/main/docs/benchmarks.md).\n\n**Will it slow my PC down?** The background service uses about 2% of one CPU core when nothing new\nis coming in. An app's memory server holds about 525 MB with EmbeddingGemma (772 MB with mpnet).\nA memory search takes about 40 ms (46 ms at the 95th percentile on a GitHub runner; mpnet 26 ms);\nthe first one after an app starts, about 1.4 s. Each new turn takes about 0.1 s to embed; the\nservice runs below normal CPU priority and embeds backlogs at half pace. On a first install, the\nconversations already on the PC are read in the background while word search answers at once.\n\n**Which search model?** [EmbeddingGemma](https://huggingface.co/google/embeddinggemma-300m)\n(about 200 MB, downloaded from Hugging Face on first use). A memory from 0.9.47 or before keeps\nsearching with its old model (mpnet) while the background service reads it again with\nEmbeddingGemma, a few pieces at a time when nothing else is due; then it moves over by itself.\n`turnbook model` shows which model is in use; `turnbook model multilingual` keeps mpnet for good.\nEmbeddingGemma is provided under the\n[Gemma Terms of Use](https://ai.google.dev/gemma/terms); it is not part of Turnbook's own code\nor license.\n\n**What does it cost?** Nothing. No API key and no model calls of its own; your AI app reads a\npacket of a few thousand tokens when a question needs the past.\n\n**Does anything leave my PC?** No, unless you share the memory with your own other PCs or a team.\nNo telemetry. Secrets (API keys, passwords, card numbers...) are masked before storing.\n\n**How is this different from CLAUDE.md or an app's built-in memory?** Those keep notes a model\nwrote, one app at a time. Turnbook keeps every turn as it was said and shares it across every\napp and model on the PC.\n\n**Korean?** Yes: Korean and English conversations, and questions in either language. With the\ndefault model, search by meaning trails English by about 3 points in Korean, and a Korean\nquestion finds an English memory about as often: [In Korean](#in-korean).\n\n**How do I remove it?** `turnbook-uninstall` disconnects every app and stops the service; the\nmemory stays in `~/.graph-mind` unless you add `--purge`.\n\nNote\n\n**Beta.** Used daily on Windows. macOS and Linux pass every test in CI but have not seen daily\nuse yet.\n\n[MCP tools](https://github.com/goyohan0611-png/turnbook/blob/main/docs/install.md#mcp-tools) · [See your memory in 3D](https://github.com/goyohan0611-png/turnbook/blob/main/docs/privacy.md#seeing-your-memory)\n(`turnbook-view`) · [Changelog](https://github.com/goyohan0611-png/turnbook/blob/main/CHANGELOG.md) · [Security](https://github.com/goyohan0611-png/turnbook/blob/main/SECURITY.md) ·\nDevelopment: `pip install -r requirements.txt`, then `python -m unittest discover -s tests`\n\nContributions are welcome. License: [Apache-2.0](https://github.com/goyohan0611-png/turnbook/blob/main/LICENSE).", "url": "https://wpnews.pro/news/same-agent-memory-accuracy-with-8x-fewer-tokens-by-changing-the-embedding-model", "canonical_source": "https://github.com/goyohan0611-png/turnbook", "published_at": "2026-10-30 04:06:35+00:00", "updated_at": "2026-10-30 04:16:47.290160+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "agent-protocols", "artificial-intelligence"], "entities": ["Turnbook", "LongMemEval", "agentmemory", "Claude Code", "Codex", "Cursor", "Copilot CLI", "Gemini CLI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/same-agent-memory-accuracy-with-8x-fewer-tokens-by-changing-the-embedding-model", "markdown": "https://wpnews.pro/news/same-agent-memory-accuracy-with-8x-fewer-tokens-by-changing-the-embedding-model.md", "text": "https://wpnews.pro/news/same-agent-memory-accuracy-with-8x-fewer-tokens-by-changing-the-embedding-model.txt", "jsonld": "https://wpnews.pro/news/same-agent-memory-accuracy-with-8x-fewer-tokens-by-changing-the-embedding-model.jsonld"}}