Searching your whole AI chat history without a vector database — approach, research, and results

A developer built a local, dependency-free system for searching AI coding-assistant chat transcripts without a vector database, using compact per-chat extracts, IDF keyword ranking with first-message and term-frequency topical boosts, and model-driven query expansion to bridge vocabulary gaps. In a test run, the tool surfaced the correct conversations about a dead-letter-queue fix and returned a cited answer that explicitly flagged what the excerpts did not contain, with only the query expansion and final answer requiring model calls.

AI coding assistants Claude Code, and similar save every conversation as a transcript on disk. Over months that becomes a big, unsearchable pile: "how did we fix that dead-letter-queue issue three weeks ago?" is effectively unanswerable. The obvious answer is "embed everything into a vector DB" — but that adds a model, a store, a key, and a network dependency. This is a write-up of a local, dependency-free alternative that gets most of the benefit, the research it draws on, and measured results versus the naïve baseline. Assistant transcripts are large JSONL files tens of MB each , keyed by opaque UUIDs, scattered across a projects directory. The content that matters — decisions, the exact command that fixed something — is buried under ~97% tool-call/metadata noise. The naïve fix, "full fetch": hand whole transcripts to the model , is slow multi-MB prompts , expensive cost scales with input , lossy the model skims , and doesn't scale past one or two chats. 1. Find the right conversation s among dozens — including ones that used different words than the question. 2. Compress transcripts to signal without losing intricate detail commands, IDs, decisions . 3. Rank by genuine relevance — not size or recency alone; handle chats that drifted into a topic. 4. Answer with citations and an honest "I don't know", not a hallucination. 5. Stay fast, cheap, offline, and dependency-free must run anywhere a shell does . | Task | Technique | Research basis | |---|---|---| | Compress | A compact extract per chat — message text + tool-call inputs + head/tail of long results + reasoning — cached, ~100× smaller than raw. | Mem0 — extract salient content, don't store raw | | Rank | IDF keyword + first-message topical boost + term-frequency topicality many mentions = "about it" — catches mid-chat drift + recency × importance tie-breakers. | Generative Agents — recency × importance × relevance ; RAG — retrieve then generate | | Find vocabulary gap | Query expansion instead of embeddings : one cheap model call expands dead-letter queue → dlq, redrive, sqs, reprocess , then feed those into the local keyword ranker. | see §5 | | Answer | Retrieve top-K, round-robin turn-selection into a small budget, one model call with strict cite-or-say-"CANNOT ANSWER" instructions. | MemGPT — page in only what's needed ; RAG | The pipeline steps 1–4 are local/offline/instant; only step 5 calls a model : tokenize the question → build the corpus skip throwaway sessions → rank locally → extract + select the most relevant turns into a small budget → one model call that answers with citations. A real run identifiers lightly redacted . Note the query never used the words "dlq" or "redrive" — the synonym expansion bridged the gap and surfaced the right chats, and the model cited each source and was honest about what the excerpts did not contain: bash $ ccask "how did we fix the dead-letter queue backup?" expanding query synonyms … Searching 23 chats for: fix, dead, letter, queue, backup + synonyms: dlq, redrive, sqs, reprocess, poison, retry, unprocessed, lambda … Most relevant: Payments Backend, Data Pipeline Ops, Release Runbook, Onboarding, Newsletter The dead-letter queue was cleared by adding the missing routing-map entry, then redriving: • "77 of 96 messages cleared after adding the <mapping entry" from "Payments Backend" . • Root cause was a mapping gap — the fix was to add the missing entry, then redrive the DLQ from "Data Pipeline Ops" . CANNOT ANSWER fully : the excerpts give the outcome add mapping entry → redrive → 77/96 cleared but not the exact commands, or what happened to the remaining 19 messages. Everything above the answer is local and instant the expanding query line is one small call; the Searching… / Most relevant lines are grep ; only the final answer is a second model call over the ~5 most-relevant chats' excerpts. Observed on a real ~23-conversation history on one machine; illustrative, not a formal benchmark. | Metric | Full fetch raw transcript → model | This approach retrieve → one call | |---|---|---| | Input per chat | ~6.5 MB raw JSONL | ~63 KB compact extract ~100× smaller | | Chats considered | 1–2 doesn't scale | all ~23 , ranked; top ~5 read | | Ranking cost | n/a you pick manually | local grep , milliseconds — no model | | Latency search across chats | ~200–280 s pointed at raw transcripts | tens of seconds one small call ; repeats reuse cache | | Single-chat summary large chat | minutes | ~34 s , bounded; instant when cached | | Vocabulary-mismatch recall | none must reuse exact words | yes synonym expansion | | Answer honesty | skims, can hallucinate | quotes exact IDs/commands, or says CANNOT ANSWER: | | Dependencies | — | none beyond a shell + the assistant CLI optional python3 | Consistent with the published Mem0 result on the LOCOMO long-conversation benchmark — structured retrieval vs. full-context gave ~90% token savings, ~91% lower p95 latency, and +26% answer quality 2504.19413 https://arxiv.org/abs/2504.19413 . Same shape here: far less input, far lower latency, and better specific, cited answers than dumping raw transcripts. The textbook way to close the vocabulary gap dead-letter queue vs DLQ redrive is embeddings + cosine similarity. Two realizations changed the approach: 1. Cosine similarity is trivial to write — the hard part is the vectors. Cosine is ~8 lines of code. But semantic vectors where synonyms land close require a trained embedding model; lexical/TF-IDF vectors you can build yourself carry the same vocabulary-mismatch blind spot as keyword search. And some providers — e.g. Anthropic — have no native embeddings API, so you'd add a third-party model, a key, and network calls. 2. You can borrow the model's semantic knowledge without vectors — one query-expansion call turns the question into the words the transcript likely used, then the free local ranker does the rest. Net: meaning-based recall with no vector store, no embedding model, no extra dependency . True embeddings and hierarchical RAPTOR 2401.18059 https://arxiv.org/abs/2401.18059 summaries remain a sensible upgrade for much larger histories. - MemGPT: LLMs as Operating Systems — Packer et al. 2023 — 2310.08560 https://arxiv.org/abs/2310.08560 - Generative Agents: Interactive Simulacra of Human Behavior — Park et al. 2023 — 2304.03442 https://arxiv.org/abs/2304.03442 - Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory — Chhikara et al. 2025 — 2504.19413 https://arxiv.org/abs/2504.19413 - RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval — Sarthi et al. 2024 — 2401.18059 https://arxiv.org/abs/2401.18059 - Retrieval-Augmented Generation RAG — Lewis et al. 2020 — 2005.11401 https://arxiv.org/abs/2005.11401 - A Survey on the Memory Mechanism of LLM-based Agents — Zhang et al. 2024 — 2404.13501 https://arxiv.org/abs/2404.13501 · tracking repo https://github.com/nuster1128/LLM Agent Memory Survey This approach ships as the ccask command in Claudius , a shell toolkit for working with your Claude Code history by name — https://github.com/SaxenaKartik/claudius https://github.com/SaxenaKartik/claudius ccask "how did we fix the DLQ redrive?" search ALL your chats default , answered + cited ccask -c "Backend Changes" "what did we decide?" scope to a specific chat ccask -c pick chat s from a menu, then type the question Tunables: CCASK TOPK chats read, default 5 , CCASK EXPAND 0 disables synonym expansion , CCASK W RECENCY / CCASK W IMPORTANCE / CCASK W FREQ / CCASK RECENCY DAYS ranking weights . Everything runs locally except the single answering call.