# Searching your whole AI chat history without a vector database — approach, research, and results > Source: > Published: 2026-08-29 13:48:31+00:00 AI coding assistants (Claude Code, and similar) save every conversation as a transcript on disk. Over months that becomes a big, unsearchable pile: *"how did we fix that dead-letter-queue issue three weeks ago?"* is effectively unanswerable. The obvious answer is "embed everything into a vector DB" — but that adds a model, a store, a key, and a network dependency. This is a write-up of a **local, dependency-free** alternative that gets most of the benefit, the research it draws on, and measured results versus the naïve baseline. Assistant transcripts are large JSONL files (tens of MB each), keyed by opaque UUIDs, scattered across a projects directory. The content that matters — decisions, the exact command that fixed something — is buried under ~97% tool-call/metadata noise. The naïve fix, **"full fetch": hand whole transcripts to the model**, is slow (multi-MB prompts), expensive (cost scales with input), lossy (the model skims), and doesn't scale past one or two chats. 1. **Find** the right conversation(s) among dozens — including ones that used*different words* than the question. 2. **Compress** transcripts to signal without losing intricate detail (commands, IDs, decisions). 3. **Rank** by genuine relevance — not size or recency alone; handle chats that*drifted* into a topic. 4. **Answer** with citations and an honest "I don't know", not a hallucination. 5. Stay **fast, cheap, offline, and dependency-free** (must run anywhere a shell does). | Task | Technique | Research basis | |---|---|---| | Compress | A **compact extract** per chat — message text + tool-call*inputs* + head/tail of long results + reasoning — cached, ~100× smaller than raw. | Mem0 — *extract salient content, don't store raw* | | Rank | **IDF keyword** +**first-message topical boost** +**term-frequency topicality** (many mentions = "about it" — catches mid-chat drift) +**recency × importance** tie-breakers. | Generative Agents — *recency × importance × relevance* ; RAG —*retrieve then generate* | | Find (vocabulary gap) | **Query expansion instead of embeddings** : one cheap model call expands`dead-letter queue` →`dlq, redrive, sqs, reprocess` , then feed those into the local keyword ranker. | see §5 | | Answer | Retrieve top-K, round-robin turn-selection into a small budget, **one** model call with strict*cite-or-say-"CANNOT ANSWER"* instructions. | MemGPT — *page in only what's needed* ; RAG | **The pipeline** (steps 1–4 are local/offline/instant; only step 5 calls a model): tokenize the question → build the corpus (skip throwaway sessions) → rank locally → extract + select the most relevant turns into a small budget → one model call that answers with citations. A real run (identifiers lightly redacted). Note the query never used the words "dlq" or "redrive" — the synonym expansion bridged the gap and surfaced the right chats, and the model cited each source and was honest about what the excerpts did *not* contain: ``` bash $ ccask "how did we fix the dead-letter queue backup?" expanding query (synonyms)… Searching 23 chats for: fix, dead, letter, queue, backup (+ synonyms: dlq, redrive, sqs, reprocess, poison, retry, unprocessed, lambda)… Most relevant: Payments Backend, Data Pipeline Ops, Release Runbook, Onboarding, Newsletter The dead-letter queue was cleared by adding the missing routing-map entry, then redriving: • "77 of 96 messages cleared after adding the entry" (from "Payments Backend"). • Root cause was a mapping gap — the fix was to add the missing entry, then redrive the DLQ (from "Data Pipeline Ops"). CANNOT ANSWER (fully): the excerpts give the outcome (add mapping entry → redrive → 77/96 cleared) but not the exact commands, or what happened to the remaining 19 messages. ``` Everything above the answer is local and instant (the `expanding query` line is one small call; the `Searching…`/` Most relevant` lines are `grep`); only the final answer is a second model call over the ~5 most-relevant chats' excerpts. *Observed on a real ~23-conversation history on one machine; illustrative, not a formal benchmark.* | Metric | Full fetch (raw transcript → model) | This approach (retrieve → one call) | |---|---|---| | Input per chat | **~6.5 MB** raw JSONL | **~63 KB** compact extract (**~100× smaller** ) | | Chats considered | 1–2 (doesn't scale) | **all ~23** , ranked; top ~5 read | | Ranking cost | n/a (you pick manually) | **local `grep`, milliseconds** — no model | | Latency (search across chats) | **~200–280 s** pointed at raw transcripts | **tens of seconds** (one small call); repeats reuse cache | | Single-chat summary (large chat) | minutes | **~34 s** , bounded; instant when cached | | Vocabulary-mismatch recall | none (must reuse exact words) | **yes** (synonym expansion) | | Answer honesty | skims, can hallucinate | quotes exact IDs/commands, or says `CANNOT ANSWER:` | | Dependencies | — | **none** beyond a shell + the assistant CLI (optional`python3` ) | **Consistent with the published Mem0 result** on the LOCOMO long-conversation benchmark — structured retrieval vs. full-context gave **~90% token savings, ~91% lower p95 latency, and +26% answer quality** ([2504.19413](https://arxiv.org/abs/2504.19413)). Same shape here: far less input, far lower latency, and *better* (specific, cited) answers than dumping raw transcripts. The textbook way to close the vocabulary gap (`dead-letter queue` vs `DLQ redrive`) is embeddings + cosine similarity. Two realizations changed the approach: 1. **Cosine similarity is trivial to write — the hard part is the vectors.** Cosine is ~8 lines of code. But*semantic* vectors (where synonyms land close) require a trained embedding model; lexical/TF-IDF vectors you can build yourself carry the same vocabulary-mismatch blind spot as keyword search. (And some providers — e.g. Anthropic — have no native embeddings API, so you'd add a third-party model, a key, and network calls.) 2. **You can borrow the model's semantic knowledge without vectors** — one query-expansion call turns the question into the words the transcript likely used, then the free local ranker does the rest. Net: meaning-based recall with **no vector store, no embedding model, no extra dependency**. True embeddings and hierarchical **RAPTOR** ([2401.18059](https://arxiv.org/abs/2401.18059)) summaries remain a sensible upgrade for *much* larger histories. - **MemGPT: LLMs as Operating Systems** — Packer et al. 2023 —[2310.08560](https://arxiv.org/abs/2310.08560) - **Generative Agents: Interactive Simulacra of Human Behavior** — Park et al. 2023 —[2304.03442](https://arxiv.org/abs/2304.03442) - **Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory** — Chhikara et al. 2025 —[2504.19413](https://arxiv.org/abs/2504.19413) - **RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval** — Sarthi et al. 2024 —[2401.18059](https://arxiv.org/abs/2401.18059) - **Retrieval-Augmented Generation (RAG)** — Lewis et al. 2020 —[2005.11401](https://arxiv.org/abs/2005.11401) - **A Survey on the Memory Mechanism of LLM-based Agents** — Zhang et al. 2024 —[2404.13501](https://arxiv.org/abs/2404.13501) ·[tracking repo](https://github.com/nuster1128/LLM_Agent_Memory_Survey) This approach ships as the `ccask` command in **Claudius**, a shell toolkit for working with your Claude Code history by name — [https://github.com/SaxenaKartik/claudius](https://github.com/SaxenaKartik/claudius) ``` ccask "how did we fix the DLQ redrive?" # search ALL your chats (default), answered + cited ccask -c "Backend Changes" "what did we decide?" # scope to a specific chat ccask -c # pick chat(s) from a menu, then type the question ``` Tunables: `CCASK_TOPK` (chats read, default 5), `CCASK_EXPAND` (0 disables synonym expansion), `CCASK_W_RECENCY` / `CCASK_W_IMPORTANCE` / `CCASK_W_FREQ` / `CCASK_RECENCY_DAYS` (ranking weights). Everything runs locally except the single answering call.