AI coding assistants (Claude Code, and similar) save every conversation as a transcript on disk. Over months that becomes a big, unsearchable pile: "how did we fix that dead-letter-queue issue three weeks ago?" is effectively unanswerable. The obvious answer is "embed everything into a vector DB" — but that adds a model, a store, a key, and a network dependency. This is a write-up of a local, dependency-free alternative that gets most of the benefit, the research it draws on, and measured results versus the naïve baseline.
Assistant transcripts are large JSONL files (tens of MB each), keyed by opaque UUIDs, scattered across a projects directory. The content that matters — decisions, the exact command that fixed something — is buried under ~97% tool-call/metadata noise.
The naïve fix, "full fetch": hand whole transcripts to the model, is slow (multi-MB prompts), expensive (cost scales with input), lossy (the model skims), and doesn't scale past one or two chats.
- Find the right conversation(s) among dozens — including ones that useddifferent words than the question.
- Compress transcripts to signal without losing intricate detail (commands, IDs, decisions).
- Rank by genuine relevance — not size or recency alone; handle chats thatdrifted into a topic.
- Answer with citations and an honest "I don't know", not a hallucination.
- Stay fast, cheap, offline, and dependency-free (must run anywhere a shell does).
| Task | Technique | Research basis |
|---|---|---|
| Compress | A compact extract per chat — message text + tool-callinputs + head/tail of long results + reasoning — cached, ~100× smaller than raw. | Mem0 — extract salient content, don't store raw |
| Rank | IDF keyword +first-message topical boost +term-frequency topicality (many mentions = "about it" — catches mid-chat drift) +recency × importance tie-breakers. | Generative Agents — recency × importance × relevance ; RAG —retrieve then generate |
| Find (vocabulary gap) | Query expansion instead of embeddings : one cheap model call expandsdead-letter queue →dlq, redrive, sqs, reprocess , then feed those into the local keyword ranker. |
see §5 |
| Answer | Retrieve top-K, round-robin turn-selection into a small budget, one model call with strictcite-or-say-"CANNOT ANSWER" instructions. | MemGPT — page in only what's needed ; RAG |
The pipeline (steps 1–4 are local/offline/instant; only step 5 calls a model): tokenize the question → build the corpus (skip throwaway sessions) → rank locally → extract + select the most relevant turns into a small budget → one model call that answers with citations.
A real run (identifiers lightly redacted). Note the query never used the words "dlq" or "redrive" — the synonym expansion bridged the gap and surfaced the right chats, and the model cited each source and was honest about what the excerpts did not contain:
$ ccask "how did we fix the dead-letter queue backup?"
expanding query (synonyms)…
Searching 23 chats for: fix, dead, letter, queue, backup
(+ synonyms: dlq, redrive, sqs, reprocess, poison, retry, unprocessed, lambda)…
Most relevant: Payments Backend, Data Pipeline Ops, Release Runbook, Onboarding, Newsletter
The dead-letter queue was cleared by adding the missing routing-map entry, then redriving:
• "77 of 96 messages cleared after adding the <mapping> entry" (from "Payments Backend").
• Root cause was a mapping gap — the fix was to add the missing entry, then redrive the
DLQ (from "Data Pipeline Ops").
CANNOT ANSWER (fully): the excerpts give the outcome (add mapping entry → redrive → 77/96
cleared) but not the exact commands, or what happened to the remaining 19 messages.
Everything above the answer is local and instant (the expanding query line is one small call; the
Searching…/ Most relevant lines are grep); only the final answer is a second model call over the
~5 most-relevant chats' excerpts.
Observed on a real ~23-conversation history on one machine; illustrative, not a formal benchmark.
| Metric | Full fetch (raw transcript → model) | This approach (retrieve → one call) |
|---|---|---|
| Input per chat | ~6.5 MB raw JSONL | ~63 KB compact extract (~100× smaller ) |
| Chats considered | 1–2 (doesn't scale) | all ~23 , ranked; top ~5 read |
| Ranking cost | n/a (you pick manually) | local grep, milliseconds — no model |
| Latency (search across chats) | ~200–280 s pointed at raw transcripts | tens of seconds (one small call); repeats reuse cache |
| Single-chat summary (large chat) | minutes | ~34 s , bounded; instant when cached |
| Vocabulary-mismatch recall | none (must reuse exact words) | yes (synonym expansion) |
| Answer honesty | skims, can hallucinate | quotes exact IDs/commands, or says CANNOT ANSWER: |
| Dependencies | — | none beyond a shell + the assistant CLI (optionalpython3 ) |
Consistent with the published Mem0 result on the LOCOMO long-conversation benchmark — structured retrieval vs. full-context gave ~90% token savings, ~91% lower p95 latency, and +26% answer quality (2504.19413). Same shape here: far less input, far lower latency, and better (specific, cited) answers than dumping raw transcripts.
The textbook way to close the vocabulary gap (dead-letter queue vs DLQ redrive) is embeddings + cosine
similarity. Two realizations changed the approach:
- Cosine similarity is trivial to write — the hard part is the vectors. Cosine is ~8 lines of code. Butsemantic vectors (where synonyms land close) require a trained embedding model; lexical/TF-IDF vectors you can build yourself carry the same vocabulary-mismatch blind spot as keyword search. (And some providers — e.g. Anthropic — have no native embeddings API, so you'd add a third-party model, a key, and network calls.)
- You can borrow the model's semantic knowledge without vectors — one query-expansion call turns the question into the words the transcript likely used, then the free local ranker does the rest.
Net: meaning-based recall with no vector store, no embedding model, no extra dependency. True embeddings and hierarchical RAPTOR (2401.18059) summaries remain a sensible upgrade for much larger histories.
- MemGPT: LLMs as Operating Systems — Packer et al. 2023 —2310.08560
- Generative Agents: Interactive Simulacra of Human Behavior — Park et al. 2023 —2304.03442
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory — Chhikara et al. 2025 —2504.19413
- RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval — Sarthi et al. 2024 —2401.18059
- Retrieval-Augmented Generation (RAG) — Lewis et al. 2020 —2005.11401
- A Survey on the Memory Mechanism of LLM-based Agents — Zhang et al. 2024 —2404.13501 ·tracking repo
This approach ships as the ccask command in Claudius, a shell toolkit for working with your Claude
Code history by name — https://github.com/SaxenaKartik/claudius
ccask "how did we fix the DLQ redrive?" # search ALL your chats (default), answered + cited
ccask -c "Backend Changes" "what did we decide?" # scope to a specific chat
ccask -c # pick chat(s) from a menu, then type the question
Tunables: CCASK_TOPK (chats read, default 5), CCASK_EXPAND (0 disables synonym expansion),
CCASK_W_RECENCY / CCASK_W_IMPORTANCE / CCASK_W_FREQ / CCASK_RECENCY_DAYS (ranking weights).
Everything runs locally except the single answering call.