Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents

Researchers replaced full-corpus injection in a legal-document analysis system with two structured retrieval modes, achieving comparable accuracy while reducing token usage by up to 29.9x and costs by 25%. The study found that cached injection is only cheaper when the corpus is less than ten times the retrieval payload.

arXiv:2607.05764v1 Announce Type: new Abstract: Answering questions over a set of transactional legal documents is most simply done by injecting the whole corpus into the LLM's context window on every query. That baseline maximises retrieval recall, but its token footprint scales with the corpus rather than the question, and long-context degradation scales with it. We report what it took to replace full-corpus injection in a legal-document analysis system, comparing it against two structured retrieval modes over our proprietary structure-aware chunking: embedding retrieval NAVEMBED and LLM navigation over a compact structured index NAVINDEX . On a 20-question benchmark with verified ground-truth answers, a position-bias-controlled, reference-anchored pairwise judge scored semantic retrieval with reranking tied with injection on 16 of 18 document-bound questions injection preferred on 2 while attending to 17.3x fewer input tokens a general-text-embedding GTE configuration reaches 29.9x at a lower tie rate ; both modes were judged tied on the 2 out-of-scope controls. NAVINDEX was judged tied on all 18 at a 1.61x smaller total token footprint, a ~56x smaller answering context, and 25% lower dollar cost. We derive a closed-form caching-crossover rule: cached injection is cheaper in dollars only while the corpus stays below roughly ten times the retrieval payload. Scope and uncertainty are quantified in Section 8.