# Stop Paying For Retrieval Latency On Chunks You Never Use In The Prompt > Source: > Published: 2026-06-16 08:12:31+00:00 You set `TOP_K=10` on your vector store. Ten candidate chunks means more signal for the model — that's the logic. Then you run `npx ragscope` and the audit prints: ``` WARN 51/100 █████░░░░░ my-rag-service │ "what are the pricing tiers?" │ │ ✗ precision 30 ███░░░░░░░ 3/10 chunks used │ ✗ efficiency 30 ███░░░░░░░ 70% tokens wasted │ ✓ uniqueness 100 ██████████ chunks are distinct │ ✓ coverage 100 ██████████ all chunks scored │ │ → Reduce TOP_K 10→3 (only 3 chunks reached LLM) │ → 70% of retrieved tokens never reached the LLM │ ``` Three of ten chunks made it into the final prompt. You paid for ten round-trips to the vector store and seven went nowhere. The other seven were fetched, scored, and discarded somewhere between your retrieval step and your prompt assembly code. This is the gap [ragscope](https://github.com/Sidd27/ragscope) ([npm](https://www.npmjs.com/package/ragscope)) was built to surface. Retrieval and prompt assembly are separate steps, but most teams treat them as one. The gap is where the waste lives. A typical RAG pipeline: `TOP_K` chunks from the vector storeStep 4 is where chunks disappear. Post-retrieval filtering — a score threshold, a hard token budget, a deduplication pass — silently drops chunks you already paid to retrieve. If your prompt assembly filters out anything below a certain confidence score and your vector store returns six chunks that don't clear it, those six fetches were wasted. The network round-trip still happened. The latency still accumulated. The problem compounds over time. `TOP_K=10` gets set as a safe default, the pipeline ships, and the setting never gets revisited. LLM eval scores look fine because the three chunks that do reach the prompt are the right ones. The waste is invisible in your evals — it only shows up in latency and cost. Vector stores typically scale retrieval latency with `TOP_K` . Fetching ten results takes measurably longer than fetching three, especially at tail latencies. When seven of those ten are discarded before the prompt, you're paying that latency premium on every query for nothing. `precision` ragscope runs as a local server (default port 4321) and receives spans from your pipeline via OTLP at `http://localhost:4321/v1/traces` . No changes to your RAG code needed — configure your OpenTelemetry exporter to point there. When both a retrieval span and an LLM span arrive for the same trace, ragscope compares them. The key field is `inContext` on each `RagChunk` . It inspects the full text of the LLM span's assembled prompt and checks whether each retrieved chunk's content appears in it — positionally, by string match. A chunk either appears in the prompt or it doesn't. No LLM, no heuristics, no sampling. `precision` starts as: ``` base = (chunks where inContext is true / total retrieved chunks) × 100 ``` There's one additional penalty: if high-retrieval-rank chunks land in the middle of a long context window — the zone where LLM attention typically falls off — ragscope subtracts 12 points per buried chunk, capped at 36. This is the lost-in-the-middle detection. A pipeline where 3/10 chunks are used but two of those three are buried mid-context will score lower than one where the same 3 chunks sit at the prompt edges. When the base falls below 60, ragscope generates a specific recommendation with the exact numbers: `Reduce TOP_K 10→3 (only 3 chunks reached LLM)` . It prints this directly below the score bars, in the terminal, at the time the trace arrives — not in a cloud dashboard after a deploy. `efficiency` is a related metric: the fraction of retrieved tokens that reached the LLM. If `precision` is 30 and your chunks are roughly uniform in size, `efficiency` will also be around 30 — meaning 70% of the tokens you transferred from the vector store never appeared in a prompt. That shows up in retrieval latency and, depending on your pipeline, in processing time downstream. Once you have `precision` data, tuning is mechanical. **Follow the recommendation.** ragscope prints `Reduce TOP_K 10→3` because `Math.max(used_chunks, 3)` gives you the minimum viable retrieval count with a floor of 3. Start there and re-run the audit against a few real queries. **Check efficiency alongside.** Low `efficiency` paired with low `precision` means your unused chunks are large — a chunking strategy problem as much as a `TOP_K` problem. If `efficiency` is high but `precision` is low, your unused chunks are small and the token cost is modest; fixing `TOP_K` will mostly recover the latency.**Check uniqueness too.** If the chunks that do reach your prompt include near-duplicates, the `uniqueness` metric will flag them. Two near-duplicate chunks in the prompt means one is redundant regardless of your `TOP_K` setting — that's a deduplication-at-ingest problem, not a retrieval count problem. ragscope computes overlap between chunk pairs and surfaces high-overlap counts in the audit output.The typical path after a low-precision audit: `3/10 chunks used` — follow the `Reduce TOP_K 10→3` recommendation`precision` should move above 75 (PASS)`uniqueness` also flagged duplicates, fix chunking and re-run`efficiency` is still low after fixing `TOP_K` , chunks may be too large for the token budgetGoing from `TOP_K=10` to `TOP_K=3` on a pipeline that was only ever using 3 chunks means 70% fewer vector store round-trips on every query. No model changes, no prompt rewriting, no reranker added. The assumption that more retrieval means better answers is almost never validated against what the model actually sees. `TOP_K` defaults get set once and forgotten. Eval scores stay flat because the chunks that do reach the LLM are the right ones — the waste doesn't affect quality, just cost and latency. `npx ragscope` gives you `precision` , `efficiency` , and `uniqueness` in the time it takes to run a dev command. If you haven't checked what fraction of your retrieved chunks survive to the prompt, that's the number to look at first. `precision` measures what fraction of retrieved chunks appear in the final LLM prompt — anything below 60/100 triggers a `Reduce TOP_K` recommendation with exact numbers`TOP_K` ; fetching chunks you discard is pure overhead with no quality benefit`efficiency` paired with low `precision` points to a chunking strategy problem, not just a `TOP_K` problem`uniqueness` means near-duplicate chunks in the prompt — fix at ingest, not by adjusting retrieval count