# Retrieval overlap went up 13 points by promoting sentences to paragraphs > Source: > Published: 2026-09-22 18:38:09+00:00 The whole trick fits in one sentence: retrieve at sentence level, then hand back the paragraph each hit came from. On a benchmark of legal contracts, that took my retrieval overlap from 37.3% to 50.4%. I used to write documentation for a living, and people still tell me I write like a spec. So this is my attempt to explain an experiment in plain words, with the numbers left attached. Fifteen contracts from the CUAD dataset, 242 questions with the correct answer spans marked by hand. Embeddings are `all-MiniLM-L6-v2`, 22 million parameters, running on CPU. FAISS does the vector search, top-5 by cosine similarity. No paid APIs anywhere in this. The baseline was [stela](https://github.com/watthem/stela) chunking each contract into paragraphs. That covered 37.3% of the ground truth answer characters across all queries. Retrieval overlap means: of the characters in the correct answers, this much of them appeared inside the chunks we returned. Here is the problem with paragraph chunks. A contract paragraph can run to hundreds of words, and the question points at one clause in it. The embedding for the whole paragraph is a blend of everything in it, so the match comes back fuzzy. A sentence embedding is a much cleaner signal for a specific question. But a bare sentence is a poor thing to feed an LLM. The answer to "what happens if the buyer defaults?" usually needs the surrounding conditions to be useful. So the pipeline does two passes: Step 3 sounds like it needs string matching, and it does not. stela records the byte offset where each chunk starts and ends while it splits the text, and verifies those offsets against a SHA-256 of the chunk before emitting anything. So containment is arithmetic: sentence `[s_start, s_end)` sits inside paragraph `[p_start, p_end)` when `p_start <= s_start` and `s_end <= p_end`. Four numbers, two comparisons. Overlapping paragraphs get deduplicated at 75% overlap, and the pipeline returns the top-5. One smaller fix rode along: CUAD questions open with about 110 characters of template text, and MiniLM only reads 256 tokens. Stripping that prefix from each query is the "strip boilerplate" row below. | Configuration | Retrieval Overlap | Precision@5 | Complete-Grounding@5 | |---|---|---|---| | stela paragraph (baseline) | 37.3% | 7.6% | 36.7% | | + strip boilerplate | 38.4% (+1.1pp) | 8.4% | 37.3% | | + multi-strategy promotion | **50.4% (+13.1pp)** | 6.5% | **50.1%** | | + hybrid BM25 (RRF) | 49.2% (-1.2pp) | 6.3% | 47.4% | A few contracts moved a lot. Todos Medical jumped from 39.1% to 84.8%. Xencor went from 19.5% to 47.5%. Goosehead Insurance from 13.0% to 36.0%. The trade-off shows up in the table too: Precision@5 dipped from 7.6% to 6.5%. Paragraphs carry context outside the annotated span, so we cover more of the right answer and a bit more of the neighborhood. For a RAG pipeline where the model has to reason about a clause, I take that trade. If you need tight extraction of exactly the marked span, stay at sentence level. I fused BM25 in with Reciprocal Rank Fusion, k=60, expecting the classic hybrid lift. Retrieval overlap dropped 1.2 points. My best guess at why: legal text is saturated with "agreement," "party," and "shall," so plain word matching has little to separate one clause from another, and fusing its rankings pulled in hits that shared words with the question but not meaning. The dense embeddings already had the useful signal, and BM25 diluted it. I used vanilla whitespace tokenization; a legal-domain tokenizer or a reranker might change the result. Fifteen contracts and 242 queries is a small sample. I only ran local FAISS, so Pinecone, other distance metrics, and approximate nearest neighbor settings could shift these numbers. A bigger embedding model (Qwen3-Embedding-0.6B, 600M params) loaded fine but was too slow on CPU for the full benchmark. It wants a GPU with CUDA compute capability 7.0 or higher. The pipeline scripts, per-document breakdowns, and metric definitions are in the repo: the [experiment README](https://github.com/watthem/stela/blob/main/docs/experiments/pinecone-integration/README.md) and the [optimization results](https://github.com/watthem/stela/blob/main/docs/experiments/pinecone-integration/optimization-results.md). The 15 CUAD contracts themselves aren't checked in yet, so a rerun isn't one command today. Sentences for matching, paragraphs for answering. That is the whole idea. [GitHub: watthem/stela](https://github.com/watthem/stela) (MIT)