{"slug": "qeschunker-a-single-objective-unifies-overlapping-and-non-overlapping-chunking", "title": "QESChunker: A Single Objective Unifies Overlapping and Non-Overlapping Chunking for RAG", "summary": "Researchers introduced QESChunker, a method that builds static chunk indices for retrieval-augmented generation from question–evidence supervision rather than heuristics, formulating overlapping and non-overlapping chunking as a single utility-maximization objective. Under the non-overlap constraint the objective decomposes over chunks and is solved exactly in polynomial time via dynamic programming; when overlap is allowed the objective is monotone submodular, exact optimization is NP-hard, and a greedy approach achieves a (1-1/√e)-approximation. Across two self-constructed and three public datasets, both variants outperformed existing chunking methods on complete-evidence recall in nearly all evaluated settings under the same overlap regime and a fixed hybrid retriever.", "body_md": "arXiv:2610.36557v1 Announce Type: cross \nAbstract: Chunking is a fundamental preprocessing step in retrieval-augmented generation (RAG), yet most existing chunkers rely primarily on heuristics. We introduce the Question--Evidence-Supervised Chunker (QESChunker), to our knowledge the first method for constructing static chunk indices from question--evidence supervision. QESChunker constructs document-specific chunk indices by optimizing over human-annotated or LLM-generated question--evidence pairs, without access to held-out test queries. We formulate non-overlapping and overlapping index construction through a unified question--evidence utility-maximization objective, turning chunking into a supervised combinatorial optimization problem. Under the non-overlap constraint, the objective decomposes over chunks and can be optimized exactly in polynomial time by dynamic programming. When overlap is allowed, the objective becomes monotone submodular; exact optimization is NP-hard, but our greedy achieves a $(1-1/\\sqrt{e})$-approximation. Across two self-constructed and three public datasets, both variants outperform existing chunking methods in complete-evidence recall in nearly all evaluated settings, under the same overlap regime and a fixed hybrid retriever. These results recast chunking as supervised, budget-aware index construction for evidence retrieval.", "url": "https://wpnews.pro/news/qeschunker-a-single-objective-unifies-overlapping-and-non-overlapping-chunking", "canonical_source": "https://www.machinebrief.com/news/qeschunker-a-single-objective-unifies-overlapping-and-non-ov-76bt", "published_at": "2026-10-28 04:00:00+00:00", "updated_at": "2026-10-28 05:41:59.013583+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "natural-language-processing"], "entities": ["QESChunker", "retrieval-augmented generation", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/qeschunker-a-single-objective-unifies-overlapping-and-non-overlapping-chunking", "markdown": "https://wpnews.pro/news/qeschunker-a-single-objective-unifies-overlapping-and-non-overlapping-chunking.md", "text": "https://wpnews.pro/news/qeschunker-a-single-objective-unifies-overlapping-and-non-overlapping-chunking.txt", "jsonld": "https://wpnews.pro/news/qeschunker-a-single-objective-unifies-overlapping-and-non-overlapping-chunking.jsonld"}}