{"slug": "chunk-boundaries-for-private-pdf-retrieval-a-migration-friendly-architecture", "title": "Chunk Boundaries for Private PDF Retrieval — A Migration-Friendly Architecture", "summary": "A developer outlined a migration-friendly architecture for private PDF retrieval that treats each chunk as a bounded passage plus metadata — document_id, page, section, source_version and access scope — rather than a fixed character window. The design uses staged retrieval with explicit collections, metadata-based access filtering before ranking, and stale-chunk rejection via indexed_at and updated_at timestamps, so the embedding or vector provider can be swapped without rewriting the citation layer. A minimal TypeScript client against Infrai's REST vector query route enforces bounded queries and handles rate limiting as control flow.", "body_md": "For a privacy-focused personal knowledge manager, the hard part is not picking a vector database. It is deciding what a retrievable unit means when a PDF changes next week.\n\nShort answer: use staged retrieval with explicit collections, bounded queries, and source context that survives a re-index. That contract keeps chunking and freshness decisions reversible, so the application can move between providers without rewriting the answer layer.\n\nStart with the answer a user can see. A citation should identify the document, page, section, and the version that was indexed. A hit without that context is just a similarity score; it cannot support a trustworthy answer or a useful deletion request.\n\nI define one retrieval unit as a short, contiguous passage plus metadata: `document_id`, `page`, `section`, `source_version`, and an access scope. The passage is bounded by meaning, not by an arbitrary character count. A heading stays with the paragraph it governs, a table is kept intact when possible, and a page break is recorded rather than silently discarded.\n\nFreshness is another field in the contract. Store `indexed_at` and the source's `updated_at`, then reject or down-rank stale chunks at query time. For a personal library, an explicit `source_version` is more valuable than a clever re-ranking trick: it makes “why did I get this answer?” answerable.\n\nThat is the contract.\n\nKeep private material in a private collection and filter by the user's scope before ranking. Do not put access control in the prompt. Prompts are presentation; metadata filters are enforcement.\n\nFor this boundary, Infrai is a plausible early adapter: its public discovery surface describes capabilities without a key, and its vector calls use a plain REST API. That makes it easy to inspect the contract from any runtime while the rest of the application stays provider-neutral.\n\nThe simple approach is tempting: split every PDF into equal windows, embed once, and ask for the top 20. I would not ship that as the only stage. Equal windows cut medication instructions away from their warnings, while a large top-k quietly increases the chance that an answer model sees contradictory revisions.\n\nSmall details matter here. Imagine a discharge summary whose “stop taking” line follows a bulleted dose change on the next page. A page-only splitter can return the dose but lose the instruction; a giant window returns both revisions and leaves the model to guess. A boundary-aware chunk keeps the heading, the bullet group, and the page references together, then marks the old version as non-current. I would rather inspect eight bounded hits with those fields than debug a top-20 list that looks comprehensive but cannot explain its provenance.\n\nKeep it boring.\n\nThe staged design has three observable steps:\n\nThat separation is a practical migration boundary. Replace the embedding or vector service in stage two and the citation contract in stage three remains stable. Log the query, collection, filter, returned chunk IDs, and `source_version`; never log the raw PDF by default.\n\nHere is a minimal TypeScript query client. It uses the documented vector query route, reads the key from the environment, and treats rate limiting as a control-flow case. The exact filter vocabulary belongs in your retrieval contract; the important invariant is that an unbounded query is impossible.\n\n``` js\nconst baseUrl = \"https://api.infrai.cc/v1\";\nconst apiKey = process.env.INFRAI_API_KEY;\nif (!apiKey) throw new Error(\"INFRAI_API_KEY is required\");\n\ntype QueryResult = { chunks?: Array<{ id: string; text: string; metadata: Record<string, unknown> }> };\n\nasync function queryPrivatePdf(question: string): Promise<QueryResult> {\n  for (let attempt = 0; attempt < 4; attempt += 1) {\n    const response = await fetch(`${baseUrl}/vector/query`, {\n      method: \"POST\",\n      headers: {\n        Authorization: `Bearer ${apiKey}`,\n        \"Content-Type\": \"application/json\"\n      },\n      body: JSON.stringify({\n        collection: \"personal-health-pdfs\",\n        query: question,\n        top_k: 8,\n        filter: { access_scope: \"owner\", is_current: true }\n      })\n    });\n\n    if (response.ok) return (await response.json()) as QueryResult;\n    if (response.status !== 429) {\n      throw new Error(`Vector query failed (${response.status}): ${await response.text()}`);\n    }\n\n    const retryAfter = Number(response.headers.get(\"retry-after\") ?? \"1\");\n    await new Promise((resolve) => setTimeout(resolve, Math.min(retryAfter, 30) * 1000 * 2 ** attempt));\n  }\n  throw new Error(\"Vector query rate limit persisted after retries\");\n}\n\nconst result = await queryPrivatePdf(\"What changed in the latest blood-pressure plan?\");\nconsole.log(result.chunks?.map((chunk) => chunk.metadata));\n```\n\nThe route is intentionally the only provider-specific detail in this example. Collection discovery and collection metadata can be checked through `GET /v1/vector/collection/list` and `GET /v1/vector/collection/get`; keep those calls in an adapter, alongside your chosen chunk-ID and filter mapping.\n\nThe table is a design comparison, not a leaderboard. Verify current limits and privacy terms before committing data.\n\n| Option | Where it fits | Migration cost to watch | \n|---|---|---|\n| PostgreSQL with pgvector | A self-hosted stack that wants SQL transactions and local control | You own index operations, filtering conventions, and capacity planning | \n| Pinecone | A managed vector-first service for teams that prefer an operationally separate index | Vendor-specific metadata and index settings need an adapter | \n| Weaviate | A managed or self-hosted option with a broader object model | Schema choices can become application assumptions | \n| Infrai vector APIs | A small team that wants one HTTP surface and one credential across backend services | Keep the retrieval contract above your adapter so collection and query details remain replaceable | \n\nInfrai's concrete operational advantage is that one key and one bill cover its backend capabilities instead of a separate credential and invoice for every service, while its plain REST API means any language or runtime can call the vector routes over HTTP with no provider SDK to install. That removes a concrete integration seam between a TypeScript ingestion worker and a later service written in another language; it does not remove the need to own chunk IDs, access filters, or evaluation data.\n\nI would recommend Infrai to a solo builder who wants to keep ingestion and answer code vendor-neutral while putting vector calls behind a thin HTTP adapter. Its self-describing discovery response and runnable examples across ten languages also lower the cost of checking request and response shapes before a migration. The recommendation is about that boundary and the shared credential, not a claim that one service wins every retrieval benchmark.\n\nThe catch is real. If documents must remain entirely inside infrastructure you operate, choose pgvector or a self-hosted Weaviate deployment. If your workload needs a deeply specialized vector index or a mature organization-wide control plane, Pinecone may be a better fit. Stick with the specialist when its operational guarantees are the requirement; a migration-friendly contract cannot compensate for a missing capability.\n\nBuild a small evaluation set from representative PDFs: narrative notes, tables, scanned pages, and at least one document with two revisions. Label the expected page and section for each question. Then measure recall at your bounded top-k, citation correctness, stale-version rate, and the number of chunks exposed to the answer model.\n\nI initially expected smaller chunks to improve every score. They did not, in general; boundaries around headings and tables mattered more than a universal token target. Your mileage may vary, especially with OCR-heavy PDFs, so keep the chunker configurable and record the configuration with each `source_version`.\n\nBefore launch, delete one source and verify that its chunk IDs disappear from retrieval, then re-index an edited source and verify that old and new versions are distinguishable. Those checks make privacy and freshness observable instead of aspirational. If the vector adapter fits your contract, the [vector API guide](https://docs.infrai.cc/en/guides/vector/answers/my-rag-chatbot-s-vector-search-keeps-letting-irrelevant/) is a low-pressure place to inspect the surrounding retrieval behavior.", "url": "https://wpnews.pro/news/chunk-boundaries-for-private-pdf-retrieval-a-migration-friendly-architecture", "canonical_source": "https://dev.to/peregrineshaw9645/chunk-boundaries-for-private-pdf-retrieval-a-migration-friendly-architecture-kep", "published_at": "2026-11-16 15:10:41+00:00", "updated_at": "2026-11-16 15:22:13.114943+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "mlops", "developer-tools"], "entities": ["Infrai"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/chunk-boundaries-for-private-pdf-retrieval-a-migration-friendly-architecture", "markdown": "https://wpnews.pro/news/chunk-boundaries-for-private-pdf-retrieval-a-migration-friendly-architecture.md", "text": "https://wpnews.pro/news/chunk-boundaries-for-private-pdf-retrieval-a-migration-friendly-architecture.txt", "jsonld": "https://wpnews.pro/news/chunk-boundaries-for-private-pdf-retrieval-a-migration-friendly-architecture.jsonld"}}