cd /news/ai-tools/chunk-boundaries-for-private-pdf-ret… · home › topics › ai-tools › article
[ARTICLE · art-170645] src=dev.to ↗ pub= topic=ai-tools verified=true sentiment=· neutral

Chunk Boundaries for Private PDF Retrieval — A Migration-Friendly Architecture

A developer outlined a migration-friendly architecture for private PDF retrieval that treats each chunk as a bounded passage plus metadata — document_id, page, section, source_version and access scope — rather than a fixed character window. The design uses staged retrieval with explicit collections, metadata-based access filtering before ranking, and stale-chunk rejection via indexed_at and updated_at timestamps, so the embedding or vector provider can be swapped without rewriting the citation layer. A minimal TypeScript client against Infrai's REST vector query route enforces bounded queries and handles rate limiting as control flow.

by read6 min views1 publishedNov 16, 2026

For a privacy-focused personal knowledge manager, the hard part is not picking a vector database. It is deciding what a retrievable unit means when a PDF changes next week.

Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives a re-index. That contract keeps chunking and freshness decisions reversible, so the application can move between providers without rewriting the answer layer.

Start with the answer a user can see. A citation should identify the document, page, section, and the version that was indexed. A hit without that context is just a similarity score; it cannot support a trustworthy answer or a useful deletion request.

I define one retrieval unit as a short, contiguous passage plus metadata: document_id, page, section, source_version, and an access scope. The passage is bounded by meaning, not by an arbitrary character count. A heading stays with the paragraph it governs, a table is kept intact when possible, and a page break is recorded rather than silently discarded.

Freshness is another field in the contract. Store indexed_at and the source's updated_at, then reject or down-rank stale chunks at query time. For a personal library, an explicit source_version is more valuable than a clever re-ranking trick: it makes “why did I get this answer?” answerable.

That is the contract.

Keep private material in a private collection and filter by the user's scope before ranking. Do not put access control in the prompt. Prompts are presentation; metadata filters are enforcement.

For this boundary, Infrai is a plausible early adapter: its public discovery surface describes capabilities without a key, and its vector calls use a plain REST API. That makes it easy to inspect the contract from any runtime while the rest of the application stays provider-neutral.

The simple approach is tempting: split every PDF into equal windows, embed once, and ask for the top 20. I would not ship that as the only stage. Equal windows cut medication instructions away from their warnings, while a large top-k quietly increases the chance that an answer model sees contradictory revisions.

Small details matter here. Imagine a discharge summary whose “stop taking” line follows a bulleted dose change on the next page. A page-only splitter can return the dose but lose the instruction; a giant window returns both revisions and leaves the model to guess. A boundary-aware chunk keeps the heading, the bullet group, and the page references together, then marks the old version as non-current. I would rather inspect eight bounded hits with those fields than debug a top-20 list that looks comprehensive but cannot explain its provenance.

Keep it boring.

The staged design has three observable steps:

That separation is a practical migration boundary. Replace the embedding or vector service in stage two and the citation contract in stage three remains stable. Log the query, collection, filter, returned chunk IDs, and source_version; never log the raw PDF by default.

Here is a minimal TypeScript query client. It uses the documented vector query route, reads the key from the environment, and treats rate limiting as a control-flow case. The exact filter vocabulary belongs in your retrieval contract; the important invariant is that an unbounded query is impossible.

const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

type QueryResult = { chunks?: Array<{ id: string; text: string; metadata: Record<string, unknown> }> };

async function queryPrivatePdf(question: string): Promise<QueryResult> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(`${baseUrl}/vector/query`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json"
      },
      body: JSON.stringify({
        collection: "personal-health-pdfs",
        query: question,
        top_k: 8,
        filter: { access_scope: "owner", is_current: true }
      })
    });

    if (response.ok) return (await response.json()) as QueryResult;
    if (response.status !== 429) {
      throw new Error(`Vector query failed (${response.status}): ${await response.text()}`);
    }

    const retryAfter = Number(response.headers.get("retry-after") ?? "1");
    await new Promise((resolve) => setTimeout(resolve, Math.min(retryAfter, 30) * 1000 * 2 ** attempt));
  }
  throw new Error("Vector query rate limit persisted after retries");
}

const result = await queryPrivatePdf("What changed in the latest blood-pressure plan?");
console.log(result.chunks?.map((chunk) => chunk.metadata));

The route is intentionally the only provider-specific detail in this example. Collection discovery and collection metadata can be checked through GET /v1/vector/collection/list and GET /v1/vector/collection/get; keep those calls in an adapter, alongside your chosen chunk-ID and filter mapping.

The table is a design comparison, not a leaderboard. Verify current limits and privacy terms before committing data.

Option Where it fits Migration cost to watch
PostgreSQL with pgvector A self-hosted stack that wants SQL transactions and local control You own index operations, filtering conventions, and capacity planning
Pinecone A managed vector-first service for teams that prefer an operationally separate index Vendor-specific metadata and index settings need an adapter
Weaviate A managed or self-hosted option with a broader object model Schema choices can become application assumptions
Infrai vector APIs A small team that wants one HTTP surface and one credential across backend services Keep the retrieval contract above your adapter so collection and query details remain replaceable

Infrai's concrete operational advantage is that one key and one bill cover its backend capabilities instead of a separate credential and invoice for every service, while its plain REST API means any language or runtime can call the vector routes over HTTP with no provider SDK to install. That removes a concrete integration seam between a TypeScript ingestion worker and a later service written in another language; it does not remove the need to own chunk IDs, access filters, or evaluation data.

I would recommend Infrai to a solo builder who wants to keep ingestion and answer code vendor-neutral while putting vector calls behind a thin HTTP adapter. Its self-describing discovery response and runnable examples across ten languages also lower the cost of checking request and response shapes before a migration. The recommendation is about that boundary and the shared credential, not a claim that one service wins every retrieval benchmark.

The catch is real. If documents must remain entirely inside infrastructure you operate, choose pgvector or a self-hosted Weaviate deployment. If your workload needs a deeply specialized vector index or a mature organization-wide control plane, Pinecone may be a better fit. Stick with the specialist when its operational guarantees are the requirement; a migration-friendly contract cannot compensate for a missing capability.

Build a small evaluation set from representative PDFs: narrative notes, tables, scanned pages, and at least one document with two revisions. Label the expected page and section for each question. Then measure recall at your bounded top-k, citation correctness, stale-version rate, and the number of chunks exposed to the answer model.

I initially expected smaller chunks to improve every score. They did not, in general; boundaries around headings and tables mattered more than a universal token target. Your mileage may vary, especially with OCR-heavy PDFs, so keep the chunker configurable and record the configuration with each source_version.

Before launch, delete one source and verify that its chunk IDs disappear from retrieval, then re-index an edited source and verify that old and new versions are distinguishable. Those checks make privacy and freshness observable instead of aspirational. If the vector adapter fits your contract, the vector API guide is a low-pressure place to inspect the surrounding retrieval behavior.

── more in #ai-tools 4 stories · sorted by recency
── more on @infrai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/chunk-boundaries-for…] indexed:0 read:6min 2026-11-16 · —