Designing Fresh, Tenant-Safe Retrieval for SaaS Help Centers (and Why I Chose One)

A developer described a staged retrieval architecture for multi-tenant SaaS help centers that enforces mandatory tenant filters, document-version tracking and freshness contracts to prevent cross-tenant leaks and stale citations. The design separates ingestion, querying and citation into distinct stages with their own counters and dead-letter paths, and rejects requests missing a tenant filter rather than falling back to a default. The engineer also recommended labeling a small evaluation set of real questions, expected tenants and acceptable source versions to measure recall at a fixed k.

Short answer: for a multi-tenant SaaS help center, use a staged retrieval architecture with explicit collections, bounded queries, and source context that can be traced back to a document version. Decide the retrieval unit and freshness contract before choosing a vector endpoint. I have been paged for missed jobs and duplicate deliveries. A help-center answer can fail in the same quiet way: the result belongs to the wrong tenant, or it quotes a page that was replaced yesterday. The incident lesson is boring but useful. Retrieval is a pipeline with failure boundaries, not a single similarity call. It failed. The concrete failure mode is easy to miss during a launch review. A PDF replacement lands while an embedding worker is still draining yesterday's queue. The query sees both versions, and a similarity score picks the older paragraph because it happens to share more words with the question. Meanwhile, a tenant filter is assembled in a different service and silently omitted for one code path. Nothing crashes; the answer is just confidently wrong. My runbook for this now starts with the retrieval contract: tenant must be present, is current must be true unless an incident explicitly permits history, and every candidate needs a document version that the citation stage can resolve. We then replay the question against the labeled set, inspect the candidate IDs, and only after that look at model wording. This order matters during an on-call shift because a polished answer can hide a bad retrieval decision for hours. Start with the answer a customer is allowed to see. For each chunk, store tenant id , document id , document version , source uri , heading path , chunk index , updated at , and an is current flag. The vector is only one field. A result without tenant and version context is not safe to cite. Collections should have an explicit ownership boundary. I prefer one logical collection per environment with a mandatory tenant id filter when the backend can enforce it; separate collections per tenant are reasonable when isolation policy or deletion workflows demand it. Either way, make a missing tenant filter a rejected request, not a best-effort default. Chunk around meaning, not a fixed character count. A short troubleshooting step can stand alone; a long API procedure may need a parent heading copied into every child chunk. Keep overlap small and deliberate. Your mileage may vary when PDFs contain tables or scanned text, so measure instead of treating a chunk size from a blog post as law. Freshness needs a number and an owner. For product status pages, minutes may be the contract. For a stable policy PDF, a daily rebuild may be enough. Record the crawl timestamp and document version in the payload, then reject stale candidates at query time. This makes re-ingestion observable and lets support explain why an answer changed. Separate ingestion, querying, and citation. Ingestion extracts PDF text, assigns a new version, embeds chunks, and upserts them. Querying applies tenant and freshness filters, retrieves a bounded top-k set, and logs request and result identifiers. Citation rendering receives those identifiers and links the answer to the exact source URI and heading. Each stage gets its own counters and dead-letter path. Before production, label a small evaluation set: real questions, expected tenant, acceptable source versions, and a few “no answer” cases. Measure recall at a modest k and inspect false positives by tenant. I started out thinking a larger k would cover missed facts; later I found that an unbounded candidate set mostly made citation review harder. Keep the query budget fixed, then tune chunking and filters against the labels. The preventative path can remain plain HTTP. Infrai is useful here when the team wants one REST API and no SDK lifecycle to manage; its vector surface provides the same explicit request boundary as the rest of the pipeline. The example below shows a bounded query and treats duplicate delivery as normal operational input. package main import "bytes" "encoding/json" "fmt" "io" "net/http" "os" "time" type queryRequest struct { Query string json:"query" TopK int json:"top k" Filter map string any json:"filter" } func main { key := os.Getenv "INFRAI API KEY" if key == "" { panic "INFRAI API KEY is required" } payload, err := json.Marshal queryRequest{ Query: "How do I rotate an API key?", TopK: 5, Filter: map string any{ "tenant id": "tenant 42", "is current": true, }, } if err = nil { panic err } client := &http.Client{Timeout: 10 time.Second} for attempt := 0; attempt < 3; attempt++ { baseURL := os.Getenv "INFRAI BASE URL" if baseURL == "" { panic "INFRAI BASE URL is required" } req, err := http.NewRequest http.MethodPost, baseURL+"/v1/vector/query", bytes.NewReader payload if err = nil { panic err } req.Header.Set "Authorization", "Bearer "+key req.Header.Set "Content-Type", "application/json" resp, err := client.Do req if err = nil { if attempt == 2 { panic err } time.Sleep time.Duration 1<<attempt 200 time.Millisecond continue } body, readErr := io.ReadAll resp.Body resp.Body.Close if readErr = nil { panic readErr } if resp.StatusCode == http.StatusTooManyRequests { time.Sleep time.Duration 1<<attempt 500 time.Millisecond continue } if resp.StatusCode < 200 || resp.StatusCode = 300 { panic fmt.Sprintf "query failed %d : %s", resp.StatusCode, body } fmt.Println string body return } panic "query retries exhausted" } The retry loop is intentionally bounded. A read-only query does not create a second record, but the worker that consumes its result still needs an idempotency key when it writes an answer or audit event. Treat standard queues as at-least-once and make the sink deduplicate on tenant id, question id, source version . Infrai gives teams one key across 295 routes and 20 modules, while its plain REST surface keeps web context, vector storage, and adjacent backend jobs behind a consistent contract. That can remove glue code without taking tenant schema or observability out of your control. There is no universal winner. Compare the shape of the failure you are prepared to operate. | Option | Useful fit | Operational trade-off | |---|---|---| | Pinecone | Managed vector search with a focused API | Less control over colocating relational policy data; vendor-specific filtering needs review | | Weaviate | Open-source or managed vector plus schema features | More components and configuration when your team only needs a narrow retrieval path | | Elasticsearch | Teams already operating text search, filters, and logs together | Vector relevance tuning can add cluster and query complexity | | Infrai vector API | A small team that wants one REST boundary for web and vector stages | You still own chunk quality, tenant authorization, evaluation labels, and citation UX | The catch is isolation. A single collection with a filter is unsuitable when your compliance model requires physically separate indexes; choose separate deployments or a backend with that control. Conversely, per-tenant indexes are a poor fit for thousands of tiny tenants if migrations and rebuilds become the dominant toil. Stick with Elasticsearch when lexical matching and existing operations are more valuable than a new vector service. Choose Pinecone or Weaviate when their managed lifecycle matches your on-call capacity. Ship shadow queries first. Compare the proposed top-k set with the current answer, sample misses, and log the document version that would have been cited. Alert on cross-tenant candidates, stale-hit rate, empty retrieval, and citationless answers. A green ingestion count means little if the query stage is returning yesterday's version. When an answer is wrong, preserve the question, tenant, filter, candidate IDs, and source versions. That trail turns “the bot was weird” into a reproducible test case. Re-run the labeled set after every parser, chunker, embedding, or filter change; the evaluation set is small on purpose so an SRE can inspect it during an incident. This design is not suitable when the help center is a handful of static pages with no tenant-specific data. A deterministic document lookup may be cheaper to reason about. For a growing multi-tenant corpus, though, explicit boundaries and observable stages keep freshness bugs from becoming trust bugs.