How to Build Customer-Support Retrieval Architecture: Query Limits That Hold

Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives all the way into the answer. For a property-management support knowledge base, that usually means a small first pass for recall, a tighter second pass for precision, and a hard budget on how much text reaches the model.

The bill is made of work, not of vague “AI usage.” In this system, the expensive terms are document parsing and embedding during ingestion, then repeated vector work and model input tokens during support queries. Keeping every chunk forever increases storage and makes the retriever inspect more near-duplicates. Keeping too few chunks makes an answer look confident while omitting the lease exception that mattered.

I start by writing the answer contract before choosing a database. A contract says which tenant may see a document, what source citation is required, how many chunks a query may return, and what happens when confidence is low. The change that moves the dominant term is usually chunk policy: deduplicate boilerplate, retain the clause that carries the exception, and cap retrieval before reranking. The deliberate stop is old, superseded text. That saves query noise, but when a resident asks about a rule from last year, the system must be able to explain that the current collection excludes it and point to an archive workflow.

Treat ingestion, querying, and citation as separate observable stages. For a property manager, an indexed item should carry tenant_id, building_id, document_id, source_uri, effective_from, effective_to, and an access label. The access label is not decoration: preserve tenant and access-control metadata on every indexed item, then apply the same predicate at query time.

Use explicit collections rather than one giant namespace. A practical split is policies-current, leases-by-building, and maintenance-runbooks. A query can search the current policy collection first, then expand to a permitted archive only when the date in the question requires it. This keeps the common path small without pretending that historical material does not exist.

Here is a bounded local planner. It does not hide the limits in a prompt; it makes them testable before any provider call.

from dataclasses import dataclass
from typing import Iterable

@dataclass(frozen=True)
class RetrievalContract:
    first_pass_k: int = 24
    rerank_k: int = 8
    max_context_chunks: int = 5
    max_chunk_chars: int = 1800

def plan_query(question: str, allowed_collections: Iterable[str],
               contract: RetrievalContract) -> dict:
    collections = list(allowed_collections)
    if not collections:
        raise ValueError("access policy produced no searchable collection")
    if contract.rerank_k > contract.first_pass_k:
        raise ValueError("rerank_k cannot exceed first_pass_k")
    return {
        "question": question.strip(),
        "collections": collections,
        "first_pass_k": contract.first_pass_k,
        "rerank_k": contract.rerank_k,
        "max_context_chunks": contract.max_context_chunks,
        "max_chunk_chars": contract.max_chunk_chars,
    }

The numbers are guardrails, not universal truths. Start with a representative set of tickets: “late fee after a payment reversal,” “water leak in a unit,” and “can a tenant sublet?” Add failure cases with similar wording but different buildings. Measure recall and precision separately. If recall is weak, raise the first pass or improve chunking; if precision is weak, tighten metadata filters or reranking. Do not solve both by sending fifty chunks to the answer model.

The query path should have a visible budget. First retrieve a bounded candidate set from an allowed collection, then rerank those candidates with a deterministic feature set: vector score, effective-date match, building match, and document freshness. Finally pass only the top context chunks, each with its source identifier, to answer generation. A trace record should include query id, collection, candidate count, selected ids, and rejection reasons.

Measure it.

For one building, I would replay a deliberately awkward set of tickets: a “late fee” question with two policy versions, a maintenance request whose photo has no text, a sublet question restricted to a different tenant, and a query with no matching clause at all. The expected output is not just an answer string. It is the permitted source id, the collection searched, the number of candidates before and after filtering, and a refusal when the evidence is absent. If the old policy wins because its wording is closer, the trace tells you whether date filtering, chunk boundaries, or reranking made that decision. That level of detail is what lets a storage-minded team change one limit at a time instead of tuning by anecdotes.

The following adapter shows the two provider calls used by this design. Set INFRAI_BASE_URL to the documented versioned API base, keep the bearer key in configuration, set the method explicitly, and retry a rate-limited request with Retry-After. The request payload is assembled by the caller because chunk schemas differ between indexes; the route and transport contract remain fixed.

import os
import time
import requests

BASE_URL = os.environ["INFRAI_BASE_URL"].rstrip("/")

def post_vector(path: str, payload: dict) -> dict:
    key = os.environ["INFRAI_API_KEY"]
    for attempt in range(4):
        response = requests.post(
            BASE_URL + path.removeprefix("/v1"),
            json=payload,
            headers={"Authorization": f"Bearer {key}"},
            timeout=20,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)
            continue
        if not response.ok:
            raise RuntimeError(f"vector request failed: {response.status_code} {response.text}")
        return response.json()
    raise RuntimeError("rate limit persisted after retries")

def index_and_query(upsert_payload: dict, query_payload: dict) -> dict:
    post_vector("/v1/vector/upsert", upsert_payload)
    return post_vector("/v1/vector/query", query_payload)

A write retry must be idempotent. Put a client-generated chunk id or idempotency key inside the payload convention used by your index, and make the consumer safe to run twice; standard queue delivery and network retries are at-least-once realities. The example intentionally does not attach the API authorization header to any returned presigned URL, because a signed URL is already its own credential.

Latency is easier to reason about when each stage has a ceiling. For example, a 24-candidate first pass, eight-item rerank, and five-chunk context cap gives you a bounded worst case even as the corpus grows. Your mileage may vary: the right values depend on chunk size and language mix, so keep them in configuration and record the resulting recall, precision, and p95 latency in evaluation runs rather than claiming a benchmark you have not measured.

There is no universally correct index. Pinecone is a managed vector database with mature namespace and filtering patterns. Weaviate offers a schema-oriented database with hybrid retrieval options. Qdrant is a focused vector engine with payload filtering and self-hosting flexibility. Infrai is useful when the surrounding application already wants one plain REST surface and one credential for several backend capabilities; swapping the provider behind that contract can leave application code unchanged. That is an integration property, not a reason to skip retrieval tests.

The catch is that a single API surface does not remove data modeling work. Infrai is not suitable when your requirement is a specialized indexing feature that the documented vector capability does not provide; stick with Pinecone, Weaviate, or Qdrant when that feature is central and already fits your operating model. Conversely, a team tired of stitching SDKs and credentials may prefer the plain HTTP contract and keep its retrieval policy in its own service.

Keep the policy in your service.

Build a small gold set before production: real anonymized questions, the expected source document, allowed tenants, and a note explaining why near-matches are wrong. Test three slices independently: ingestion coverage, query recall/precision, and citation correctness. A result without the right source id is a retrieval failure even when the prose sounds plausible.

Log the rejected candidates as well as the selected ones. That exposes a common property-management failure mode: a building filter is syntactically valid but silently removes the only current lease addendum. Alert on empty candidate sets, missing access metadata, and answers that cannot cite a permitted source. Those checks are more useful than a generic “confidence” number.

Retention is a product decision with a technical cost. Keep current policy chunks hot; move expired leases and resolved tickets to an archive collection with explicit dates and narrower access. When an answer needs history, the planner should say why it expanded the search. When it does not, the archive should not compete for top-k slots. I am not sure one retention window will suit every portfolio, so validate it against your legal hold rules and the oldest question you promise to answer.

Start with one building and a few hundred representative documents, then replay the gold set after every chunking or query-limit change. Promote a configuration only when recall, precision, citation rate, and latency all stay inside the contract. The least complex design that passes those checks is usually the one worth operating.