Metadata filtering in vector search: post-filter loses results, measured

Multi-tenant RAG needs a filter: “only search documents this user may see.” Vector indexes are built for nearest neighbours, not for “nearest neighbours among these 1,000.” So where does the filter go: after the search, inside it, or before it?

Short answer: post-filtering silently returns fewer results as the filter gets more selective. In my test, a filter that keeps 1% of the data returned 1.9 results out of 10 when I fetched the top 100 and filtered. In-graph filtering fixed recall but got slow. For selective filters, exact search over the filtered subset was the fastest and always correct.

100,000 vectors, each with a tenant id. I ran two layouts:

Selectivity is the fraction of the corpus the filter keeps: about 50%, 10%, 1% and 0.1%. Ground truth is exact search restricted to the tenant’s vectors, so recall@10 here means “how many of the true filtered top 10 did the method return.”

Methods:

filter argument, evaluated while the graph is traversed (

def post_filter(index, mask, q, fetch=100, k=10):
    labels, _ = index.knn_query(q, k=fetch)
    return [l for l in labels[0] if mask[l]][:k]

def in_graph(index, mask, q, k=10):
    labels, _ = index.knn_query(q, k=k, filter=lambda i: bool(mask[i]))
    return list(labels[0])

def pre_filter(X_tenant, ids_tenant, q, k=10):  # X_tenant = X[ids_tenant], built once
    s = X_tenant @ q
    top = np.argpartition(-s, k)[:k]
    return list(ids_tenant[top[np.argsort(-s[top])]])

Recall@10 / average results returned (max 10) / median latency.

Post-filtering behaves like a coin flip with the selectivity as the odds: fetch 10, keep about selectivity × 10. The query is never told that most of its neighbours are off-limits. Even fetching 1,000 candidates returned only 2.1 results per query at 0.1%.

In-graph filtering never lost recall, but its cost grows as the filter tightens, from 0.44 ms at 50% to 74.22 ms at 0.1%. The traversal has to wander through many forbidden nodes to find allowed ones. At 0.1% it took 74.22 ms, while the exact scan of the 100-vector subset took 0.01 ms.

When tenants own their own regions of the space, the problem mostly disappears.

Recall is 1.000 for every method. The query’s nearest neighbours are already inside the allowed set, so the filter rejects almost nothing. This is the trap: post-filtering looks fine in a demo where a user asks about their own topic, then fails when a filter is independent of the query. Examples are a date range, a document type, or a tenant whose content overlaps with everyone else’s. Real tenants are somewhere between my two layouts, so test with your own data.

ef to 1,000, which cost 1.88 ms versus 0.22 ms.np.sqrt(D) returns a float64 scalar, and under NumPy 2 this upcast the query, so each dot product converted the whole float32 matrix. I fixed it by casting queries to float32 and reran everything. Check dtypes if your brute-force numbers look too slow.len(results) == k on an independent filter would have caught it here. For the basics of embeddings and indexes, see the free lesson Embeddings & vector databases.

Originally published on RecallRun. I run experiments like this every week at RecallRun. If you want to build RAG systems end to end, the AI Engineering course starts free: https://recallrun.dev/learn/ai-engineering/?utm_source=devto&utm_campaign=metadata-filter