Metadata filtering in vector search: post-filter loses results, measured

A developer benchmarked metadata filtering strategies in vector search on 100,000 vectors and found that post-filtering silently loses results as filters become more selective, returning only 1.9 of 10 requested results at 1% selectivity even when fetching 100 candidates. In-graph filtering preserved recall but latency grew from 0.44 ms at 50% selectivity to 74.22 ms at 0.1%, while exact search over the filtered subset was both fastest and always correct for selective filters. The results also showed that when tenants occupy distinct regions of the embedding space, all methods achieve perfect recall, meaning post-filtering failures only surface when the filter is independent of the query.

Multi-tenant RAG needs a filter: “only search documents this user may see.” Vector indexes are built for nearest neighbours, not for “nearest neighbours among these 1,000.” So where does the filter go: after the search, inside it, or before it? Short answer: post-filtering silently returns fewer results as the filter gets more selective. In my test, a filter that keeps 1% of the data returned 1.9 results out of 10 when I fetched the top 100 and filtered. In-graph filtering fixed recall but got slow. For selective filters, exact search over the filtered subset was the fastest and always correct. 100,000 vectors, each with a tenant id. I ran two layouts: Selectivity is the fraction of the corpus the filter keeps: about 50%, 10%, 1% and 0.1%. Ground truth is exact search restricted to the tenant’s vectors, so recall@10 here means “how many of the true filtered top 10 did the method return.” Methods: filter argument, evaluated while the graph is traversed python def post filter index, mask, q, fetch=100, k=10 : labels, = index.knn query q, k=fetch return l for l in labels 0 if mask l :k def in graph index, mask, q, k=10 : labels, = index.knn query q, k=k, filter=lambda i: bool mask i return list labels 0 def pre filter X tenant, ids tenant, q, k=10 : X tenant = X ids tenant , built once s = X tenant @ q top = np.argpartition -s, k :k return list ids tenant top np.argsort -s top Recall@10 / average results returned max 10 / median latency. | Selectivity | post k=10 | post k=100 | post k=1000 ef=1000 | in-graph filter | pre-filter partitioned | |---|---|---|---|---|---| | 50% | 0.553 / 5.5 / 0.21 ms | 1.000 / 10 / 0.24 ms | 1.000 / 10 / 1.94 ms | 1.000 / 10 / 0.44 ms | 1.000 / 10 / 1.38 ms | | 10% | 0.181 / 1.8 / 0.20 ms | 0.908 / 9.1 / 0.22 ms | 1.000 / 10 / 1.83 ms | 1.000 / 10 / 2.19 ms | 1.000 / 10 / 0.16 ms | | 1% | 0.110 / 1.1 / 0.20 ms | 0.191 / 1.9 / 0.22 ms | 0.884 / 9.2 / 1.88 ms | 1.000 / 10 / 17.03 ms | 1.000 / 10 / 0.03 ms | | 0.1% | 0.100 / 1.0 / 0.14 ms | 0.105 / 1.0 / 0.16 ms | 0.214 / 2.1 / 1.79 ms | 1.000 / 10 / 74.22 ms | 1.000 / 10 / 0.01 ms | Post-filtering behaves like a coin flip with the selectivity as the odds: fetch 10, keep about selectivity × 10 . The query is never told that most of its neighbours are off-limits. Even fetching 1,000 candidates returned only 2.1 results per query at 0.1%. In-graph filtering never lost recall, but its cost grows as the filter tightens, from 0.44 ms at 50% to 74.22 ms at 0.1%. The traversal has to wander through many forbidden nodes to find allowed ones. At 0.1% it took 74.22 ms, while the exact scan of the 100-vector subset took 0.01 ms. When tenants own their own regions of the space, the problem mostly disappears. | Selectivity | post k=10 | in-graph filter | pre-filter partitioned | |---|---|---|---| | 50% | 1.000 / 0.20 ms | 1.000 / 0.22 ms | 1.000 / 1.34 ms | | 10% | 1.000 / 0.20 ms | 1.000 / 0.29 ms | 1.000 / 0.16 ms | | 1% | 1.000 / 0.12 ms | 1.000 / 0.61 ms | 1.000 / 0.03 ms | | 0.1% | 1.000 / 0.07 ms | 1.000 / 0.16 ms | 1.000 / 0.01 ms | Recall is 1.000 for every method. The query’s nearest neighbours are already inside the allowed set, so the filter rejects almost nothing. This is the trap: post-filtering looks fine in a demo where a user asks about their own topic, then fails when a filter is independent of the query. Examples are a date range, a document type, or a tenant whose content overlaps with everyone else’s. Real tenants are somewhere between my two layouts, so test with your own data. ef to 1,000, which cost 1.88 ms versus 0.22 ms. np.sqrt D returns a float64 scalar, and under NumPy 2 this upcast the query, so each dot product converted the whole float32 matrix. I fixed it by casting queries to float32 and reran everything. Check dtypes if your brute-force numbers look too slow. len results == k on an independent filter would have caught it here. For the basics of embeddings and indexes, see the free lesson Embeddings & vector databases //../../learn/ai-engineering/06-vector-search.html . Originally published on RecallRun https://recallrun.dev/posts/rag-metadata-filtering/ . I run experiments like this every week at RecallRun. If you want to build RAG systems end to end, the AI Engineering course starts free: https://recallrun.dev/learn/ai-engineering/?utm source=devto&utm campaign=metadata-filter https://recallrun.dev/learn/ai-engineering/?utm source=devto&utm campaign=metadata-filter