{"slug": "permission-filters-and-pgvector-why-your-most-restricted-users-get-the-fewest", "title": "Permission Filters and pgvector: Why Your Most Restricted Users Get the Fewest Answers", "summary": "A developer measured how pgvector's HNSW approximate index silently starves permission-filtered AI search results, showing that a caller allowed to read only 10 percent of documents received a median of 4 rows out of a requested 20, while a caller with 90 percent access got all 20. The writeup demonstrates that pgvector 0.8.0's iterative index scans (hnsw.iterative_scan = strict_order or relaxed_order) restore full recall at a latency cost, with median query time rising from 1.7 ms to 27.8 ms under strict_order versus 47.5 ms for exact search.", "body_md": "If you build AI search over documents that not everyone may read, you filter by permission. In PostgreSQL with pgvector, that usually means a WHERE clause on the caller's groups and an ORDER BY on vector distance. With no vector index, that query is exact and always complete. Add an HNSW index for speed, and something quiet happens: the people allowed to read the least start getting the fewest answers. Nothing errors. Nothing leaks, because the filter still holds. The list just comes back short.\n\nHere is the mechanism, measured, and the two fixes. Everything below ran on PostgreSQL 17.10 with pgvector 0.8.4, and the script is at the end.\n\nChunks of documents carry embeddings. Documents carry the groups allowed to read them. A search for one caller looks like this:\n\n```\nSELECT c.id\nFROM chunk c\nJOIN document d ON d.id = c.document_id\nWHERE d.allowed_principals && ARRAY['group:hr']\nORDER BY c.embedding <=> $1\nLIMIT 20;\n```\n\nThe `&&` operator is array overlap: a document qualifies if the caller holds any one of its groups. With no vector index, pgvector does exact nearest neighbor search, which its documentation describes as providing perfect recall. This query returns 20 rows whenever 20 readable chunks exist.\n\npgvector's README says it plainly: with approximate indexes, filtering is applied after the index is scanned. The scan produces a fixed list of candidates, sized by `hnsw.ef_search`, which is 40 by default, and the WHERE clause runs on that list. If the caller may read 10 percent of the rows, about 4 of the 40 survive.\n\nWe measured it. 100,000 chunks across 10,000 documents: one document in ten readable by `group:hr`, the other nine by `group:staff`. Random 64-dimension vectors, an HNSW index with default settings, the same 40 random queries for each caller, LIMIT 20:\n\n| Caller | Can read | Rows returned, of 20 | \n|---|---|---|\n| group:staff | 90 percent | 20 on every query | \n| group:hr | 10 percent | median 4, as few as 1, never more than 8 | \n\nSame query, same index, same data. The only difference is who asked.\n\nA category filter is the same for everyone, so a short result shows up the first time anyone tests it. A permission filter depends on who is asking. Developers test with broad accounts, and a broad account gets full results. The short list only appears for the restricted user: the new hire, the contractor, the three-person team with its own folder.\n\nAnd a short list does not look like an error. An AI assistant handed 4 passages instead of 20 answers from the 4. If the passage that mattered was number 5, it can tell a new hire there is no policy on something that has one, and the new hire has no reason to doubt it.\n\npgvector 0.8.0 added iterative index scans. When the filter leaves too few rows, the scan keeps going through the index instead of stopping at the candidate list:\n\n```\nBEGIN;\nSET LOCAL hnsw.iterative_scan = strict_order;\n-- the search query\nCOMMIT;\n```\n\n`SET LOCAL` ends with the transaction that runs the search, so a pooled connection cannot carry the setting anywhere else. Outside a transaction it does nothing. The other mode, `relaxed_order`, can return rows slightly out of distance order in exchange for better recall, and pgvector's README shows a materialized CTE that puts them back in order.\n\nOn the same 40 queries, the HR caller got 20 of 20 every time, in either mode. It costs time, because the scan now looks at more of the index. The median query went from 1.7 ms to 27.8 ms with `strict_order`, and to 18.1 ms with `relaxed_order`, on one desktop. For comparison, exact search, with the index turned off, took 47.5 ms.\n\nIterative scans have limits. A scan stops after visiting about `hnsw.max_scan_tuples` entries, 20,000 by default, or when it reaches a memory cap of `work_mem` times `hnsw.scan_mem_multiplier`. A caller who may read almost nothing can hit those limits before 20 matches turn up.\n\nSo we added a third group, `group:legal`, readable on one document in a thousand: 0.1 percent of the chunks. Left alone, the planner did the right thing. It used the GIN index on `allowed_principals` to find the ten readable documents, joined their chunks, and sorted them by exact distance: 20 of 20 on every query, at a median of 33 ms.\n\nThen we forced the query through the HNSW index, to see what happens when the planner picks it. With iterative scans on and the default limits, the median was 16 of 20, and the worst query returned 10. Raising `hnsw.max_scan_tuples` to 100,000 alone changed nothing. Raising `hnsw.scan_mem_multiplier` to 4 alone changed nothing. Raising both got 20 of 20, at a median of 732 ms: about 22 times slower than the exact plan the planner had chosen on its own.\n\nSo the second fix is not a setting. It is an ordinary index on the permission column. pgvector's README calls an index on the filter column \"a good place to start\" and notes that exact indexes work well for conditions that match a low percentage of rows. With one in place, the planner has a fast, complete path for your narrowest callers. Confirm it with EXPLAIN as your most restricted real user, not as yourself.\n\nEvery number in this post came from asking as a specific group. That is the test that finds this problem, and a test run as an administrator never will. If your retrieval has a test set, give it cases that run as a restricted group and expect a specific document back. When an index change starves that group, the case fails before a user notices.\n\nThe retrieval engine we build for clients works this way. Its vector search is exact, with no approximate index, until a corpus is large enough to need one and the recall cost has been measured. Its test sets run questions as a named group, and a case fails when an expected document is missing from the top five results or a forbidden one appears in them.\n\nRandom vectors keep the arithmetic clean: what a caller loses tracks what fraction of the rows they can read. Real embeddings cluster, and permissions often follow topic, so an HR user asking an HR question may lose less, and the same user asking about something outside their documents may lose more. The timings come from one desktop and will differ on yours. Both are reasons to measure your own documents, as your own restricted users.\n\nIf you are putting AI in front of documents that not everyone may read, this is the kind of detail we work through with organizations that keep their data in-house: [private AI](https://agaveis.com/arizona-private-ai).\n\nNeeds PostgreSQL 17 with pgvector 0.8.0 or later; run it with psql. Each count comes from one random query, so yours will differ from the medians above. The index build may print a notice that it no longer fits in `maintenance_work_mem`; that only makes the build slower.\n\n```\n-- PostgreSQL 17 with pgvector 0.8.0 or later. Run with psql.\nCREATE EXTENSION IF NOT EXISTS vector;\n\n-- 10,000 documents: one in ten readable by group:hr, one in a thousand also by group:legal.\nCREATE TABLE document (id int PRIMARY KEY, allowed_principals text[] NOT NULL);\nINSERT INTO document\nSELECT g, CASE WHEN g % 1000 = 0 THEN ARRAY['group:hr', 'group:legal']\n               WHEN g % 10 = 0 THEN ARRAY['group:hr']\n               ELSE ARRAY['group:staff'] END\nFROM generate_series(1, 10000) g;\nCREATE INDEX ON document USING GIN (allowed_principals);\n\n-- 100,000 chunks, ten per document, random 64-dimension embeddings.\nCREATE TABLE chunk (id int PRIMARY KEY, document_id int NOT NULL REFERENCES document(id), embedding vector(64) NOT NULL);\nINSERT INTO chunk\nSELECT g, (g % 10000) + 1, (SELECT array_agg(random() + 0 * g + 0 * i) FROM generate_series(1, 64) i)::vector(64)\nFROM generate_series(1, 100000) g;\nANALYZE document;\nANALYZE chunk;\n\n-- One random query vector.\nSELECT (SELECT array_agg(random()) FROM generate_series(1, 64))::vector(64)::text AS qv \\gset\n\n\\echo 'No vector index (exact search). Rows returned of 20, for group:hr:'\nSELECT count(*) FROM (SELECT c.id FROM chunk c JOIN document d ON d.id = c.document_id\n  WHERE d.allowed_principals && ARRAY['group:hr'] ORDER BY c.embedding <=> :'qv' LIMIT 20) t;\n\n-- Single process build, so it fits a small container's shared memory.\nSET max_parallel_maintenance_workers = 0;\nCREATE INDEX ON chunk USING hnsw (embedding vector_cosine_ops);\n\n\\echo 'HNSW index, defaults. group:staff (reads 90 percent), then group:hr (reads 10 percent):'\nSELECT count(*) FROM (SELECT c.id FROM chunk c JOIN document d ON d.id = c.document_id\n  WHERE d.allowed_principals && ARRAY['group:staff'] ORDER BY c.embedding <=> :'qv' LIMIT 20) t;\nSELECT count(*) FROM (SELECT c.id FROM chunk c JOIN document d ON d.id = c.document_id\n  WHERE d.allowed_principals && ARRAY['group:hr'] ORDER BY c.embedding <=> :'qv' LIMIT 20) t;\n\n\\echo 'Iterative scans on. group:hr again:'\nSET hnsw.iterative_scan = strict_order;\nSELECT count(*) FROM (SELECT c.id FROM chunk c JOIN document d ON d.id = c.document_id\n  WHERE d.allowed_principals && ARRAY['group:hr'] ORDER BY c.embedding <=> :'qv' LIMIT 20) t;\n```\n\nAgave Information Solutions builds [on-premises AI systems](https://agaveis.com/local-ai), [data architecture](https://agaveis.com/database-architecture), and [custom software](https://agaveis.com/custom-development) out of Scottsdale, Arizona. If your AI search has to respect who may read what, [get in touch](https://agaveis.com/about).\n\n*Originally published at [agaveis.com](https://agaveis.com/blog/pgvector-permission-filters-restricted-users).*", "url": "https://wpnews.pro/news/permission-filters-and-pgvector-why-your-most-restricted-users-get-the-fewest", "canonical_source": "https://dev.to/agave_info_solutions/permission-filters-and-pgvector-why-your-most-restricted-users-get-the-fewest-answers-2e63", "published_at": "2026-09-26 00:15:23+00:00", "updated_at": "2026-09-26 00:30:26.289294+00:00", "lang": "en", "topics": ["ai-search", "ai-infrastructure", "mlops", "developer-tools"], "entities": ["PostgreSQL", "pgvector", "HNSW"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/permission-filters-and-pgvector-why-your-most-restricted-users-get-the-fewest", "markdown": "https://wpnews.pro/news/permission-filters-and-pgvector-why-your-most-restricted-users-get-the-fewest.md", "text": "https://wpnews.pro/news/permission-filters-and-pgvector-why-your-most-restricted-users-get-the-fewest.txt", "jsonld": "https://wpnews.pro/news/permission-filters-and-pgvector-why-your-most-restricted-users-get-the-fewest.jsonld"}}