Most internal knowledge assistants start the same way: index everything in the shared drive, put a chat box on top, and add "only answer using documents the user is allowed to see" to the system prompt.
That last line is not access control. It's a suggestion to a language model. If a restricted document is retrieved and placed in the context, the model has already seen it, and no instruction reliably stops it from leaking a number, a name or a summary.
The rule that fixes this is simple: a user must never be able to retrieve a chunk they couldn't open in the source system. Here is how to build that.
When you ingest a document, store its access-control list with every chunk: the user IDs, group IDs or roles that can read it in the source system (SharePoint, Google Drive, Confluence, your CRM). Store them as filterable metadata, not as text in the chunk.
{
"chunk_id": "policy-2291#4",
"text": "...",
"source": "sharepoint://hr/policies/leave.docx",
"allowed_groups": ["hr-all", "managers-emea"],
"last_modified": "2026-10-30T09:12:00Z"
}
At query time, resolve the user's identity (from your SSO token) into the groups they belong to, and pass that as a filter in the vector or hybrid search itself. Most vector databases support metadata pre-filtering.
Avoid post-filtering, where you retrieve the top 10 and then drop the ones the user can't see. It leaks through ranking (the user may get fewer, worse results that hint at what exists), and it breaks easily when someone changes k.
This is the part teams forget. Someone is removed from a group, or a folder is locked down, and the index still holds the old ACL until the next full re-sync, sometimes days later.
Treat ACL changes as first-class events: subscribe to the source system's change notifications where they exist, re-sync permissions on a short schedule where they don't, and alert when an ACL sync job fails. A stale permission is a data leak with a delay.
Retrieval is not the only path. Watch for:
Require every claim in the answer to cite a chunk ID that was actually retrieved for this user. Validate the citations in code before showing the answer: if a citation points to a chunk outside the retrieved set, drop the claim or refuse.
This gives you two things at once: users can verify answers, and you have an audit trail of exactly which documents influenced each response.
Add permission tests to your evaluation set, not just quality tests:
Run these on every change to the ingestion pipeline, the chunker or the cache.
None of this is exotic. It's the same rule we already apply to databases and file shares, applied to a new kind of search. The prompt can shape how the model answers; it should never be the thing deciding who can see what.
I'm VP Marketing at SoluLab, where our teams build RAG systems and AI agents for enterprises. This post reflects the patterns we design for; your stack and compliance needs will shape the details.