When the data arrives with someone else's rules

On Monday's article about whose permissions an MCP search runs with, the hard part was a single question: who is asking. This one is the next question, and people are now running into it in public.

A commenter on Hacker News described how their team feeds company knowledge to agents: everything ingested stays linked "to the source" and to "the authorisation layer of the source that was ingested". Their example is a wiki whose permissions live in Windows AD, so that information "needs to be collected when you ingest". And then the line that's the whole problem: "Such connectors are source specific and must be aware how the system manages permissions."

That's right, and it gets harder with every source you add. A shared index rarely holds one company's one system. It holds the wiki (AD groups), the ticket tracker (project roles), a shared drive (link sharing, folder inheritance), and sooner or later a partner's records that arrived under the partner's terms. Each of those has its own rulebook. One retrieval step has to honour all of them at once, and it's the same step that ranks.

There are two ways to build that, and I think it's worth being plain about what each one makes you own.

At ingest, the connector reads the source's permissions for each document and writes them into the chunk's payload: a list of users or groups allowed to read it. At query time, the search filters on that list.

The appeal is obvious. Permissions come from the system that already has them, nobody re-types anything, and a document that's private in the wiki is private in the index.

What you take on:

We actually specified a version of this, built it, and took it out. Our authorization spec has the post-mortem: each chunk was to carry hashes of the principals allowed to read it, with a background sweep keeping them fresh and an alert on how far behind it fell. The tags were computed, the sweep ran, the alert paged, and no query ever read them. The reasons for removing rather than finishing it are in that section; the one that matters here is that a copied permission is "stale in the wrong direction". It can only ever lag the real one.

The other way is to not copy the source's rules at all. Each source, or each part of a source with its own audience, goes into its own container. Access is granted on the container, and the search resolves the caller's permitted containers on every request, from the live grant table.

In Nacre that container is a layer. Scopes form a tree, workspace → layer → document; grants name a user, group or service account; and with no grant there's no access. The reference connectors work this way: a rule per layer (a path in a repository, a key prefix in a bucket, a query over a table) decides where each source item lands, and the connector's own key only needs write on those layers. It never searches.

What this makes you own is different, and I'd rather say it straight than have you find it out:

What you get back is that revocation is a write to one table and takes effect on the next search, because nothing downstream holds a copy. We measured that on the core API in August: 79 ms for a direct grant and 58 ms for leaving a group.

Whichever design you pick, apply it inside the search, not after it. If you rank the whole index and then strip what the caller can't see, two things go wrong. The caller asked for ten results and gets four. And anything built from the pre-strip set, a summary, a citation count, a "3 more results hidden", tells them that documents exist in a place they can't read.

That second leak is the one to test for, and it's easy to test: ask, as someone who can't read a source, for results from that source by name, and compare the answer to what you get for a source that doesn't exist. If the two differ in any way, status code, body, error wording, timing you can see without trying hard, the boundary leaks its own existence.

Our public stand has a demo company with two read-only logins, printed on demo.nacre.work. The engineer reads two layers, engineering and handbook. The contractor reads only handbook. The search API takes an optional layers list, and the API docs say it narrows only: it can't reach a layer no grant reaches, and "naming a layer you cannot read returns nothing from it and is indistinguishable from naming one that does not exist."

Here's that claim as a script. The passwords rotate, so copy them from the demo page rather than from here.

#!/usr/bin/env bash
set -euo pipefail
API=https://playground.nacre.work/v1
Q='what is the contract number for Northwind'

token() {   # $1 login name, $2 password
  curl -s "$API/auth/login" -H 'content-type: application/json' \
    -d "{\"email\":\"$1@demo.nacre.work\",\"password\":\"$2\",\"organization\":\"demo\"}" \
  | jq -r .access_token
}

ask() {     # $1 who, $2 token, $3 layer name
  body=$(curl -s "$API/search" \
    -H "authorization: Bearer $2" -H 'content-type: application/json' \
    -d "{\"query\":\"$Q\",\"top_k\":10,\"layers\":[\"$3\"]}")
  echo "$1 asks for $3: $(jq -r '"\(.items|length) results from [\([.items[].layer]|unique|join(","))]"' <<<"$body"), $(printf %s "$body" | wc -c) bytes"
}

ENG=$(token engineer "$ENGINEER_PASSWORD")
CON=$(token contractor "$CONTRACTOR_PASSWORD")

ask engineer   "$ENG" engineering
ask contractor "$CON" engineering
ask contractor "$CON" no-such-layer

What it printed today, 12 November:

engineer asks for engineering: 5 results from [engineering], 3435 bytes
contractor asks for engineering: 0 results from [], 12 bytes
contractor asks for no-such-layer: 0 results from [], 12 bytes

Side by side, with the plain search (no layers at all) for the contractor added:

The contractor's answer for a real layer they can't read is byte-for-byte the answer for a layer that was never there. And with no layers at all, the contractor still gets three results, from the one layer they can read, rather than three minus whatever was stripped.

One thing this test doesn't show, and I'd want you to know it: response time. I didn't measure it carefully enough to make a claim either way, so treat it as an open item if timing matters in your threat model.

If your sources are a few systems with clear audiences, I'd grant at the boundary. If one source has fine-grained, per-document sharing that people genuinely rely on, copying its ACLs may be the only way to honour it, and then the questions to ask of that design are how often it syncs and what happens in the gap.

The open-source build is on GitHub under Apache-2.0, and the authorization spec is the document to read before trusting any of this, including from us.

Originally published at https://nacre.work/blog/when-the-data-arrives-with-someone-elses-rules.