# Building a multi-agent document-search copilot — Part 2: adaptive Hybrid, and a permission gate after the rank > Source: > Published: 2026-07-14 12:13:19+00:00 *This is Part 2 of two. Part 1 opened on a document-search copilot whose v1 produced muddy results — two retrieval lanes fused into one rerank, where metadata rows that carry no text corrupted the relevance scores. The fix was two reframes: collapse the router into one structured Bedrock call (with a deterministic fallback), and pick exactly one retrieval strategy per query — MetadataOnly, ContentOnly, or Hybrid — instead of fusing the lanes. We left off at the one strategy that refuses to be tidy: Hybrid, where the user wants a topic and filters at the same time.* Picking up where Part 1 stopped, we're now in the back half of the same pipeline — the `direct_search → rerank → permission_filter → finalize_results` tail of the search graph. Two decisions left: how `Hybrid` actually retrieves, and where permissions get enforced. `Hybrid` is the interesting one, because "topic + filters" is genuinely a tradeoff and not a single right answer. There was a real debate about it — the kind that goes a few rounds on a whiteboard: run the filter and the rank in **parallel** (fast, but you have to reconcile two sets), or in **sequence** (precise, but slower)? We resolved it by refusing to pick — and routing on *selectivity* instead. The trick is one cheap question asked once: **how many documents match the filters?** `direct_search` peeks the filtered-universe size from the Documents service a single time, with a threshold (default `1000` ), and branches: ``` elif strategy == "Hybrid": threshold = cfg.get("filter_first_threshold", 1000) id_page = cfg.get("filter_id_page_size", 100) scope_ids, total_records = await _fetch_filtered_ids(metadata_filters, threshold, id_page) if scope_ids is not None: candidates = await _content_hits(scope_ids=scope_ids) # filter-first filters_enforced_upstream = True else: candidates = await _content_hits() # rank-then-filter ``` `terms` id-scope (over the version field on chunks, the parent field on attachment chunks). Every filter is enforced authoritatively at the source — nothing off-filter can possibly rank, and nothing on-filter is missed. It sets a flag so the later permission gate knows not to re-filter: ``` # filter-first scoped the topic ranking to the service-filtered id set, so the filters # are already enforced authoritatively; permission_filter must NOT re-enforce them. filters_enforced_upstream = True ``` `_row_passes_filters` — a faithful mirror of the Documents service's operation semantics across every searchable column (string / number / date / boolean / user_list, with `contains` / `equal` / `greaterThan` / the rest): ``` php def _row_passes_filters(row: dict, filters: list[dict]) -> bool: """Faithful local mirror of the Documents service filter semantics ... any hit whose row fails a hard filter is dropped, fail-closed.""" return all(_row_passes_one(row, f) for f in filters) ``` The thing that makes this *safe* rather than a sloppy approximation: when the filtered set is huge, the filter is nearly a no-op anyway (it matches most of the library), so the recall ceiling from ranking-first-then-dropping is harmless. The case where dropping would lose good matches — a *selective* filter — is exactly the case that takes the filter-first path, where nothing is dropped. The branch picks the strategy where its own weakness doesn't apply. I'll be honest: it took me a beat to trust this — it feels like a cheat until you sit with it — but the two failure modes genuinely cancel out. | filter-first (selective) | rank-then-filter (broad) | | |---|---|---| | Filtered universe | ≤ threshold (default 1000) | > threshold | | Semantic query | scoped to the filtered id set | unscoped | | Filters enforced | at the Documents service (authoritative) | locally, in `permission_filter` | | Recall risk | none — nothing off-filter can rank | harmless — filter matches most docs | | Round trips for the peek | one (read the id set) | one (just the count, then bail) | One peek, no extra round trips, and the precision/latency tradeoff is made *explicit and data-driven* instead of being a coin flip baked into the architecture. 💡 The pattern, generalized: when you can't decide between two retrieval strategies, measure the one property that distinguishes them (here, filter selectivity) with a single cheap probe, and route on it. Each branch is chosen precisely where its failure mode is benign. The content path is ranked by Cohere (over the top `rerank_candidates` , default `30` ), with the OpenSearch fused score / RRF as the fallback when the reranker is unreachable: ``` results = await bedrock_client.rerank(query=query_text, documents=documents, model_id=model_id) # ... on any failure or empty result -> _fallback() sorts by the OpenSearch raw score ``` Metadata results skip the reranker entirely — there's nothing to score — and keep the Documents service order: ``` if all(c.get("match_kind") == "metadata" for c in candidates): return {"reranked": candidates, "cached_top_30": candidates} ``` Then comes the part that makes security people lean forward in their chair: the per-document **view-permission gate runs after rerank**, not before retrieval. Stay with me — it's safe, and the reason it's safe is the whole point. `permission_filter` takes the reranked content hits, confirms each one is viewable via a single bulk `export_by_ids` call against the system of record, and drops the ones that aren't — fail-closed, so if the service is unreachable, every content hit is dropped rather than leaked: ``` rows = await documents_service.export_by_ids(content_ids, jwt) for row in rows: dvid = row.get("document_version_id") if dvid: viewable.add(dvid) # ... survivors = metadata rows (trusted) + content hits whose id is in `viewable` ``` Why is gating *after* rerank safe? Because there are two different boundaries doing two different jobs: `member_id` , and that id comes from the JWT — never from the request body, never from the index. A user physically cannot retrieve another tenant's vectors. That boundary is upstream and absolute.Rerank only *orders* the candidates. It exposes nothing — every candidate it sees already passed the tenant wall, and the set it's ordering is bounded (the reranked top set), so the view gate is one bounded bulk call, not a fan-out. The metadata lane skips the gate entirely because its rows came back already view-filtered by the service's own SQL — re-checking trusted rows would be wasted work. Identity from the token, tenant at retrieval, view-visibility after, fail-closed throughout. Here's a small but real consequence of "one strategy per query" that landed squarely on my side of the stack — and nearly slipped past me. Content hits carry a relevance score. Metadata hits don't — a satisfied filter is not a relevance measurement, and showing the OpenSearch fused score (which for an exact field match is ~0) would read as "barely relevant" for what is actually a perfect match. So a metadata hit emits a ** null** score in the SSE payload, and the UI has to render that as "matched your filter," not "0% relevant": ``` php def _shape_source_for_emit(s: dict) -> dict: # A metadata match is a satisfied filter, not a relevance measure, so its score stays null # (the card shows just the match reason); content matches keep their fused/rerank score. score = None if s.get("score_signal") == "metadata" else ( s.get("aggregate_score") or s.get("score") or 0.0 ) ... ``` It's a tiny detail, but it's the kind of thing that breaks if the frontend assumes "every result has a number." A `null` score is a first-class state — "this matched a filter, there's no relevance to show" — not a missing value to default to zero. Defaulting it to `0.0` would have quietly buried every exact metadata match at the bottom of the list, sorted *below* loosely-relevant content — a perfect match dead last, and no error anywhere to tell you why. The contract had to carry the absence on purpose. The throughline across all four decisions — the two in Part 1 and the two here — is the same instinct: **make the system commit to one clear shape per turn, and make the tradeoffs explicit instead of averaging them away.** One router call instead of three. One retrieval strategy instead of a fused blend. One selectivity probe instead of a parallel-vs-sequential guess. One bounded permission check after a bounded rerank. The muddy-results version tried to do everything at once and let the reranker sort it out. It couldn't. Picking, on purpose, is what made the rank mean something again. 👋 Thanks for sticking through both parts. If you've fought the same muddy-rank fight, I'd genuinely love to hear how you cut yours — the failure modes are weirdly universal. *Missed the setup? Part 1 covers the muddy-results problem, the single-call router, and the reframe to one strategy per query.* Thanks for reading all the way through 🙌 If you're building agents and fighting the same *"is this finding even real?"* problem, I'd genuinely like to compare notes — come say hi on [LinkedIn](https://www.linkedin.com/in/rodrigo-diego-67867185/).