{"slug": "building-a-multi-agent-document-search-copilot-part-2-adaptive-hybrid-and-a-gate", "title": "Building a multi-agent document-search copilot — Part 2: adaptive Hybrid, and a permission gate after the rank", "summary": "A developer building a multi-agent document-search copilot implemented an adaptive hybrid retrieval strategy that branches based on filter selectivity. When filters match few documents, the system scopes semantic search to the filtered ID set for authoritative enforcement; when filters match many documents, it ranks first and filters locally, avoiding recall loss. The approach resolves the tradeoff between parallel and sequential hybrid retrieval by routing on selectivity, ensuring no off-filter documents rank while maintaining performance.", "body_md": "*This is Part 2 of two. Part 1 opened on a document-search copilot whose v1 produced muddy results — two retrieval lanes fused into one rerank, where metadata rows that carry no text corrupted the relevance scores. The fix was two reframes: collapse the router into one structured Bedrock call (with a deterministic fallback), and pick exactly one retrieval strategy per query — MetadataOnly, ContentOnly, or Hybrid — instead of fusing the lanes. We left off at the one strategy that refuses to be tidy: Hybrid, where the user wants a topic and filters at the same time.*\n\nPicking up where Part 1 stopped, we're now in the back half of the same pipeline — the `direct_search → rerank → permission_filter → finalize_results`\n\ntail of the search graph. Two decisions left: how `Hybrid`\n\nactually retrieves, and where permissions get enforced.\n\n`Hybrid`\n\nis the interesting one, because \"topic + filters\" is genuinely a tradeoff and not a single right answer. There was a real debate about it — the kind that goes a few rounds on a whiteboard: run the filter and the rank in **parallel** (fast, but you have to reconcile two sets), or in **sequence** (precise, but slower)? We resolved it by refusing to pick — and routing on *selectivity* instead.\n\nThe trick is one cheap question asked once: **how many documents match the filters?** `direct_search`\n\npeeks the filtered-universe size from the Documents service a single time, with a threshold (default `1000`\n\n), and branches:\n\n```\nelif strategy == \"Hybrid\":\n    threshold = cfg.get(\"filter_first_threshold\", 1000)\n    id_page = cfg.get(\"filter_id_page_size\", 100)\n    scope_ids, total_records = await _fetch_filtered_ids(metadata_filters, threshold, id_page)\n    if scope_ids is not None:\n        candidates = await _content_hits(scope_ids=scope_ids)   # filter-first\n        filters_enforced_upstream = True\n    else:\n        candidates = await _content_hits()                       # rank-then-filter\n```\n\n`terms`\n\nid-scope (over the version field on chunks, the parent field on attachment chunks). Every filter is enforced authoritatively at the source — nothing off-filter can possibly rank, and nothing on-filter is missed. It sets a flag so the later permission gate knows not to re-filter:\n\n```\n  # filter-first scoped the topic ranking to the service-filtered id set, so the filters\n  # are already enforced authoritatively; permission_filter must NOT re-enforce them.\n  filters_enforced_upstream = True\n```\n\n`_row_passes_filters`\n\n— a faithful mirror of the Documents service's operation semantics across every searchable column (string / number / date / boolean / user_list, with `contains`\n\n/ `equal`\n\n/ `greaterThan`\n\n/ the rest):\n\n``` php\n  def _row_passes_filters(row: dict, filters: list[dict]) -> bool:\n      \"\"\"Faithful local mirror of the Documents service filter semantics ...\n      any hit whose row fails a hard filter is dropped, fail-closed.\"\"\"\n      return all(_row_passes_one(row, f) for f in filters)\n```\n\nThe thing that makes this *safe* rather than a sloppy approximation: when the filtered set is huge, the filter is nearly a no-op anyway (it matches most of the library), so the recall ceiling from ranking-first-then-dropping is harmless. The case where dropping would lose good matches — a *selective* filter — is exactly the case that takes the filter-first path, where nothing is dropped. The branch picks the strategy where its own weakness doesn't apply. I'll be honest: it took me a beat to trust this — it feels like a cheat until you sit with it — but the two failure modes genuinely cancel out.\n\n| filter-first (selective) | rank-then-filter (broad) | |\n|---|---|---|\n| Filtered universe | ≤ threshold (default 1000) | > threshold |\n| Semantic query | scoped to the filtered id set | unscoped |\n| Filters enforced | at the Documents service (authoritative) | locally, in `permission_filter`\n|\n| Recall risk | none — nothing off-filter can rank | harmless — filter matches most docs |\n| Round trips for the peek | one (read the id set) | one (just the count, then bail) |\n\nOne peek, no extra round trips, and the precision/latency tradeoff is made *explicit and data-driven* instead of being a coin flip baked into the architecture.\n\n💡 The pattern, generalized: when you can't decide between two retrieval strategies, measure the one property that distinguishes them (here, filter selectivity) with a single cheap probe, and route on it. Each branch is chosen precisely where its failure mode is benign.\n\nThe content path is ranked by Cohere (over the top `rerank_candidates`\n\n, default `30`\n\n), with the OpenSearch fused score / RRF as the fallback when the reranker is unreachable:\n\n```\nresults = await bedrock_client.rerank(query=query_text, documents=documents, model_id=model_id)\n# ... on any failure or empty result -> _fallback() sorts by the OpenSearch raw score\n```\n\nMetadata results skip the reranker entirely — there's nothing to score — and keep the Documents service order:\n\n```\nif all(c.get(\"match_kind\") == \"metadata\" for c in candidates):\n    return {\"reranked\": candidates, \"cached_top_30\": candidates}\n```\n\nThen comes the part that makes security people lean forward in their chair: the per-document **view-permission gate runs after rerank**, not before retrieval. Stay with me — it's safe, and the reason it's safe is the whole point.\n\n`permission_filter`\n\ntakes the reranked content hits, confirms each one is viewable via a single bulk `export_by_ids`\n\ncall against the system of record, and drops the ones that aren't — fail-closed, so if the service is unreachable, every content hit is dropped rather than leaked:\n\n```\nrows = await documents_service.export_by_ids(content_ids, jwt)\nfor row in rows:\n    dvid = row.get(\"document_version_id\")\n    if dvid:\n        viewable.add(dvid)\n# ... survivors = metadata rows (trusted) + content hits whose id is in `viewable`\n```\n\nWhy is gating *after* rerank safe? Because there are two different boundaries doing two different jobs:\n\n`member_id`\n\n, and that id comes from the JWT — never from the request body, never from the index. A user physically cannot retrieve another tenant's vectors. That boundary is upstream and absolute.Rerank only *orders* the candidates. It exposes nothing — every candidate it sees already passed the tenant wall, and the set it's ordering is bounded (the reranked top set), so the view gate is one bounded bulk call, not a fan-out. The metadata lane skips the gate entirely because its rows came back already view-filtered by the service's own SQL — re-checking trusted rows would be wasted work. Identity from the token, tenant at retrieval, view-visibility after, fail-closed throughout.\n\nHere's a small but real consequence of \"one strategy per query\" that landed squarely on my side of the stack — and nearly slipped past me. Content hits carry a relevance score. Metadata hits don't — a satisfied filter is not a relevance measurement, and showing the OpenSearch fused score (which for an exact field match is ~0) would read as \"barely relevant\" for what is actually a perfect match. So a metadata hit emits a ** null** score in the SSE payload, and the UI has to render that as \"matched your filter,\" not \"0% relevant\":\n\n``` php\ndef _shape_source_for_emit(s: dict) -> dict:\n    # A metadata match is a satisfied filter, not a relevance measure, so its score stays null\n    # (the card shows just the match reason); content matches keep their fused/rerank score.\n    score = None if s.get(\"score_signal\") == \"metadata\" else (\n        s.get(\"aggregate_score\") or s.get(\"score\") or 0.0\n    )\n    ...\n```\n\nIt's a tiny detail, but it's the kind of thing that breaks if the frontend assumes \"every result has a number.\" A `null`\n\nscore is a first-class state — \"this matched a filter, there's no relevance to show\" — not a missing value to default to zero. Defaulting it to `0.0`\n\nwould have quietly buried every exact metadata match at the bottom of the list, sorted *below* loosely-relevant content — a perfect match dead last, and no error anywhere to tell you why. The contract had to carry the absence on purpose.\n\nThe throughline across all four decisions — the two in Part 1 and the two here — is the same instinct: **make the system commit to one clear shape per turn, and make the tradeoffs explicit instead of averaging them away.** One router call instead of three. One retrieval strategy instead of a fused blend. One selectivity probe instead of a parallel-vs-sequential guess. One bounded permission check after a bounded rerank. The muddy-results version tried to do everything at once and let the reranker sort it out. It couldn't. Picking, on purpose, is what made the rank mean something again.\n\n👋 Thanks for sticking through both parts. If you've fought the same muddy-rank fight, I'd genuinely love to hear how you cut yours — the failure modes are weirdly universal.\n\n*Missed the setup? Part 1 covers the muddy-results problem, the single-call router, and the reframe to one strategy per query.*\n\nThanks for reading all the way through 🙌 If you're building agents and fighting the same *\"is this finding even real?\"* problem, I'd genuinely like to compare notes — come say hi on [LinkedIn](https://www.linkedin.com/in/rodrigo-diego-67867185/).", "url": "https://wpnews.pro/news/building-a-multi-agent-document-search-copilot-part-2-adaptive-hybrid-and-a-gate", "canonical_source": "https://dev.to/rdiegoss/building-a-multi-agent-document-search-copilot-part-2-adaptive-hybrid-and-a-permission-gate-3k40", "published_at": "2026-07-14 12:13:19+00:00", "updated_at": "2026-07-14 12:30:52.722141+00:00", "lang": "en", "topics": ["ai-agents", "ai-products", "ai-tools", "developer-tools", "machine-learning"], "entities": ["Bedrock", "Documents service"], "alternates": {"html": "https://wpnews.pro/news/building-a-multi-agent-document-search-copilot-part-2-adaptive-hybrid-and-a-gate", "markdown": "https://wpnews.pro/news/building-a-multi-agent-document-search-copilot-part-2-adaptive-hybrid-and-a-gate.md", "text": "https://wpnews.pro/news/building-a-multi-agent-document-search-copilot-part-2-adaptive-hybrid-and-a-gate.txt", "jsonld": "https://wpnews.pro/news/building-a-multi-agent-document-search-copilot-part-2-adaptive-hybrid-and-a-gate.jsonld"}}