To run agentic search over a hierarchy of document clusters, first cluster the corpus: embed every document, then split the embeddings recursively into a tree, and give each cluster a label and a short summary. At query time an agent navigates the tree top-down. It reads the summaries at one level, chooses which clusters to open, descends into them, and runs vector or keyword search only inside the chosen leaf clusters. It stops when the evidence it has gathered answers the question. The agent never sees the whole corpus, only the summaries on its path and the chunks from a few leaves. The rest of this post covers why that works, how to build the tree, what the loop looks like in code, and where it fails.
Flat vector search ranks every chunk against the query. On a large, varied corpus the top results often include chunks that share vocabulary with the query and come from the wrong topic. An agent that retrieves those chunks has to spend context reading them.
A cluster tree changes three things:
Sports > Skateboarding) is a record of why a document was returned. You can log it, show it to a user, and debug it when the answer is wrong.
The tree has three ingredients: embeddings, a clustering algorithm applied recursively, and a label plus summary for every node.
Embed. Use one embedding model for the whole corpus so distances mean the same thing at every level.
Cluster recursively. Run a clustering algorithm on the whole set to get top-level groups, then run it again inside each group that is big enough to split, and keep going until groups are small or you reach a depth limit. K-means works when you know roughly how many groups you want at each level. HDBSCAN works when you do not, and it marks outliers as noise instead of forcing them into a group. In both cases, reducing dimensionality first (UMAP is common) tends to make the splits cleaner.
Label and summarize. For each cluster, sample member documents and ask an LLM for a short label and a one or two sentence summary. These two fields are what the agent reads while navigating, so their quality sets the ceiling on the whole system. A label such as "Species" on a parent that holds both animals and landforms will send queries about mountains down the wrong branch.
The best-known paper on this family of ideas is RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (Sarthi, Abdullah, Tuli, Khanna, Goldie and Manning, January 2024). Its abstract describes recursively embedding, clustering and summarizing chunks of text to build a tree with different levels of summarization, and reports that pairing RAPTOR retrieval with GPT-4 improved the best prior result on the QuALITY benchmark by 20 percentage points of absolute accuracy. RAPTOR retrieves from the tree. The loop below adds an agent that decides which branches to open.
Give the agent three tools:
list_children(cluster_id) returns the child clusters with their labels and summaries. With no argument it returns the top level.read_summary(cluster_id) returns the full summary of one cluster.search_within(cluster_id, query) runs vector or keyword search over documents in that cluster and its descendants.
A sketch in Python, with llm_choose and enough_evidence standing in for whatever model calls you use to pick clusters and judge sufficiency:
def agentic_search(query, tree, llm_choose, beam=3, max_depth=4, max_hits=20):
frontier = [tree.root] # clusters we are about to open
leaves = []
for _ in range(max_depth):
candidates = []
for node in frontier:
kids = tree.list_children(node)
if not kids:
leaves.append(node) # nothing below: search here
continue
candidates += kids
if not candidates:
break
frontier = llm_choose(query, candidates, k=beam)
leaves += frontier
hits = []
for leaf in leaves:
hits += tree.search_within(leaf, query)
if enough_evidence(query, hits):
return hits[:max_hits]
if beam >= 9:
return hits[:max_hits]
return agentic_search(query, tree, llm_choose, beam=beam + 2)
Three details matter in practice.
First, ask the model to score every child, not to name one. Scores let you keep the top beam paths and rank finished paths against each other. Averaging scores with a geometric mean keeps a deep leaf from being penalized for having more decisions behind it.
Second, show a parent cluster to the model as the leaves reachable below it. A parent label is a compromise, and the leaf names tell the model what is actually inside.
Third, give the agent an explicit stop rule. Without one it keeps descending or searching after the answer is already in context.
A wrong branch early. The biggest risk is the first decision. If the agent opens the wrong top-level cluster, everything below it is searching the wrong region, and the final answer can be confidently wrong. Two mitigations help. Keep a beam of k branches at each level, so a near-miss on the first choice is still alive at the second. And let the agent backtrack: if search_within returns weak hits, return to the last level where a sibling scored close and try that one.
Model calls per level. Every level costs a model call. A tree of depth three with a beam of three costs a handful of calls before any retrieval happens. Batch the candidates for all paths at one level into one request, and use a small, fast model for navigation decisions since each one is a short classification.
Stale clusters. Clusters describe the corpus as it was when you ran the clustering. New documents either land in the nearest existing cluster or sit outside the tree, and labels drift out of date as content shifts. Re-cluster on a trigger: a schedule, a document count threshold, or a quality signal such as falling cohesion or a rising share of outliers.
Summary quality. A vague summary hides a cluster from the agent. Spot-check labels and summaries when you build the tree, and keep a set of queries with known answers to run after every rebuild.
Mixpeek's Agent Search retriever stage has a cluster_navigation strategy. Per the docs, a cluster run writes one centroid document per cluster with its label, summary and parent_cluster_id. The stage starts at the top-level clusters, asks a decision model which child cluster most likely holds documents that answer the query, and repeats until it reaches leaf clusters. It then returns the member documents of the best leaves. The walk is a beam search: beam_width defaults to 3, and max_leaf_clusters sets how many leaves return members. Each returned document carries cluster_path and cluster_path_score, and the stage metadata includes a navigation_trace with the candidates and probabilities at every level. The stage returns the members of the chosen clusters. It does not run a separate vector search inside them, though you can chain a search stage after it. The docs do not describe backtracking.
On the build side, the Clusters docs describe vector clustering with eight algorithms including HDBSCAN and K-Means, optional UMAP preprocessing, and hierarchical sub-clustering through hierarchical: true and max_hierarchy_depth. LLM labeling is configurable per cluster. Each execution reports quality metrics, including a should_recluster recommendation, and clusters can be re-run automatically with cron, interval or event-based triggers. The hierarchical search tutorial walks through the whole sequence from clustering to a retriever that navigates the tree.
Disclosure: I work on Mixpeek. Mixpeek builds clusters over collections, including hierarchical vector clusters with LLM-written labels (Clusters docs), and its Agent Search stage can navigate those clusters top-down with the cluster_navigation strategy (Agent Search docs). The tutorial linked above shows the end-to-end setup.