The Search Agent That Stopped Fooling Itself

A team from the National University of Defense Technology has identified a failure mode in reinforcement learning-trained search agents, called retrieval-equivalence collapse, where agents repeatedly retrieve the same documents despite generating different queries. Their proposed fix, Harness-G, replaces free-form queries with a fixed menu of choices and improves performance by nearly 11 points on average across six benchmarks at smaller model sizes and about 4 points at larger ones.

Member-only story The Search Agent That Stopped Fooling Itself Why teaching an AI to pick from a menu beats letting it write its own questions Here is a strange failure that shows up when you train an AI agent to search for answers using reinforcement learning. You ask it to research a question. It writes a search query, gets some results, decides it needs more information, and writes a new query. On paper this looks like exploration. The agent is trying different phrasings, chasing different angles, behaving like a curious researcher. But if you actually check what documents it pulled back, something disappointing happens. Ten different rewritten queries for the same question often return the same handful of documents. The agent thinks it is exploring. It is actually going in circles wearing different outfits. A team from the National University of Defense Technology ran the numbers on this and gave the problem a name: retrieval-equivalence collapse. Their paper, “Harness-G: A Graph-Structured Harness for Search Agents,” does two things. First, it proves this collapse is real and explains why it quietly wrecks the training process. Second, it proposes a fix that swaps free-form search queries for a fixed menu of choices, paired with a smarter way to reward the agent for good decisions. The result beats the strongest existing method by close to 11 points on average across six question-answering benchmarks at smaller model sizes, and by about 4 points at larger ones.