Member-only story
Why teaching an AI to pick from a menu beats letting it write its own questions #
Here is a strange failure that shows up when you train an AI agent to search for answers using reinforcement learning. You ask it to research a question. It writes a search query, gets some results, decides it needs more information, and writes a new query. On paper this looks like exploration. The agent is trying different phrasings, chasing different angles, behaving like a curious researcher.
But if you actually check what documents it pulled back, something disappointing happens. Ten different rewritten queries for the same question often return the same handful of documents. The agent thinks it is exploring. It is actually going in circles wearing different outfits.
A team from the National University of Defense Technology ran the numbers on this and gave the problem a name: retrieval-equivalence collapse. Their paper, “Harness-G: A Graph-Structured Harness for Search Agents,” does two things. First, it proves this collapse is real and explains why it quietly wrecks the training process. Second, it proposes a fix that swaps free-form search queries for a fixed menu of choices, paired with a smarter way to reward the agent for good decisions. The result beats the strongest existing method by close to 11 points on average across six question-answering benchmarks at smaller model sizes, and by about 4 points at larger ones.