{"slug": "agentic-search-retrieval-harness-or-model", "title": "Agentic search - retrieval, harness, or model?", "summary": "Agentic search can be implemented in three distinct ways: retrieval-centric, harness-centric, and model-centric. The retrieval-centric approach relies on building high-quality search to guide agents, but search remains difficult and imperfect. The harness-centric approach puts the agent in charge of using basic retrieval tools, while the model-centric approach fine-tunes an LLM to understand how to search proprietary data.", "body_md": "Agentic search gets interesting when agents do not know how to find the right answer.\n\nOh, the agent might think it knows. It might confidently BS us. But the agent’s poor domain intuition steers itself astray.\n\nAgents make false assumptions about what *our* users think is relevant. Our fashionista users think “red shoes” should return high-heels. When I worked at one company ABE wasn’t a president, it was an A/B testing tool. Agents need context to know these things - and [context engineering needs agentic search](https://www.youtube.com/watch?v=ynJyIKwjonM)\n\nConfusingly, depending who you talk to, agentic search can mean one of three distinct implementation patterns:\n\n**Retrieval-centric**- just build good search so agents can use it to fill in missing context** Harness-centric**- steer the agent towards needed context, even with bad search** Model-centric**- fine-tune an LLM to know how to search*our*data\n\nIn this post, I’ll walk through each. With a bit more context, you might appreciate which flavor your colleagues seem to mean when they say “agentic search”\n\n## Retrieval-Centric Implementation\n\nWhen frontier models don’t know, they search. Ask ChatGPT a question about news, it’ll search. Ask it about a very specific technical problem, it’ll search. During training, LLMs see *search* examples as a technique to learn what they don’t know.\n\nTherefore, we just need to build good search to solve any conceivable query from an agent.\n\nLet’s assume we run an e-commerce catalog. For us, when users search “red shoes” they mean “red high heels”. But the agent doesn’t know that. Luckily it asks *search*. Below we see it returns the right results.\n\nBelow we see initial retrieval to lexical / vector backends pulls back some reasonable, if naive, “red shoe” candidates. Still that’s not quite right. Luckily the reranker shapes the results towards our understanding of that intent.\n\nOf course we might have other components here - query understanding, diversity, custom embedding models, and more.\n\nThe important point is that *search leads the agent by the nose* towards what’s relevant. We assume search can define what a good “red shoe” is, overriding the agent’s perspective.\n\n### When RAG answers don’t look like answers\n\nMost teams build retrieval-centric approaches with classic RAG. Chunks of answers and embeddings trained to recognize them as answers.\n\nUnfortunately, answers don’t always look tied to the question. For example:\n\nQuestion:\n\nSynopsis of the book Ubik\n\nAnswer:\n\nBy the year 1992, humanity has colonized the Moon and psychic powers are common. The protagonist, Joe Chip, is a debt-ridden technician working for Runciter Associates, a “prudence organization” employing “inertials”—people with the ability to negate the powers of telepaths and “precogs”—to enforce the privacy of clients.\n\nIf you don’t know the book Ubik, it’s not clear that this answers the question. The agent says “cool story bro” and ignores the info.\n\nSearch like this actually is divorced from the web search trusted by frontier models. The web contains titles, headings, and other elements placing the answer in context.\n\n### The big downside? Search remains hard. We’re not Google.\n\nMost importantly, we don’t have perfect search like Google or Bing. Even with good search - almost nobody builds *Google-quality results.* And since agents trust search - simple distractors in retrieval can, as Lester Solbakken says, [easily confuse reasoning](https://2026.berlinbuzzwords.de/session/when-better-retrieval-makes-agents-worse/).\n\n## Harness-centric implementations\n\nWho should be in charge? Should the agent manage the search process? Or should search drive the agent?\n\nMoving the emphasis to the harness, we put the agent in charge.\n\nImagine stripping search tools down to core retrieval primitives. Just a BM25 backend. Or a filesystem with CLI tools. We tell the agent to find what it needs with these untuned tools. The agent might struggle more, but hey, it’s smart, it can figure it out. Right?\n\nStill, the agent might find what it thinks is relevant, as it does in the image below. Sadly that’s not what’s actually relevant. To help the agent, we inject external knowledge. We let a judge direct the agent, correcting its mistakes, and guiding it towards better search strategies.\n\nSo when the agent returns results, it’s not the “user” that receives the candidates. Instead a judge labels results as relevant or not. The agent gets the hint, finding results similar to those labeled relevant. Avoiding those labeled irrelevant. To steal from Jo Kristian Bergem, this is [relevance feedback on steroids](https://program.berlinbuzzwords.de/bbuzz26/talk/PLNTP9/).\n\nThis works. Look at the ESCI dataset. If I have an oracle labeling with judgments from these datasets, I get quite significant improvements.\n\n| Variant | NDCG@10 | Description |\n|---|---|---|\n| ESCI BM25 | 0.2895 | Simple BM25 weighing name / description |\n| ESCI agentic | 0.4101 | GPT5-mini tool calling loop w/ BM25+e5 embeddings tools |\n| ESCI agentic w/ oracle | 0.5843 | GPT5-mini tool calling loop w/ BM25+e5 embeddings tools. Judge responds with feedback once |\n\nHere the second row we trust the agent’s interpretation of queries. That provides a large gain over BM25. But inject more domain knowledge via a judge (the third row) and agent performance improves dramatically.\n\nWe can guide agents with information ahead of time too. For example, I’ve seen teams take inspiration from skills in coding agents. Imagine a targeted query plan that tells the agent how to use its tools to solve a specific problem.\n\n### Optimizing content helps harnesses\n\nWe help agents even further by optimizing content for findability. That’s the trick we’ve played on ourselves with coding agents. We write docs to explain to agents why something’s useful. We don’t just rely on that dumb chunk, we document the purpose of the knowledge:\n\n/books/scifi/ubik.txt\n\n## About Book Ubik Ubik is a book by Philip K. Dick\n\n## Synopsis\n\nBy the year 1992, humanity has colonized the Moon and psychic powers are common. The protagonist, Joe Chip, is a debt-ridden technician working for Runciter Associates, a “prudence organization” employing “inertials”—people with the ability to negate the powers of telepaths and “precogs”—to enforce the privacy of clients.\n\nUnlike a naive chunk, this bit of information has purpose. It’s clear, when its in the context, what problem it solves.\n\nBy organizing + cataloging information we’ve done what every search developer wishes their content team would do: actually optimize content to be findable.\n\n### The downside: cost\n\nThe major downside here? Token costs. The exploration here requires more than just issuing a search and retrieving one set of results. It’s an agent’s exploration of the environment. And that exploration starts fresh with every new context window.\n\n## Model-centric approach\n\nWhat if we literally fine-tuned the LLM to search efficiently?\n\nThe harness based approach takes a long, winding path towards relevant results. With every fresh context, it wastes calls relearning what works over-and-over. It’s costly and slow.\n\nCould we train a model that doesn’t repeat the same mistakes with our tools?\n\nOur judge helps label some of the agent’s reasoning / tool-calling paths as useful. And label others less so. Once you have labeled traces, you can fine-tune an open weights model to proactively, efficiently make the right tool calls given the input.\n\nBelow, for example, the model knows exactly how to search for our “red shoes” query given its extensive fine-tuning.\n\nNow we’ve built a model targeting the search task. We don’t tune our retrievers much. Instead we continue to use the simpler / dumb search tools.\n\nEven cooler, the agentic search model’s behavior can change just by adjusting the system prompt. “This is an e-commerce search task, optimize for top 10 ranking” might be different from “You’re retrieving chunks for financial research, maximize diversity and relevance at the same time.”\n\nFurther the models can stay smaller. By just focusing on the search part - not the entire domain problem - they can shrink to a manageable size allowing them to be self-hosted.\n\n### The downside? Early days\n\nThe OG in the space is [SID](http://sid.ai). Other contenders like Glean’s Waldo model have since entered. They’re all promising, but not widely adopted. Many unknown unknowns exist. In a sense, we don’t yet know the upside or downsides yet. I suspect though this paradigm has tremendous promise in the years ahead.\n\nIn other words - stay tuned!\n\n## The alchemy of ambiguity\n\nPeople use “agentic search” confusingly and inconsistently. They focus on their part of the problem. That means we have a collection of solutions like enriched base metals - and it’s unclear what alchemy brings them together.\n\nWe’re converging on a fascinating mix of easier to organize data (like [PageIndex](https://pageindex.ai/about)). I’m seeing patterns in harness design that mirror coding agents - hooks / evaluators that steer agents. Or “skills” that guide them a priori. Much like with coding - the agents eat the harnesses. When a successful pattern emerges, agents, in our case agentic search models, train to memorize these patterns.\n\nRetrieval also seems to be moving towards agent-centric approaches. Agents have [their own specific, unique retrieval patterns](https://hornet.dev/blog/this-is-what-agentic-retrieval-looks-like). We’re finally seeing late interaction + learned sparse retrieval approaches have their day from companies like [LightOn](http://lighton.ai) and [MixedBread](http://mixedbread.ai)\n\nWe’re swimming in an interesting retrieval primordial goop, what comes out may be an obvious combination - or look nothing like the component parts. Exciting times!\n\n### Enjoy softwaredoug in training course form!\n\n#### Starting May 18!\n\nSignup here -[http://maven.com/softwaredoug/cheat-at-search](http://maven.com/softwaredoug/cheat-at-search)\n\n[I hope you join me at](https://maven.com/softwaredoug/cheat-at-search)\n\n[Cheat at Search with Agents](https://maven.com/softwaredoug/cheat-at-search)to learn use agents in search. build better RAG and use LLMs in query understanding.", "url": "https://wpnews.pro/news/agentic-search-retrieval-harness-or-model", "canonical_source": "http://softwaredoug.com/blog/2026/06/08/three-kinds-of-agentic-search.html", "published_at": "2026-06-08 00:00:00+00:00", "updated_at": "2026-07-04 14:54:59.051932+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research", "natural-language-processing"], "entities": ["ChatGPT", "Google", "Bing", "Runciter Associates", "Lester Solbakken"], "alternates": {"html": "https://wpnews.pro/news/agentic-search-retrieval-harness-or-model", "markdown": "https://wpnews.pro/news/agentic-search-retrieval-harness-or-model.md", "text": "https://wpnews.pro/news/agentic-search-retrieval-harness-or-model.txt", "jsonld": "https://wpnews.pro/news/agentic-search-retrieval-harness-or-model.jsonld"}}