{"slug": "equal-accuracy-unequal-evidence-search-apis-as-decision-surfaces-for-tool-using", "title": "Equal Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents", "summary": "A study comparing Brave, Tavily, and Firecrawl search APIs as retrieval layers for AI agents found that while all three achieve similar answer accuracy (25-26 out of 100 questions), they differ significantly in evidence economy and decision-surface properties, such as snippet quality and URL ranking. The research introduces metrics like surface contradiction-to-gold URL ratio, which varied from 0.92 to 2.59, arguing that search API choice is a retrieval-budget and policy decision rather than just a recall decision.", "body_md": "arXiv:2607.10198v1 Announce Type: new\nAbstract: Search APIs are the fundamental retrieval layer for many agents and are often their most frequently used tool. Traditional search APIs provide URLs, titles, and snippets that preview website contents. Because full-page retrieval is token-intensive, agent retrieval architectures increasingly use progressive disclosure: the agent first sees snippets and then chooses whether to fetch full pages. In such systems, search API performance is often evaluated primarily by answer accuracy. We argue that a commercial search API is better understood as a decision surface: the ranked snippets, URLs, and metadata that determine whether an agent answers immediately, searches again, or spends tokens opening pages. We test this claim with one frozen GPT-5.4 agent, two tools (search_web and fetch_page), and 100 questions from SEALQA-HARD, varying only the search provider (Brave, Tavily, Firecrawl). A Kimi-K2.6 oracle labels every content element visible to the agent (URL, title, snippet, and fetched page, when fetched), producing 6,869 valid per-URL judgments. We use an audited correct-answer label, semantic match, which preserves exact matches while accepting harmless formatting and naming variants. Under this measure, the providers remain close (25, 25, 26 / 100), but their evidence economies differ sharply: Brave offers gold-answer-rich snippets, Tavily concentrates gold-supporting URLs at rank 1, and Firecrawl is associated with broader exploration under this fixed agent policy. We also introduce a surface contradiction-to-gold URL ratio, which varies from 0.92 to 2.59. Provider choice is therefore a retrieval-budget and policy decision, not merely a recall decision.", "url": "https://wpnews.pro/news/equal-accuracy-unequal-evidence-search-apis-as-decision-surfaces-for-tool-using", "canonical_source": "https://arxiv.org/abs/2607.10198", "published_at": "2026-07-14 04:00:00+00:00", "updated_at": "2026-07-14 04:34:08.097173+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-research", "natural-language-processing", "ai-infrastructure"], "entities": ["Brave", "Tavily", "Firecrawl", "GPT-5.4", "Kimi-K2.6", "SEALQA-HARD"], "alternates": {"html": "https://wpnews.pro/news/equal-accuracy-unequal-evidence-search-apis-as-decision-surfaces-for-tool-using", "markdown": "https://wpnews.pro/news/equal-accuracy-unequal-evidence-search-apis-as-decision-surfaces-for-tool-using.md", "text": "https://wpnews.pro/news/equal-accuracy-unequal-evidence-search-apis-as-decision-surfaces-for-tool-using.txt", "jsonld": "https://wpnews.pro/news/equal-accuracy-unequal-evidence-search-apis-as-decision-surfaces-for-tool-using.jsonld"}}