New benchmark ranks search APIs for AI agents on quality, cost, and speed

Artificial Analysis released the Search Index, a benchmark ranking search API providers for AI agents on quality, cost, and speed, with Parallel, Firecrawl, and Parallel (turbo) achieving the best cost-performance mix. The index tests providers including Parallel, Exa, Firecrawl, You.com, Tavily, Keenable, and Brave using GPT-5.6 Luna in a standardized agent setup, finding that better search quality reduces total costs—Parallel Search (advanced) cut token use by over 40% versus Basic, lowering total cost to $0.084 per task from $0.11.

New benchmark ranks search APIs for AI agents on quality, cost, and speed Artificial Analysis has released the "Search Index," a benchmark that measures how well search API providers work for AI agents across quality, cost, and speed. The initial lineup includes Parallel, Exa, Firecrawl, You.com, Tavily, Keenable, and Brave. Each one is tested with the same model GPT-5.6 Luna in a standardized agent setup. Only the search provider changes. The agent runs on Stirrup https://stirrup.artificialanalysis.ai/ , an open-source framework from Artificial Analysis, and gets 25 runs per task to search and pull up web pages. The index combines three equally weighted benchmarks. DeepSearchQA has 900 research questions, each requiring multiple search queries. A BrowseComp subset tests 200 hard-to-find facts that need multi-step browsing. AA-Omniscience covers 600 questions across six knowledge domains. A tool-free baseline, where the model answers on its own, provides the comparison point. Better search quality also lowers total costs. The model uses fewer tokens when it gets good results up front. With Parallel Search advanced , token use drops by over 40 percent compared to the Basic version. Per-task search costs go up, but total cost comes in lower $0.084 vs. $0.11 . Raw speed per query doesn't always mean faster results overall. Parallel Search turbo clocks the shortest response time per query 0.51 seconds vs. 1.03 seconds for Basic , but its lower quality 67 vs. 73 forces the agent to run more passes. Total time per task winds up about the same. Artificial Analysis says Parallel, Firecrawl, and Parallel turbo hit the best mix of cost and performance. Other providers can apply https://artificialanalysis.ai/contact to join the benchmark. The full methodology https://artificialanalysis.ai/methodology/search-api is public. AI News Without the Hype – Curated by Humans Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section. Subscribe now via X https://x.com/ArtificialAnlys/status/2089755262915936661/photo/1