{"slug": "serpapi-achieves-97-6-accuracy-on-simpleqa-verified", "title": "SerpApi Achieves 97.6% Accuracy on SimpleQA Verified", "summary": "SerpApi reported 97.6% accuracy on the SimpleQA Verified benchmark, answering 937 of 960 questions correctly after excluding 40 responses where search results exposed benchmark answers or evaluation hints. The run connected SerpApi's Google Search API and Extract API to GPT-6.1 Sol with low reasoning effort, and the result ties Tavily Search's published 97.6% for joint #1 among the compared providers, ahead of Parallel Advanced at 97.0%. SerpApi tested only its own system; the Tavily and Parallel scores come from those companies' published evaluation reports.", "body_md": "[SimpleQA Verified](https://arxiv.org/abs/2509.07968) is a benchmark from Google DeepMind and Google Research for testing how accurately AI models answer factual questions. It contains 1,000 questions, selected from OpenAI's original SimpleQA dataset after checking reference answers, removing similar questions, and balancing topics.\n\nWe wanted to see how Google search results and web page content help AI agents search the web and answer factual questions. So we connected SerpApi's [Google Search API](https://serpapi.com/search-api) and [Extract API](https://serpapi.com/extract-api) to GPT-6.1 Sol, using low reasoning effort.\n\nAfter reviewing the saved model inputs, we excluded 40 questions where we detected exposure to public benchmark answers or guidance. The model answered **937 of the remaining 960 questions correctly**, giving us **97.6% accuracy**.\n\nThat puts SerpApi **joint #1 by reported accuracy among the results below**, matching the 97.6% in [Tavily's published evaluation](https://www.tavily.com/blog/tavily-ranks-1-on-sealqa-and-simpleqa).\n\nWe tested only SerpApi in the eval; the other providers' scores come from [Tavily's](https://www.tavily.com/blog/tavily-ranks-1-on-sealqa-and-simpleqa) and [Parallel's](https://parallel.ai/articles/best-web-search-api-for-ai-applications) published evaluation reports.\n\n## How we tested Google Search and Extract for AI agents\n\nA web search API helps an AI agent find sources, but a search result may not contain enough detail to answer the question. Our setup let the agent use Google SERP data to find relevant pages, read them with Extract, and search again if it needed more information.\n\nThe model decided when to search and when to read a page, within the limits below.\n\n| Setting | What we used | \n|---|---|\n| Answer model | GPT-6.1 Sol ( `gpt-6.1-sol` ), low reasoning | \n| Search | SerpApi Google Search API, `engine=google` | \n| Search output | Markdown, using `output=md` | \n| Page content | SerpApi Extract API | \n| Search & tool call limit | Multiple searches and fetches allowed per question | \n| Grader | GPT-5.6 Sol, high reasoning | \n\n### Google SERP data in Markdown\n\nWe used the regular [SerpApi Google Search API](https://serpapi.com/search-api) with [Markdown output](https://serpapi.com/markdown-output). This gave the model the useful content returned by Google, including organic results, answer boxes, and AI Overviews when available. \n\nWe removed metadata and pagination details before passing the results to the model. Searches used `no_cache=true` to fetch fresh results for every run.\n\n### Web page extraction with SerpApi Extract API\n\nWhen the model needed more detail from a source, it could call the [SerpApi Extract API](https://serpapi.com/extract-api). We used `engine=extract` with `mode=scrape` and returned the page's Markdown through `include_markdown=true` and `json_restrictor=text_content.markdown`.\n\nThe model could then use that page content to answer or make another search. We limited each page fetch to 48,000 characters and each model input to 56,000 tokens.\n\n### Grading and exclusions\n\nAfter the final answers were saved, the grader reviewed each question, reference answer, and prediction without access to search tools.\n\nWe detected benchmark leaks by reviewing the search results and page content shown to the model. We excluded 40 responses where search results exposed benchmark answers or evaluation hints.\n\n## SimpleQA Verified results and sources\n\nTo put our result in context, we've included the SimpleQA Verified scores published by Tavily and Parallel below.\n\n| Search provider | Accuracy | Source | \n|---|---|---|\n| SerpApi Google + Extract | **97.6%** | Our run, 937/960 after exposure exclusions | \n| Tavily Search | 97.6% | [Tavily's evaluation](https://www.tavily.com/blog/tavily-ranks-1-on-sealqa-and-simpleqa) | \n| Parallel Advanced | 97.0% | [Parallel's evaluation](https://parallel.ai/articles/best-web-search-api-for-ai-applications) | \n| Perplexity Search | 95.0% | [Parallel's evaluation](https://parallel.ai/articles/best-web-search-api-for-ai-applications) | \n| Exa Auto | 91.0% | [Parallel's evaluation](https://parallel.ai/articles/best-web-search-api-for-ai-applications) | \n\nOur 97.6% matches Tavily’s published score and is the joint-highest reported accuracy in this comparison. Our agent used Google search data and SerpApi Extract to find sources, read pages, and make follow-up searches when needed.\n\nThese scores come from separate evaluations with different setups, so they provide context rather than a direct comparison under identical conditions.\n\n## What's next?\n\nWe are also evaluating [SealQA](https://arxiv.org/abs/2506.01062), which includes questions with conflicting or unhelpful search evidence. Our completed Seal-0 run scored **58.56% (65/111)** using GPT-6.1 Sol at medium reasoning with SerpApi Google and Extract.\n\nWe'll share the full SealQA results and methodology in a follow-up post.", "url": "https://wpnews.pro/news/serpapi-achieves-97-6-accuracy-on-simpleqa-verified", "canonical_source": "https://serpapi.com/blog/serpapi-achieves-97-6-accuracy-on-simpleqa-verified/", "published_at": "2026-10-16 08:46:51+00:00", "updated_at": "2026-10-16 09:09:34.930621+00:00", "lang": "en", "topics": ["ai-search", "ai-agents", "artificial-intelligence", "large-language-models", "ai-tools"], "entities": ["SerpApi", "SimpleQA Verified", "Google DeepMind", "Google Research", "GPT-6.1 Sol", "Tavily", "Parallel", "Perplexity"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/serpapi-achieves-97-6-accuracy-on-simpleqa-verified", "markdown": "https://wpnews.pro/news/serpapi-achieves-97-6-accuracy-on-simpleqa-verified.md", "text": "https://wpnews.pro/news/serpapi-achieves-97-6-accuracy-on-simpleqa-verified.txt", "jsonld": "https://wpnews.pro/news/serpapi-achieves-97-6-accuracy-on-simpleqa-verified.jsonld"}}