SerpApi Achieves 58.56% Accuracy on SEAL-0 and 66.93% on SEAL-Hard

SerpApi's Google Search and Extract APIs, paired with GPT-6.1 Sol at medium reasoning, answered 65 of 111 SEAL-0 questions correctly for 58.56% accuracy and 170 of 254 SEAL-Hard questions correctly for 66.93% accuracy, SerpApi reported. The agent could run up to 10 searches and 24 total tool calls per question, reading pages through the Extract API at up to 48,000 characters per page and 56,000 input tokens per request, with GPT-5.6 Sol grading against the official SealQA rubric. SerpApi said conservatively excluding one search snippet from the benchmark paper that repeated a question without revealing its answer gives 66.80%, or 169 of 253 questions, on SEAL-Hard; the results follow an earlier 97.6% score on SimpleQA Verified.

We tested SerpApi’s Google Search and Extract APIs with GPT-6.1 Sol on SealQA, a benchmark built around factual questions that are difficult to resolve through web search. The agent correctly answered 65 of 111 SEAL-0 questions , achieving 58.56% accuracy , and 170 of 254 SEAL-Hard questions , achieving 66.93% accuracy . SealQA https://arxiv.org/abs/2506.01062 tests how well models reason when search results contain conflicting, noisy, or unhelpful information. This makes it a useful benchmark for demanding agentic use cases, where high-quality search must surface the evidence needed to resolve contradictions, verify facts, and answer accurately. SEAL-0 contains the benchmark’s hardest core questions. SEAL-Hard includes those same 111 questions plus 143 additional challenging questions. This follows our earlier evaluation, where SerpApi achieved 97.6% accuracy on SimpleQA Verified. These evaluations show how SerpApi’s Google Search and Extract APIs help agents tackle difficult questions by finding relevant sources, reading pages, cross-checking evidence, and making follow-up searches when the initial results leave gaps. How we tested Google Search and Extract for AI agents We connected GPT-6.1 Sol, using medium reasoning, to SerpApi’s Google Search and Extract APIs. The agent searched Google, read the returned evidence, and opened relevant pages when snippets were insufficient. It could make follow-up searches and compare sources before answering. | Setting | Configuration | |---|---| | Answer model | GPT-6.1 Sol gpt-6.1-sol , medium reasoning | | Search | SerpApi Google Search, engine=google | | Search output | Markdown, output=md | | Page content | SerpApi's Extract API | | Search and tool limits | Up to 10 searches and 24 total tool calls per question | SERP data in Markdown We used the Google Search API https://serpapi.com/search-api with Markdown output. The model received useful search content, including organic results, answer boxes, and AI Overviews when available. The eval harness lets the model read the result pages and make follow-up queries. Web page extraction with SerpApi Extract API When search results did not provide enough evidence, the agent used the Extract API https://serpapi.com/extract-api.md to read page content as Markdown. The model could use this content to check details, resolve disagreements between sources, or identify a more specific search query. Fetched content was limited to 48,000 characters per page, with a maximum of 56,000 input tokens per model request. Grading and exclusions The grader GPT-5.6 Sol evaluated each prediction against the reference answer using high reasoning and the official SealQA rubric, with search and other tools disabled. Our exposure review found one search snippet from the benchmark paper that repeated a question without revealing its answer or additional solving guidance. We retained that case in the reported SEAL-Hard score. Conservatively excluding it gives 66.80% , or 169 of 253 questions correct . SEAL-0 and SEAL-Hard results | Benchmark | Questions | Correct answers | Accuracy | |---|---|---|---| | SEAL-0 | 111 | 65 | 58.56% | | SEAL-Hard | 254 | 170 | 66.93% | The higher SEAL-Hard score reflects the composition of the dataset: SEAL-0 is its more difficult subset. These are overlapping question sets, so the results cover 254 unique questions . These results make SerpApi a strong choice for adding web search to your AI agents. Together with our SimpleQA Verified result, they demonstrate strong accuracy across straightforward factual questions and more challenging tasks involving conflicting or incomplete evidence. What’s next? Add SerpApi to your agents and give them access to live web search: Building a Python agent? Use SerpApi Search Tools https://serpapi.github.io/serpapi-search-tools-python/ to add ready-to-use search tools to OpenAI Agents, LangChain, Pydantic AI, CrewAI, and other supported frameworks. Using an MCP-compatible client? Connect our MCP server https://serpapi.com/integrations/mcp to bring SerpApi search into Claude Desktop, Cursor, VS Code, and other AI tools. Pair search with the Extract API https://serpapi.com/extract-api.md when your agent needs to read the pages it finds.