{"slug": "serpapi-achieves-58-56-accuracy-on-seal-0-and-66-93-on-seal-hard", "title": "SerpApi Achieves 58.56% Accuracy on SEAL-0 and 66.93% on SEAL-Hard", "summary": "SerpApi's Google Search and Extract APIs, paired with GPT-6.1 Sol at medium reasoning, answered 65 of 111 SEAL-0 questions correctly for 58.56% accuracy and 170 of 254 SEAL-Hard questions correctly for 66.93% accuracy, SerpApi reported. The agent could run up to 10 searches and 24 total tool calls per question, reading pages through the Extract API at up to 48,000 characters per page and 56,000 input tokens per request, with GPT-5.6 Sol grading against the official SealQA rubric. SerpApi said conservatively excluding one search snippet from the benchmark paper that repeated a question without revealing its answer gives 66.80%, or 169 of 253 questions, on SEAL-Hard; the results follow an earlier 97.6% score on SimpleQA Verified.", "body_md": "We tested SerpApi’s Google Search and Extract APIs with GPT-6.1 Sol on SealQA, a benchmark built around factual questions that are difficult to resolve through web search.\n\nThe agent correctly answered **65 of 111 SEAL-0 questions**, achieving **58.56% accuracy**, and **170 of 254 SEAL-Hard questions**, achieving **66.93% accuracy**.\n\n[SealQA](https://arxiv.org/abs/2506.01062) tests how well models reason when search results contain conflicting, noisy, or unhelpful information. This makes it a useful benchmark for demanding agentic use cases, where high-quality search must surface the evidence needed to resolve contradictions, verify facts, and answer accurately.\n\nSEAL-0 contains the benchmark’s hardest core questions. SEAL-Hard includes those same 111 questions plus 143 additional challenging questions.\n\nThis follows our earlier evaluation, where SerpApi achieved 97.6% accuracy on SimpleQA Verified. \n\nThese evaluations show how SerpApi’s Google Search and Extract APIs help agents tackle difficult questions by finding relevant sources, reading pages, cross-checking evidence, and making follow-up searches when the initial results leave gaps.\n\n## How we tested Google Search and Extract for AI agents\n\nWe connected GPT-6.1 Sol, using medium reasoning, to SerpApi’s Google Search and Extract APIs.\n\nThe agent searched Google, read the returned evidence, and opened relevant pages when snippets were insufficient. It could make follow-up searches and compare sources before answering.\n\n| Setting | Configuration | \n|---|---|\n| Answer model | GPT-6.1 Sol ( `gpt-6.1-sol` ), medium reasoning | \n| Search | SerpApi Google Search, `engine=google` | \n| Search output | Markdown, `output=md` | \n| Page content | SerpApi's Extract API | \n| Search and tool limits | Up to 10 searches and 24 total tool calls per question | \n\n### SERP data in Markdown\n\nWe used the [Google Search API](https://serpapi.com/search-api) with Markdown output. The model received useful search content, including organic results, answer boxes, and AI Overviews when available.\n\nThe eval harness lets the model read the result pages and make follow-up queries.\n\n### Web page extraction with SerpApi Extract API\n\nWhen search results did not provide enough evidence, the agent used the [Extract API](https://serpapi.com/extract-api.md) to read page content as Markdown.\n\nThe model could use this content to check details, resolve disagreements between sources, or identify a more specific search query. Fetched content was limited to 48,000 characters per page, with a maximum of 56,000 input tokens per model request.\n\n### Grading and exclusions\n\nThe grader (GPT-5.6 Sol) evaluated each prediction against the reference answer using high reasoning and the official SealQA rubric, with search and other tools disabled.\n\nOur exposure review found one search snippet from the benchmark paper that repeated a question without revealing its answer or additional solving guidance. We retained that case in the reported SEAL-Hard score. Conservatively excluding it gives **66.80%**, or **169 of 253 questions correct**.\n\n## SEAL-0 and SEAL-Hard results\n\n| Benchmark | Questions | Correct answers | Accuracy | \n|---|---|---|---|\n| **SEAL-0** | 111 | 65 | **58.56%** | \n| **SEAL-Hard** | 254 | 170 | **66.93%** | \n\nThe higher SEAL-Hard score reflects the composition of the dataset: SEAL-0 is its more difficult subset. These are overlapping question sets, so the results cover **254 unique questions**.\n\nThese results make SerpApi a strong choice for adding web search to your AI agents. Together with our SimpleQA Verified result, they demonstrate strong accuracy across straightforward factual questions and more challenging tasks involving conflicting or incomplete evidence.\n\n## What’s next?\n\nAdd SerpApi to your agents and give them access to live web search:\n\n**Building a Python agent?** Use [SerpApi Search Tools](https://serpapi.github.io/serpapi-search-tools-python/) to add ready-to-use search tools to OpenAI Agents, LangChain, Pydantic AI, CrewAI, and other supported frameworks.\n\n**Using an MCP-compatible client?** Connect our [MCP server](https://serpapi.com/integrations/mcp) to bring SerpApi search into Claude Desktop, Cursor, VS Code, and other AI tools.\n\nPair search with the [Extract API](https://serpapi.com/extract-api.md) when your agent needs to read the pages it finds.", "url": "https://wpnews.pro/news/serpapi-achieves-58-56-accuracy-on-seal-0-and-66-93-on-seal-hard", "canonical_source": "https://serpapi.com/blog/serpapi-achieves-58-56-accuracy-on-seal-0-and-66-93-on-seal-hard/", "published_at": "2026-11-06 10:58:32+00:00", "updated_at": "2026-11-06 11:20:29.622489+00:00", "lang": "en", "topics": ["ai-search", "ai-agents", "ai-tools", "large-language-models"], "entities": ["SerpApi", "GPT-6.1 Sol", "GPT-5.6 Sol", "SealQA", "SEAL-0", "SEAL-Hard", "SimpleQA Verified", "Google Search"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/serpapi-achieves-58-56-accuracy-on-seal-0-and-66-93-on-seal-hard", "markdown": "https://wpnews.pro/news/serpapi-achieves-58-56-accuracy-on-seal-0-and-66-93-on-seal-hard.md", "text": "https://wpnews.pro/news/serpapi-achieves-58-56-accuracy-on-seal-0-and-66-93-on-seal-hard.txt", "jsonld": "https://wpnews.pro/news/serpapi-achieves-58-56-accuracy-on-seal-0-and-66-93-on-seal-hard.jsonld"}}