We tested SerpApi’s Google Search and Extract APIs with GPT-6.1 Sol on SealQA, a benchmark built around factual questions that are difficult to resolve through web search.
The agent correctly answered 65 of 111 SEAL-0 questions, achieving 58.56% accuracy, and 170 of 254 SEAL-Hard questions, achieving 66.93% accuracy.
SealQA tests how well models reason when search results contain conflicting, noisy, or unhelpful information. This makes it a useful benchmark for demanding agentic use cases, where high-quality search must surface the evidence needed to resolve contradictions, verify facts, and answer accurately.
SEAL-0 contains the benchmark’s hardest core questions. SEAL-Hard includes those same 111 questions plus 143 additional challenging questions.
This follows our earlier evaluation, where SerpApi achieved 97.6% accuracy on SimpleQA Verified.
These evaluations show how SerpApi’s Google Search and Extract APIs help agents tackle difficult questions by finding relevant sources, reading pages, cross-checking evidence, and making follow-up searches when the initial results leave gaps.
How we tested Google Search and Extract for AI agents #
We connected GPT-6.1 Sol, using medium reasoning, to SerpApi’s Google Search and Extract APIs.
The agent searched Google, read the returned evidence, and opened relevant pages when snippets were insufficient. It could make follow-up searches and compare sources before answering.
| Setting | Configuration |
|---|---|
| Answer model | GPT-6.1 Sol ( `gpt-6.1-sol` ), medium reasoning |
| Search | SerpApi Google Search, engine=google |
| Search output | Markdown, output=md |
| Page content | SerpApi's Extract API |
| Search and tool limits | Up to 10 searches and 24 total tool calls per question |
SERP data in Markdown
We used the Google Search API with Markdown output. The model received useful search content, including organic results, answer boxes, and AI Overviews when available.
The eval harness lets the model read the result pages and make follow-up queries.
Web page extraction with SerpApi Extract API
When search results did not provide enough evidence, the agent used the Extract API to read page content as Markdown.
The model could use this content to check details, resolve disagreements between sources, or identify a more specific search query. Fetched content was limited to 48,000 characters per page, with a maximum of 56,000 input tokens per model request.
Grading and exclusions
The grader (GPT-5.6 Sol) evaluated each prediction against the reference answer using high reasoning and the official SealQA rubric, with search and other tools disabled.
Our exposure review found one search snippet from the benchmark paper that repeated a question without revealing its answer or additional solving guidance. We retained that case in the reported SEAL-Hard score. Conservatively excluding it gives 66.80%, or 169 of 253 questions correct.
SEAL-0 and SEAL-Hard results #
| Benchmark | Questions | Correct answers | Accuracy |
|---|---|---|---|
| SEAL-0 | 111 | 65 | 58.56% |
| SEAL-Hard | 254 | 170 | 66.93% |
The higher SEAL-Hard score reflects the composition of the dataset: SEAL-0 is its more difficult subset. These are overlapping question sets, so the results cover 254 unique questions.
These results make SerpApi a strong choice for adding web search to your AI agents. Together with our SimpleQA Verified result, they demonstrate strong accuracy across straightforward factual questions and more challenging tasks involving conflicting or incomplete evidence.
What’s next? #
Add SerpApi to your agents and give them access to live web search:
Building a Python agent? Use SerpApi Search Tools to add ready-to-use search tools to OpenAI Agents, LangChain, Pydantic AI, CrewAI, and other supported frameworks.
Using an MCP-compatible client? Connect our MCP server to bring SerpApi search into Claude Desktop, Cursor, VS Code, and other AI tools.
Pair search with the Extract API when your agent needs to read the pages it finds.