Build a web-aware agent with OpenAI Agents SDK and Zenrows

A developer guide demonstrates how to build a web-aware agent by combining OpenAI's Agents SDK with Zenrows' scraping API. The approach registers Zenrows Fetch as a function tool alongside OpenAI's WebSearchTool, enabling the agent to retrieve live, JavaScript-rendered page content when search results are stale or incomplete. The tutorial includes code examples and addresses limitations of built-in search for dynamic content.

This article was originally published on the Zenrows blog. Read the original here: https://www.zenrows.com/blog/web-aware-agent-openai-agents-sdk-zenrows This guide shows you how to register Zenrows Fetch as a @function tool next to WebSearchTool in a single OpenAI Agents SDK agent, so the model routes between discovery and live-page retrieval on its own each turn. You need Python 3.9+, an OpenAI API key, and a Zenrows API key. All the code is on GitHub https://github.com/ZenRows/web-aware-agent-openai-agents-sdk-zenrows . Install dependencies: python3 -m pip install openai-agents python-dotenv Add both keys to a .env file and add it to .gitignore : OPENAI API KEY=your openai api key here ZENROWS API KEY=your zenrows api key here Load them in your script: python from dotenv import load dotenv load dotenv WebSearchTool is a hosted tool that runs on OpenAI's infrastructure. Enabling it takes one line. python import asyncio from dotenv import load dotenv from agents import Agent, Runner, WebSearchTool load dotenv agent = Agent name="Product Researcher", instructions= "You research products and web pages. " "Report exactly what you find, and state plainly when " "specific information is missing from your results." , tools= WebSearchTool , async def main : result = await Runner.run agent, "Which laptops did Apple release most recently, and where can I buy them?", print result.final output asyncio.run main This works for broad discovery queries. The agent returns relevant pages with source links. The gap appears on live or protected targets. Built-in search returns OpenAI's cached copy of a page, not the current version. For documentation, that's usually fine. For prices, stock levels, or anything that changes frequently, the cached copy lags. Many pages ship an HTML shell and fill in prices, inventory, and reviews via JavaScript at runtime. The search index stores the shell. The data you want was never captured. python import asyncio from dotenv import load dotenv from agents import Agent, Runner, WebSearchTool load dotenv agent = Agent name="Product Researcher", instructions= "You retrieve product details from web pages. " "Report exactly what you find, and say so plainly if content is missing." , tools= WebSearchTool , async def main : result = await Runner.run agent, "What is the current price and stock status of " "https://www.amazon.com/dp/B0GR1JKMBV/ref=fs a mbt2 us1?th=1", print result.final output asyncio.run main The agent finds the page. It returns the product title and a clear statement that the price wasn't accessible: I visited the Amazon product page for ASIN B0GR1JKMBV... The page displays: "To see product details, add this item to your cart." Therefore I could not retrieve the current price or availability. python import os import requests from agents import function tool @function tool def fetch page content url: str - str: """Fetch the full, current content of a specific web page as Markdown. Use this when you need the complete content of a known URL rather than a search result summary, such as live prices, stock levels, or data that loads via JavaScript after the page opens. Args: url: The full URL of the page to retrieve. """ response = requests.get "https://api.zenrows.com/v1/", params={ "url": url, "apikey": os.getenv "ZENROWS API KEY" , mode=auto lets Zenrows pick the right settings per site "mode": "auto", "response type": "markdown", }, timeout=90, return the error as a string so the agent can react instead of crashing if response.status code = 200: return f"Zenrows returned {response.status code} for {url}" return response.text Four things to note: @function tool mode=auto response type=markdown python import asyncio import os import requests from dotenv import load dotenv from agents import Agent, Runner, WebSearchTool, function tool load dotenv @function tool def fetch page content url: str - str: """Fetch the full, current content of a specific web page as Markdown. Use this when you need the complete content of a known URL rather than a search result summary, such as live prices, stock levels, or data that loads via JavaScript after the page opens. Args: url: The full URL of the page to retrieve. """ response = requests.get "https://api.zenrows.com/v1/", params={ "url": url, "apikey": os.getenv "ZENROWS API KEY" , "mode": "auto", "response type": "markdown", }, timeout=90, if response.status code = 200: return f"Zenrows returned {response.status code} for {url}" return response.text the model routes between both tools from their descriptions alone agent = Agent name="Web-Aware Researcher", instructions= "You research products on the web. " "Use web search to find the product page URL. " "Then call fetch page content on that URL and report the price " "and stock status from the fetched page content. " "State which tool each figure came from." , tools= WebSearchTool , fetch page content , def print trace items : for item in items: raw = getattr item, "raw item", None name = getattr raw, "name", None or getattr raw, "type", "" print f" {item.type} {name}" if item.type == "tool call item" and name == "fetch page content": print " args:", getattr raw, "arguments", "" if item.type == "tool call output item": out = str getattr item, "output", "" print f" output: {len out } chars" print f" body: {out :500 }" async def main : result = await Runner.run agent, "Find the PriceOye page for the iPhone 17 Pro " "and report its current price and stock status.", print trace result.new items print "\n---\n" print result.final output asyncio.run main The trace shows the two-step routing. Turn one: web search call for discovery. Turn two: fetch page content with the URL it found. tool call item web search call message output item message tool call item fetch page content args: {"url":"https://priceoye.pk/mobiles/apple/apple-iphone-17-pro"} tool call output item --- Here are the details for the iPhone 17 Pro from its PriceOye product page: Current Price: Rs 471,999 Stock Status: Only 1 left in stock Price and stock status are directly extracted from the fetched page content functions.fetch page content tool . No conditionals. No orchestration code. The routing lives entirely in the tool descriptions. Use WebSearchTool for discovery: finding pages, open-ended research, answering questions from indexed public content. Use Zenrows as the retrieval layer when you have a specific URL and need the live contents — protected targets, JavaScript-rendered pages, anything where freshness matters. For structured field extraction instead of full-page Markdown, use Zenrows Extract https://docs.zenrows.com/extract/introduction . For high-volume or scheduled retrieval across many URLs, use Zenrows Batch https://www.zenrows.com/products/batch . The same pattern carries over to multi-agent setups: building a web research multi-agent system with AG2 and Zenrows https://www.zenrows.com/blog/web-research-multi-agent-ag2-zenrows . And to smolagents, under a @tool decorator: Zenrows smolagents guide https://www.zenrows.com/blog/zenrows-smolagents . You now have an agent with two retrieval paths. WebSearchTool handles discovery. Zenrows reads the full live page once the URL is known, including JavaScript-rendered and anti-bot-defended targets. The model routes between them from the tool descriptions alone.