Deterministic Data: The Case for Web Scraping Over LLM Web Search Tools

In “Why Your RAG Pipelines Are Only as Good as Your Data Acquisition Layer”, I discussed that web search tools have some limitations and do not replace web scraping.

To help you understand that better, I decided to write a dedicated article on that matter. This article explains what tools are for LLMs, how tool calls work, and presents the limitations of web search tools. Also, it presents two practical sections where I test commonly used models and their web search tools. This will help you visually understand the whys behind the statements.

Let’s dive into it!

Give your AI a web data layer – Decodo’s Web Scraping API turns any site into clean, structured data your models can actually use.

What are Tools in LLMs and How Does Tool Calling Work #

Large Language Models are stateless systems. This means that their knowledge is frozen at a training cutoff, so they have no familiarity with the world beyond them when training is completed. This architectural constraint is a deliberate design boundary that keeps inference deterministic and safe. However, it creates a profound usability gap: a model that cannot act on the “current version of the world” is only marginally useful. This is the exact gap that tools were built to close.

Defining Tools: A Formal Perspective

In the context of LLMs, a tool is a formally declared, schema-bound capability that the model can invoke. But pay attention here: LLMs can invoke tools, but cannot execute them. This distinction is essential because the model does not run code: it produces a structured intent, a precise, machine-readable declaration that it wishes a specific function to be called with specific arguments.

This means that execution authority remains entirely with the host application or orchestration layer. For this reason, a tool can be best defined as a typed contract between the model and its runtime environment: it specifies a function name, a set of typed input parameters, and a natural-language description that the model uses to reason about when and why to invoke it.

A canonical tool definition, as described in the OpenAI function-calling specification, looks structurally like the following example:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Retrieve the current weather for a given city. Use this when the user asks about live weather conditions.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and country, e.g. 'Rome, Italy'"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"]
                    }
                },
                "required": ["location"]
            }
        }
    }
]

This schema is injected into the model’s context, typically as a structured system-level payload, before inference begins. So, the model does not “learn” tools: it reads them at runtime, the same way a human engineer reads an API specification before writing an integration.

AI can help you make sense of web data, but first you need to collect it. For teams building controlled, repeatable scraping workflows, anyIP provides the access layer with high-quality residential and mobile proxies worldwide.

The Tool Calling Lifecycle

You can use LLMs via chat and via APIs, depending on your preference and needs. Regardless of the use, LLMs have the possibility to call tools. This paragraph presents the lifecycle of explicitly calling tools via APIs, from the first to the last step.

Step #1: Context Injection

The developer serializes all available tool definitions into the model’s context alongside the system prompt and conversation history. Modern APIs expose a dedicated tools parameter for this purpose, keeping tool schemas structurally separated from the natural-language turns—though under the hood, they are always rendered into the token stream the model attends to.

Step #2: Model Reasoning and Tool Selection

During inference, the model performs implicit reasoning over the user’s intent and the available tool schemas. But note that this works the same as the next-token prediction mechanism driving all LLM outputs. The only difference is that now it is conditioned on a richer context that includes typed function signatures.

At this point, the model must determine: “Is a tool call warranted here? If so, which tool? What arguments should be passed?“.

This decision is governed entirely by the attention mechanism attending to both the user query and the tool descriptions. This is precisely why the description field in a tool schema is a signal that directly influences the model’s routing behavior.

Step #3: Structured Output Generation

When the model decides to invoke a tool, it does not generate a natural-language response. Instead, it emits a tool call object. This is a structured payload containing the resolved function name and a JSON-serialized argument map. In the OpenAI API format, this surfaces as a tool_calls array on the assistant message object, with finish_reason set to “tool_calls” rather than “stop”, signaling the host application that the turn is not complete:

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_abc123",
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "arguments": "{\"location\": \"Rome, Italy\", \"unit\": \"celsius\"}"
      }
    }
  ]
}

Note that arguments is a JSON-encoded string, not a nested object. This is intentional because it decouples the model’s output format from the host’s parsing logic and makes partial streaming of arguments tractable.

Step #4: Host-side Execution

The orchestration layer intercepts the tool_calls payload, deserializes the arguments, validates them against the declared schema, and dispatches the call to the actual function implementation. This may be a REST API call, a database query, a local Python function, a subprocess, or any arbitrary computation, depending on the case. Note that the model has no visibility into this execution layer: it simply waits until the process is completed.

Step #5: Result Injection and Continuation

Once execution completes, the host appends the function result back into the conversation as a tool message, which is a distinct message role (“role”: “tool”) that carries the tool_call_id for correlation and the result payload as a string:

{
  "role": "tool",
  "tool_call_id": "call_abc123",
  "content": "{\"temperature\": 28, \"condition\": \"Sunny\", \"humidity\": 45}"
}

The full conversation history is then fed back into the model for a continuation inference pass. Only at this point does the model generate the final response in natural language. This way, the response is grounded in that the model could not have accessed it from its weights alone.

Start your scraping journey with Byteful: 10GB New Customer Trial | Use TWSC for 15% OFF | $1.75/GB Residential Data | ISP Proxies in 15+ Countries

Web Search Tools Do Not Substitute Traditional Web Scraping: Here’s Why #

One of the most common tools surfaced to LLMs is the web search tool, and its use is understandable: it is an obvious remedy to the knowledge cutoff problem. However, practitioners often make a critical architectural assumption: they treat the web search tool as a general-purpose solution for web-based information retrieval.

This assumption is a huge mistake, because web search and web scraping are different mechanisms operating at different layers of the information stack. This section discusses the reasons behind the fact that conflating the two is wrong.

What a Web Search Tool Actually Does

As previously introduced, when a model invokes a tool, the orchestration layer dispatches the call to the actual function implementation. When it comes to web search tools, what typically happens is that the underlying LLM’s implementation delegates to a search engine API.

Now, depending on the usage (aka, you are using the LLM via chat or via API), and also depending on the actual type of web search tool integrated into the LLM, after accessing the web, the tool can return one or more of the following:

• Page titles
• URLs
• Short snippets/descriptions (often 160–3000+ chars depending on the provider)
• Metadata (e.g., publication dates)
• The entire page content or rich excerpts, pre-formatted for LLM consumption (but this only happens for specific web search tools)

Now, the point that is interesting for scraping professionals is that the output of the web search tool feeds the LLM, and it responds to the user query using that output. This means that you can write a query to an LLM, and it can confidently hallucinate a wide response that is based only on the data retrieved in the URL and page title.

The point is that you actually have no clue about what has happened under the hood. For example, you may be using an external web search tool called via APIs, thinking you have more control over the process. But the tool may fail at fetching the whole page content because of anti-bots. So, it retrieves only metadata, page title, and URL, and the model hallucinates the response based on that.

Yes, this is very bad news…

The Semantic Query Constraint of Web Search

Another thing to consider is that web search tools operate in query space. This means that the model generates a natural-language or query string, the search API processes it, and the response is a ranked list of results (regardless of the actual access to the complete web content, which comes in a second step). This is basically a semantic approximation of the information that users need, mediated by the search engine’s interpretation of query intent. In other words: the result you obtain is basically a black box, because no one knows how search engines rank the results.

This has a direct consequence on information precision. When a model needs to answer a question that requires information from a specific document, a table on a specific page, a structured dataset at a known URL, or a real-time feed from a specific API, a search query is a lossy and unreliable retrieval mechanism. What happens in this case is that the model must express its information need as a query string, the search engine must interpret that string, rank the relevant URLs, and return snippets, and only then can the model reason over that output. At each step in this chain, there is information loss and potential for a mismatch between the actual information needed and what is returned.

Web scraping, in contrast, operates in URL space. The retrieval is deterministic given a target URL: the scraper issues a request to a specific resource and returns its content. End of the story.

This is the correct retrieval primitive for any workflow where the information source is known in advance or where the crawl path can be programmatically determined.

Why Web Search Tools Cannot Serve as a RAG Retrieval Layer

This is perhaps the most consequential architectural confusion in modern LLM system design, and it deserves specific treatment. It is also highly tied to the fact that web search tools operate in the query space.

Retrieval-Augmented Generation, in its canonical form, is a two-phase architecture:

• In the offline phase, a corpus of documents is chunked, embedded into a dense vector space using an embedding model, and stored in a vector database.
• In the online phase, a query is embedded into the same vector space, a nearest-neighbor search retrieves the most semantically relevant chunks, and those chunks are injected into the model’s context window as grounding material.

The key property of this architecture is that the retrieval operates over a controlled, pre-indexed, owned corpus, and the retrieved content is the full, raw text of the original document chunks, not a summary of them.

Web search tools, as discussed above, operate on an entirely different substrate. So, there is no owned corpus, no embedding model under your control, no vector store you can inspect, audit, or fine-tune.

The HTML-to-Markdown Pipeline: The Correct Content Format for LLM Ingestion

Even setting aside all the preceding architectural arguments, there is a final, purely pragmatic reason why web scraping provides a qualitatively superior content retrieval primitive for LLM-based systems. Web scraping ****gives you the raw document, which you can transform into any downstream representation you need.

A web scraper that issues a request to a target URL receives the full HTTP response body. This raw HTML is the ground truth of the page’s content. From it, you can extract structured data via DOM traversal using libraries like BeautifulSoup or lxml, execute JavaScript to capture dynamically rendered content via headless browsers, parse semantic HTML elements with full structural fidelity, and apply custom filtering to strip navigation chrome, cookie banners, and boilerplate that would otherwise pollute the model’s context.

Another important point to consider is that raw HTML can be converted to Markdown. Markdown is the optimal serialization format for LLM context injection for a set of concrete reasons:

• It preserves document hierarchy.
• It encodes list and table structure without the verbose tag overhead of HTML.
• It represents code blocks with fenced syntax that models trained on large code corpora recognize and parse with high reliability.
• It strips rendering-irrelevant attributes and style information that consume context tokens without contributing to informational density.
• It is the format in which the vast majority of LLM training corpora are natively expressed.

For such reasons, a model reasoning over a Markdown-serialized document is, in a certain way, reasoning in its native register.

Testing Web Search Tools #

In this section, you will understand the whys behind what you read above. You will see the tests I performed and the results I obtained. Specifically, the tool calls will be performed via chat. This is because the (partial) results of what happens under the hood are easier to show, and these are useful for your understanding.

This section has two different testing sub-sections:

• In the first one, you’ll see what happens when prompting different LLMs, asking them to search for specific sources.
• In the second one, you’ll see what happens when you specifically ask an LLM to open a specific URL.

DISCLAIMER: Note that native tool calling (calling tools via the chat) is more probabilistic and less controllable than invoking them via APIs. This is because the exact internal mechanisms by which specific providers implement native tool invocation policies are not publicly documented. However, the examples still fit the needs of this article.

Testing Examples Type #1: Asking LLMs to Search for Consolidating Trends

Let’s see how different LLMs react when asked to search for specific sources.

Claude

Let’s start the tests by prompting Claude with the following:

Tell me about recent analyst reports and news articles about consolidation trends in the AI industry.

Here’s the chat image:

Below is what happens:

As you can see, Claude searches the web. And it does so with two different mechanisms, based on the prompt:

• The first one actually searches for AI industry consolidating trends. That is basically exactly what it has been prompted to do.
• The second one searches for AI startup acquisitions in June…sure, startup acquisitions can describe trends. But how can June be taken as a representation of the whole 2026, until now? This is basically an assumption that the model made itself under the hood.

Now, let’s try to slightly change the prompt like so:

Find recent analyst reports and news articles about consolidation trends in the AI industry.

Here’s what happens:

Now, the results about “AI industry consolidation trends” are the same, but the search and the results in the second box are completely different. And the prompt only changed from “tell me” to “find”…

So, as you can see, the model makes its own assumptions. And this influences the response, of course.

ChatGPT

ChatGPT will use different web sources, given the same prompts as the two above. Below is the result for the first prompt:

And then for the second prompt:

This means that, if you want to make a benchmark between ChatGPT’s and Claude’s responses, you will have to account for the fact that they use different sources. This is exactly the point stated previously: using the research engine is a black box that users can not manage.

Also, as the image shows, the results are not based on two different web searches. Actually, the model does not show if it uses one or more searches.

Perplexity

Perplexity’s approach is a mix between the ones from Claude and ChatGPT:

• It uses different sources to ground the responses.
• It uses different searches on the web. You can count 5 (the ones with the magnifying glass icons).

Below is what happens under the hood with the first prompt:

The following image is about the second prompt:

Testing Examples Type #2: Forcing Search Tools to Read URLs

Imagine a scenario where you insert a prompt, the search tool returns the list of URLs, and you want it to read them, but the model can not access it. This may happen for several reasons:

• The content is behind a paywall.
• The web search tool was not able to go beyond the anti-bot wall.
• There was an error during the request.

These cases can happen more than you think. For the sake of this tutorial, let’s get a target page beyond a paywall so that the result is surely inaccessible:

Claude

Let’s see what happens with Claude:

As expected, the web search tool can not access content behind a paywall. However, as you can see, it did get a response because it says: ”Based on other coverage of the same story, here’s what it’s about”.

Now the response it gave is not interesting, so I’m not showing it for brevity. What is interesting is trying to understand what it has done under the hood. Basically, it was able to understand from the URL that the article is about IBM and a profit warning. So it autonomously made a web search. Let’s unpack it:

The image shows exactly the kind of non-deterministic approach described previously. As you can see, the system:

• Tried to open the proposed URL, but failed at it.
• Autonomously tried a different approach, searching on the web, based on the metadata extracted from the URL.
• It gave the response based on the content it was able to “read” from the sources it got from the web search.

So here is an interesting thing that happened: the system told you that it was not able to access the specific URL and that it used “other coverage of the same story”, but it did not tell you how. So, how do you think you can get deterministic results?

ChatGPT and Perplexity

What is worth considering is that ChatGPT and Perplexity use similar but different approaches between them, and with respect to Claude. Below is the result for ChatGPT:

As you can see, ChatGPT does not tell you that it cannot access the target URL, unlike Claude. Also, its search mode is different from Claude’s, because it searches (also) for other content from the WSJ that can refer to the target one.

Below is Perplexity’s result:

As you can see, Perplexity produces a result that is similar to ChatGPT’s:

• It doesn’t tell you it can access the target URL.
• It searches for content (also) on other articles on the WSJ.

Overall, this demonstrates that the result you get is non-deterministic and depends on the model you use.

Architectural Guidance: When to Use Each Primitive #

At this point you may be asking: ”So, when should I use web search tools?”. Or:” Should I use web search tools at all?”.

Good questions! The first answer is: “Yes, you should use them”. But let’s subdivide the case for the explanations on “when”.

Using Web Search Tools via Chat

I would argue that the use of web search tools via chat is “the canonical” one. This is basically because when you prompt a model, and it kind of understands that it should use the web search tool, it will. This is included in the “non-deterministic” things already presented.

However, you’ve also seen that there are cases when the web search fails. Regarding the reasons why this happens, a solution to improve web search via chat is using MCPs from scraping providers. Here at The Web Scraping Club, we already made the case with “How to Give Claude Real-Time Web Access With the Decodo MCP”.

Using Web Search Tools via APIs

Considering using web search tools via APIs, the correct engineering posture is to treat web search tools and web scrapers not as alternatives but as complementary primitives operating at different layers of a retrieval architecture:

• Web search tools are appropriate at the discovery layer : When the model needs to identify which URLs are relevant to a user’s query, a search tool is an efficient mechanism for surfacing candidates from the indexed web.
• Web scrapers are appropriate at the retrieval layer : Once a target URL is known, either because a search surfaced it, because it is statically configured, or because a crawl traversal discovered it, a scraper is the correct instrument for extracting full, structured, high-fidelity content from that resource.

Below is a schema that represents this complementarity:

For example, a production-grade retrieval pipeline for a domain-specific RAG system that needs to retrieve fresh and up-to-date data every once in a while will typically combine both:

• A search tool for open-domain candidate discovery.
• A scraper for full-document extraction from the identified URLs, a Markdown conversion step for context-optimal serialization, a chunking and embedding step for vector indexing, and a nearest-neighbor retrieval step for query-time grounding.

In such cases, the web search tool and the web scraper are each necessary components of this pipeline. Neither is sufficient alone, and treating either as a drop-in substitute for the other is a mistake.

As a final consideration, note that it makes sense to use web search tools on the discovery layer because of how they work under the hood. As covered above, the web search tool is basically a trigger for APIs that retrieve content ranked from search engines. This is basically what every major scraping provider has to offer in terms of SERP/search APIs. In other words, if you are already a customer of one of the most common scraping providers, instead of using the web search tool that your favourite LLM has, you can directly use your scraping provider’s SERP/search API.

Conclusion #

In this article, you learned how tools work in LLMs. You’ve also seen the limitations of web search tools and their best use cases, using them via chat or via API.

Overall, for a deterministic and scientific approach, web scraping remains the best data retrieval layer.

So, let us know in the comments: are you using web search tools or web scraping for ingesting LLMs with recent data?

Did you like this article? Share it with someone who might find it useful and get a discount on paid plans.

source & further reading

scraping.club — original article The LAB #115: Manus at Work - Datadome on Leboncoin.fr Crawl of Fame - September 2026 AntsData Review: A Deep Dive into its SERP and Web Search APIs