Built-In Tools vs. Custom Tools in LLM Agents

A technical comparison of built-in versus custom tools in LLM agents finds that provider-run tools such as OpenAI's and Anthropic's hosted web search execute the entire tool loop inside a single API request, while custom tools require the developer to run the code on their own infrastructure. OpenAI's function-calling documentation describes a five-step flow for custom tools — send tools, get tool call, execute on the application side, send tool output back, get the final answer — whereas Anthropic's web-search flow lets Claude decide when to search and return results multiple times within one request. The distinction matters because providers including DeepSeek support tool calls but offer no hosted web-search tool, forcing developers to bring and run their own search tool.

My First Week With GPT-6 Astra GPT-6 Astra was my main Codex driver for the last week, and I am back on GPT-5.5. That sounds harsher than my… When building AI agents, a tool can be anything that the model can ask to use such as a search engine, a database lookup, a shell command, etc. The model doesn’t directly operate those tools, but rather decides what tool would be helpful, requests a tool use with parameters, gets the result back, and then continues it's work. A custom tool is a tool that you define and run on your own infrastructure. You provide a schema such as search web or get customer balance to the model, the model requests it, and your application runs the actual code on your infrastructure. A built-in tool is different. The provider runs it in its own runtime. In practice this mostly means hosted web search. OpenAI https://developers.openai.com/api/docs/guides/tools-web-search and Anthropic https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool both offer provider-run web search in their APIs. That means if you use ChatGPT or Claude or Codex CLI or Claude Code , they can just search the web and access websites, which internally means they use their built-in web search tool. Other providers like DeepSeek https://api-docs.deepseek.com/guides/tool calls also support tool calls any llm does so more or less but don’t offer a hosted web-search tool themselves, so you’ll have to bring and run the search tool yourself. So a built-in tool feels more “native” mostly because it happens "magically" under the hood. The question I'm interested in here is, what's the difference between using built-in tools vs your own custom tools that you give the LLM. In other words, what's the difference between using ChatGPT/Codex with its integrated web search vs. using DeepSeek while providing my own custom made web search tool. The architecture for a normal tool you provide is roughly like this: The key boundary here is that the model doesn’t execute your function. It returns a structured request that says, effectively: { "tool": "search web", "arguments": { "query": "latest Nvidia earnings" } } Your program sees that, executes something, and sends the result back. The current function-calling documentation https://developers.openai.com/api/docs/guides/function-calling from OpenAI describes this five-step flow pretty much exactly: send tools, get tool call, execute on your application side, send tool output back, get continuation/final answer. Now move the orchestrator inside the provider. The entire loop can happen in one API request from your end. Anthropic’s current web-search flow https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool works like this: Claude decides when to search, the API searches and returns results to Claude, and this can happen multiple times in the request before Claude returns the final answer. OpenAI also refers to the use of a reasoning model for web search as agentic search , where the model manages the search process, analyzes the results, and can choose to continue searching. The search events are exposed in their Responses API as web search call items https://developers.openai.com/api/docs/guides/tools-web-search . So that distinction you're noticing is real. This is where terminology gets tricky. Conceptually, yes: OpenAI documentation currently describes agentic web search as being able to do searches “as part of its chain of thought.” But do not interpret that as an HTTP request that somehow occurs between transformer layer 63 and transformer layer 64. That’s almost certainly not the right mental model. A better abstraction would be the generated text hitting a tool boundary, where the external runtime does the tool work and the model continues from the returned observation. The generation is effectively interrupted/suspended at a tool boundary, where an external system gets an observation and generation proceeds with that observation available. The exact internal implementation KV cache handling, worker scheduling, separate inference passes, etc. is a provider-private implementation detail. Do not assume that the tool is literally implemented within one transformer forward pass. At the abstract level: LLM → action → environment → observation → LLM So, no new cognitive operation is magically available only with provider tools. This distinction is important. This is where the practical difference becomes significant. There are several advantages a provider can have. This is potentially the biggest difference. Imagine these two tool schemas. Your tool: internet lookup query, search depth, domains, freshness Provider tool: web search ... The provider may have trained the model on millions of trajectories like the following during post-training: question → reason → web search → inspect → reason → web search → inspect → answer with citation They can optimize things like: Should I search? What query should I issue? Should I search again? Which result should I open? Which information is relevant? Should I trust it? When do I have enough evidence? How do I cite it? That is much more than learning JSON syntax. It is tool-use policy learning . For example, Anthropic discusses interleaved thinking https://platform.claude.com/docs/en/build-with-claude/extended-thinking for supported tool-use modes. In this case, the model can reason between tool calls to decide what to do next. So, if you plug in your own unfamiliar search tool, the model might generalize very well, but it might not have exactly the same amount of post-training on your particular interface . This is another important point. Suppose your implementation is: results = bing.search query return results :10 The model receives 10 chunks. A provider's web search might conceptually be closer to: query generation ↓ multiple search backends ↓ ranking ↓ fetch pages ↓ extract readable content ↓ deduplicate ↓ spam/quality filtering ↓ reranking ↓ token-budget optimization ↓ citation metadata ↓ model context And there can be additional loops around it. Anthropic’s current web search is a good concrete example. Newer versions can get Claude to run code to filter search results before they get into the model context , so irrelevant content takes up fewer context tokens. OpenAI also allows controls over search context and returned-token budgets, and its search in reasoning mode can perform search, page opening, and find-in-page tasks. So when comparing: provider web search versus mySearchTool you may actually be comparing two very different retrieval systems. Your loop may involve model inference, an API round trip to your application, your own process, a search API call, another process step, another API round trip, and then model inference again. Perhaps repeated 5 times. A hosted loop may look more like a model worker that calls an internal tool service and then continues on the model worker. The provider has opportunities to optimize for scheduling, networking, result serialization, caching, streaming, etc. That can make multi-step research materially faster. I would not, though, assume things like “they definitely preserve the exact KV cache across searches” unless the provider explicitly documents it. That is implementation-specific. With a client tool, suppose you return this: { "results": { "title": "...", "content": "8,000 tokens..." }, { "title": "...", "content": "10,000 tokens..." }, ... } You've now dumped a mountain of text into the model context. A tool integrated with a provider can tightly control search corpus, filtering, extraction, reranking, selection, compression and model context. And metadata can potentially be kept separately. That often gives you a better: useful-information / context-token ratio. This can have a surprisingly large effect on agent quality. Provider-hosted doesn't inherently mean superior. Imagine you're building a programming agent. Instead of generic web search, you give it: search github code search stackoverflow search npm fetch package docs search internal docs lookup symbol plus carefully optimized schemas and result formatting. That system may dramatically outperform generic web search for your application. Or for a financial agent: get sec filing get realtime price get earnings transcript query bloomberg is probably preferable to blindly searching the internet. So provider tools tend to win on: general-purpose integration zero setup latency citations model/tool co-optimization while your own tools win on: control domain specificity private data deterministic APIs custom ranking observability security boundaries cost control provider independence Modern agent architectures actually have something like four levels. Level 1: A client function. Level 2: A remote tool or MCP server. Level 3: A tool hosted by a provider. Level 4: A full product-level agent environment with shell, filesystem, browser, search, git, task state, etc. At this point you're not really comparing models anymore. You're comparing agent systems . And that's becoming increasingly important. Suppose you have: Model A - excellent reasoning - excellent function calling - no web-search feature and: Model B - excellent reasoning - native web search You can absolutely build this around Model A: while True: response = model messages, tools=tools if response.tool call: result = run tool response.tool call messages += response.tool call, result continue return response.text Architecturally, you've recreated the same agent loop. There's no fundamental reason Model A couldn't do excellent web research. The main variables become: reasoning ability × tool-use training × quality of your search stack × context management × agent-loop design not simply: native web search: yes/no There is a more fundamental distinction here. Consider three models: Model A Excellent reasoning Excellent native/function tool use Model B Excellent reasoning Tool calling supported but mediocre Model C Plain text model, no meaningful tool-use training You can bolt tools onto all three. For C you could say: When you want to search, output: <search query </search and parse it. Technically it works. However, agent quality may be much worse if the model hasn’t learned a robust policy for: when to act which action argument construction result interpretation error recovery multi-step exploration stopping That’s one reason that modern “agentic models” feel qualitatively different from older LLMs, even if they can both emit JSON on paper. I'd model an agentic LLM as approximately implementing a policy: \pi a t \mid s t where its current state is something like: s t = \{ \text{prompt}, \text{conversation}, \text{reasoning state}, \text{previous observations}, \text{available tools} \} and the next action might be: a t \in \{ \text{text}, \text{web search}, \text{shell}, \text{read file}, \text{write file}, ... \} A tool execution changes the environment: o {t+1} = Tool a t and that observation is fed back: s {t+1} = s t + a t + o {t+1} Then the model chooses again. From this perspective, provider search and your custom search are mathematically the same kind of operation . What differs is who implements: Tool a t and who implements the surrounding control loop. This is a particularly important consequence. Suppose the naked model has capability M . You might think: AgentCapability = M But it's more like: AgentCapability = f M, tools, tool\ policy, retrieval, context\ management, orchestration A good search-enabled model may seem much smarter because it can repeatedly turn uncertainty into information: That's qualitatively different from classic one-shot RAG. The model controls retrieval dynamically . That's a huge part of the power. Traditional RAG works by retrieving before inference. Agentic retrieval allows the model to retrieve, see, continue, and then retrieve again. The information-gathering policy itself becomes part of the reasoning process. This architecture is particularly easy and efficient with provider-hosted search, since the provider owns that entire inner loop. But you can implement precisely this pattern yourself. | | Provider web search | Your own web-search tool | |---|---|---| | Execution | Provider infrastructure | Your infrastructure | | Agent loop | Usually provider-side | Usually your orchestrator | | Extra API round trips | Fewer/exposed less | Usually yes | | Model specialization | Often highly optimized | Depends on model | | Retrieval pipeline | Provider-controlled | Fully yours | | Search ranking | Provider-controlled | Fully yours | | Context optimization | Often built in | You implement it | | Citations | Usually first-class | You implement them | | Observability | Limited | Excellent | | Customization | Limited/moderate | Unlimited | | Private sources | Limited to integrations | Excellent | | Vendor lock-in | Higher | Lower | | Domain-specific quality | General purpose | Can be much better | If I were designing agents, I'd think of it like this: MODEL ↓ action ORCHESTRATOR ↓ execute TOOLS ↑ observation The conceptual algorithm is the same . The differences are in integration, post-training, latency, retrieval quality, context management, and operational control. And so I would not choose an LLM provider purely because it has native web search . If a model has good reasoning and good generic tool-use capabilities, you can build a great or for specialized use cases, better search agent on top of it. The native tool is much more important if you want strong general-purpose research with minimal engineering. The most interesting next layer here is how reasoning models are actually trained to decide when to call tools -- i.e. how tool use appears in SFT/RL trajectories and why that produces better agents than merely teaching a model a JSON function-calling grammar. That's where the architecture starts connecting directly to the model training itself. Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request. Take a look at vroni.com