How AI Agents Choose the Right Tool

A practitioner guide lays out how AI agents should choose among tools by scale: standard model-driven selection for 1–20 tools, semantic or rule-based selection for 20–100 tools, and hierarchical, embedding-based or hybrid approaches for 100+ tools. The guide notes that standard selection requires no additional training, embeddings or toolset hierarchy but can add a couple of seconds of latency per model call, and recommends concise tool names such as get_city_weather_forecast plus short purpose descriptions to reduce ambiguity. It concludes that modern enterprise AI systems rarely rely on a single strategy and combine multiple approaches as tool counts grow.

“What problem is the agent solving for the end user?” The answer to this question drives the design of the tools the agent will use to accomplish its tasks. For example, suppose we are building an AI agent for financial analysis. Users may ask the agent to: · Provide an overview of the Indian stock exchange or Asian markets. · Retrieve historical stock prices for specific securities. · Analyze trends in financial instruments. To keep the system modular and maintainable, we might create separate tools: 1. Market Analysis Tool — Provides overall market insights, trends, and summaries. 2. Price History Tool — Retrieves historical pricing data for stocks and other securities. As the solution grows, users may also request information on: · Bonds · Mutual Funds · ETFs · Commodities · Derivatives · Cryptocurrencies This often leads to the creation of additional specialized tools. Over time, the number of tools can increase significantly, making it more challenging for the agent to select the most appropriate tool for a given request. To address this challenge, various tool selection strategies are used in agent architectures. The appropriate tool selection strategy depends largely on the scale of the agent ecosystem: · Small tool set 1–20 tools : Standard Tool Selection is often sufficient. · Medium tool set 20–100 tools : Semantic or Rule-Based Selection can improve efficiency. · Large tool set 100+ tools : Hierarchical, Embedding-Based, or Hybrid approaches are generally preferred. In practice, modern enterprise AI systems rarely rely on a single strategy. Instead, they combine multiple approaches to ensure that the right tool is selected efficiently and accurately, even as the number of available tools continues to grow. A well-designed tool selection strategy is often the difference between an agent that works in a demo and one that scales successfully in production. Lets look at them in detail. One of the simplest way to select a tool. This is implemented by providing the tool’s information, its description, its definition to the foundational and the model is asked to select appropriate tool from the given context. The output from the foundation model is then compared with the toolset, and the closest one is chosen. This approach is easy to implement, and requires no additional training, embedding, or a toolset hierarchy to use. Major drawback to using this approach is latency: Since tool selection often requires a model call, this can add a couple of additional seconds to the overall response time. Inorder for the model to select the tool effectively, it is important you describe the tool’s capability clearly and unambiguously. Each tool being registered must have a concise descriptive name e.g. get city weather forecase, instead of get wether forcast and few words to describe its unique purpose “Returns 7 day weather forecast for a given city” . By providing clear name and description and its purpose, it clears the ambiguity and helps foundational model to select the most appropriate tool. Code Example Tool Defination """LangChain tool definitions.Two tools are exposed to the LLM: 1. get market data — returns market data for a given country/region 2. get stock price — returns a quote + fundamentals for a given company tickerThe LLM ChatOllama sees these tool schemas and decides which to callbased on the user's question. LangGraph's prebuilt ReAct agent handlesthe tool-call → execute → respond loop automatically."""from langchain core.tools import toolfrom .market data import fetch market data, format market data as text, fetch stock data, format stock data as text, MarketDataError, Country/region → section key that appears in the SerpAPI indexes response Used so the LLM can say "India" and we know which part of the response to highlightCOUNTRY SECTION MAP: dict str, str = { Americas "us": "us", "usa": "us", "united states": "us", "america": "us", Europe "europe": "europe", "european": "europe", "eu": "europe", "uk": "europe", "germany": "europe", "france": "europe", Asia "asia": "asia", "india": "asia", "japan": "asia", "china": "asia", "hong kong": "asia", Other sections always in response "crypto": "crypto", "cryptocurrency": "crypto", "bitcoin": "crypto", "currencies": "currencies", "forex": "currencies", "fx": "currencies", "futures": "futures", "commodities": "futures",}@tooldef get market data country or region: str - str: """ Fetch live stock market data for a given country or region. Use this tool when the user asks about market performance for a specific country, region, or asset class such as: - A country: "India", "US", "Japan", "Germany" - A region: "Europe", "Asia" - An asset class: "crypto", "currencies", "futures" Parameters ---------- country or region : str The country, region, or asset class name, e.g. "India", "US", "crypto". Returns ------- str Formatted market snapshot text with prices and movements. """ try: raw = fetch market data trend="indexes" except MarketDataError as exc: return f"Error fetching market data: {exc}" full text = format market data as text raw Try to filter to the relevant section if a specific country was asked for key = COUNTRY SECTION MAP.get country or region.lower .strip if key: markets = raw.get "markets", {} section data = markets.get key if section data and isinstance section data, list : from .config import MARKET SECTIONS label = MARKET SECTIONS.get key, key.upper lines = f"=== {label} ===" for item in section data: name = item.get "name", "N/A" price = item.get "price", 0 or 0 mv = item.get "price movement", {} or {} direction = mv.get "movement", "" pct = mv.get "percentage", 0.0 or 0.0 val = mv.get "value", 0.0 or 0.0 arrow = "▲" if direction == "Up" else "▼" if direction == "Down" else "─" sign = "+" if direction == "Up" else "-" try: lines.append f" {arrow} {name:<22} {float price : 12,.2f} " f" {sign}{abs pct :.2f}% {sign}{abs val :.2f} " except TypeError, ValueError : lines.append f" {arrow} {name}" return "\n".join lines No specific region matched — return everything return full text@tooldef get stock price ticker: str - str: """ Fetch the current stock price, key statistics, financials, and recent news for a specific company. Use this tool when the user asks about: - A specific stock or company, e.g. "Apple", "Reliance", "Tesla", "TCS" - A ticker symbol, e.g. "AAPL", "RELIANCE:NSE", "TSLA:NASDAQ" - Price, valuation, fundamentals, or news for a single company The ticker should be in SYMBOL:EXCHANGE format, e.g.: AAPL:NASDAQ, GOOGL:NASDAQ, TSLA:NASDAQ, RELIANCE:NSE, TCS:NSE, INFY:NSE, BTC-USD, ETH-USD Parameters ---------- ticker : str The stock ticker in SYMBOL:EXCHANGE format. Returns ------- str Formatted snapshot with price, movement, key stats, financials, and news. """ ticker = ticker.strip if not ticker: return "Error: no ticker provided." try: raw = fetch stock data ticker except MarketDataError as exc: return f"Error fetching stock data for '{ticker}': {exc}" return format stock data as text raw The list passed to llm.bind tools and the ReAct agentALL TOOLS = get market data, get stock price Now the tool is defined we bind them together. def run native tool agent user question: str, llm: Any, - Iterator str : """ReAct loop using llm.bind tools — works only on tool-capable models.""" llm with tools = llm.bind tools ALL TOOLS messages: list = SystemMessage content=SYSTEM PROMPT , HumanMessage content=user question , When number of tools grow and standard tool selection process start to show performance degradation, semantic selection comes to the rescue. As shown in the figure above, the semantic tool selection, first applies embeddings to the tool name, descriopn using embedding modles like Titan, and indexed in the lightweight vector database. Once agent receives the query from the user, it applies embddings to the user query using the same semnatic model it used for tools and sercches in the vector database. Based on top K candiate it selects the most appropriate tool for the task. Tool Registry php @tool@ safedef get stock quote ticker: str - str: """Get the current/latest stock price and today's price change for a single company. Use for questions like "what is Apple trading at" or "how is TSLA doing today". ticker: Google Finance symbol, ideally TICKER:EXCHANGE e.g. AAPL:NASDAQ, JPM:NYSE, TCS:NSE . """ return dump quote summary serp.resolve ticker ticker @tool@ safedef get stock price history ticker: str, window: Window = "1M" - str: """Get historical price chart data and performance over a time window for a stock. Use for trends, performance over time, highs/lows, or "how has X moved this year". window: one of 1D, 5D, 1M, 6M, YTD, 1Y, 5Y, MAX. """ data = serp.resolve ticker ticker, window=window graph = data.get "graph", prices = p "price" for p in graph if isinstance p.get "price" , int, float if not prices: return dump {"summary": quote summary data , "note": "No chart data returned."} step = max 1, len graph // 30 keep the payload small for the LLM return dump { "ticker": data "summary" .get "stock", ticker , "window": window, "start price": prices 0 , "end price": prices -1 , "high": max prices , "low": min prices , "change pct": round prices -1 - prices 0 / prices 0 100, 2 , "points": {"date": p.get "date" , "price": p.get "price" , "volume": p.get "volume" } for p in graph ::step , } @tool@ safedef get company key stats ticker: str - str: """Get company fundamentals and key statistics: market cap, P/E ratio, dividend yield, 52-week range, previous close, employees, CEO, headquarters, and a company description. Use for valuation questions or "tell me about company X". """ data = serp.resolve ticker ticker kg = data.get "knowledge graph", {} stats = {s.get "label" : s.get "value" for s in kg.get "key stats", {} .get "stats", } about = for item in kg.get "about", : entry = {"title": item.get "title" } if isinstance item.get "description" , dict : entry "description" = item "description" .get "snippet" if item.get "info" : entry "info" = {i.get "label" : i.get "value" for i in item "info" } about.append entry return dump {"quote": quote summary data , "key stats": stats, "about": about} @tool@ safedef get company financials ticker: str, statement: Literal "income statement", "balance sheet", "cash flow", "all" = "all", - str: """Get a company's latest financial statements: revenue, net income, EPS, EBITDA, profit margin, total assets/liabilities, cash flow, from quarterly/annual reports. Use for earnings, profitability, balance sheet, or cash flow questions. """ data = serp.resolve ticker ticker wanted = statement.replace " ", " " out = for section in data.get "financials", : title = section.get "title", "" if statement = "all" and wanted not in title.lower : continue results = section.get "results", if not results: continue latest = results 0 out.append { "statement": title, "period": latest.get "date" , "rows": {k: r.get k for k in "title", "value", "change" } for r in latest.get "table", , } return dump out or {"note": f"No financial statements returned for {ticker}."} @tool@ safedef get stock news ticker: str, limit: int = 5 - str: """Get the latest news headlines and articles about a specific company or stock. Use for "why did X drop/rise", recent events, announcements, or sentiment. """ data = serp.resolve ticker ticker articles = for item in data.get "news results", : articles.extend item.get "items", item keys = "title", "snippet", "source", "date", "link" return dump {k: a k for k in keys if k in a} for a in articles :limit @tool@ safedef compare stocks tickers: list str - str: """Compare current prices and daily price changes of several stocks side by side. Use when the user names two or more companies, e.g. "compare Microsoft vs Google". """ results = {} for t in tickers :6 : try: results t = quote summary serp.resolve ticker t except SerpApiError as exc: results t = {"error": str exc } return dump results @tool@ safedef get market indexes region: Literal "americas", "europe-middle-east-africa", "asia-pacific" = "americas", - str: """Get major stock market indexes S&P 500, Dow Jones, Nasdaq, FTSE, DAX, Nikkei, Sensex... for an overall view of how the market is doing today. """ data = serp.google finance markets "indexes", index market=region trends = {"title": t.get "title" , "results": simplify entries t.get "results", , 15 } for t in data.get "market trends", return dump trends @tool@ safedef get market movers category: Literal "gainers", "losers", "most-active", "climate-leaders" = "gainers", - str: """Get today's top market movers: biggest stock gainers, biggest losers, or most actively traded stocks. Use for "what stocks are up the most today" style questions. """ data = serp.google finance markets category trends = {"title": t.get "title" , "results": simplify entries t.get "results", , 10 } for t in data.get "market trends", return dump trends @tool@ safedef get crypto price symbol: str, currency: str = "USD" - str: """Get the current price of a cryptocurrency such as Bitcoin BTC , Ethereum ETH or Solana SOL . symbol: crypto ticker like BTC, ETH, SOL. currency: quote currency, default USD. """ q = f"{symbol.strip .upper }-{currency.strip .upper }" return dump quote summary serp.google finance q @tool@ safedef get currency exchange rate from currency: str, to currency: str - str: """Get the live foreign exchange forex rate between two fiat currencies, e.g. USD to EUR, GBP to INR. Use for currency conversion questions. """ q = f"{from currency.strip .upper }-{to currency.strip .upper }" return dump quote summary serp.google finance q --------------------------------------------------------------------------- Local calculators no API calls --------------------------------------------------------------------------- @tooldef calculate investment return amount invested: float, buy price: float, current price: float, years held: float | None = None, dividends per share: float = 0.0, - str: """Calculate profit/loss, total return % and annualized return CAGR of a past investment given the amount invested, buy price and current price. Does not fetch prices itself. """ if buy price <= 0 or amount invested <= 0: return "Error: amount invested and buy price must be positive." shares = amount invested / buy price value = shares current price + dividends per share total return = value / amount invested - 1 result = { "shares": round shares, 4 , "current value": round value, 2 , "profit loss": round value - amount invested, 2 , "total return pct": round total return 100, 2 , } if years held and years held 0 and value 0: result "annualized return pct" = round value / amount invested 1 / years held - 1 100, 2 return dump result @tooldef calculate compound growth principal: float, annual rate pct: float, years: int, monthly contribution: float = 0.0, - str: """Project the future value of savings or an investment with compound interest and optional monthly contributions SIP / retirement planning . Rate is the expected annual return in percent. """ r = annual rate pct / 100 / 12 n = years 12 if r == 0: fv = principal + monthly contribution n else: fv = principal 1 + r n + monthly contribution 1 + r n - 1 / r contributed = principal + monthly contribution n return dump { "future value": round fv, 2 , "total contributed": round contributed, 2 , "total growth": round fv - contributed, 2 , } ALL TOOLS = get stock quote, get stock price history, get company key stats, get company financials, get stock news, compare stocks, get market indexes, get market movers, get crypto price, get currency exchange rate, calculate investment return, calculate compound growth, Extra example queries per tool. These are embedded alongside each tool's description so the selector matches on how users actually phrase questions, not just on docstrings.TOOL EXAMPLES: dict str, list str = { "get stock quote": "What is the current price of Apple stock?", "How much is NVDA trading at right now?", "Is Tesla up or down today?", , "get stock price history": "How has Amazon stock performed over the last year?", "Show me Microsoft's price trend for the past 6 months", "What was the 5 day high and low for Meta?", , "get company key stats": "What is Google's market cap and P/E ratio?", "Does Coca-Cola pay a dividend? What's the yield?", "Who is the CEO of Nvidia and where is it headquartered?", , "get company financials": "What was Apple's revenue and net income last quarter?", "Show me Tesla's balance sheet", "How profitable is Microsoft? What's its operating cash flow?", , "get stock news": "Why did Netflix stock drop today?", "Any recent news about Boeing?", "What are the latest headlines on AMD?", , "compare stocks": "Compare Apple, Microsoft and Google stock prices", "Which is doing better today, Coke or Pepsi?", "AAPL vs MSFT", , "get market indexes": "How is the stock market doing today?", "What's the S&P 500 and Dow at?", "How are Asian markets performing? Nikkei, Sensex", , "get market movers": "What are the top gaining stocks today?", "Which stocks fell the most today?", "Most actively traded stocks right now", , "get crypto price": "What is the price of Bitcoin?", "How much is one Ethereum worth in USD?", "Current Solana price", , "get currency exchange rate": "Convert 100 US dollars to euros", "What is the USD to INR exchange rate?", "How many yen per British pound?", , "calculate investment return": "If I invested $1000 in Apple at $150 and it's now $220, how much did I make?", "What's my profit if I bought at 50 and the price is 75?", "Calculate my annualized return over 3 years", , "calculate compound growth": "If I invest $500 a month at 8% for 20 years, how much will I have?", "Project retirement savings with compound interest", "What will $10,000 grow to in 10 years at 7%?", ,} Semantic Tool Selector python class SemanticToolSelector: def init self, tools: Sequence BaseTool , embeddings: Embeddings, examples: dict str, list str | None = None, top k: int = 4, min score: float = 0.0, always include: Iterable str = , : self.tools = {t.name: t for t in tools} self.top k = top k self.min score = min score self.always include = n for n in always include if n in self.tools docs = for t in tools: docs.append Document page content=f"{t.name}: {t.description}", metadata={"tool": t.name} for example in examples or {} .get t.name, : docs.append Document page content=example, metadata={"tool": t.name} self. num docs = len docs self.store = InMemoryVectorStore embeddings self.store.add documents docs def rank self, query: str - list ToolMatch : """Score every tool against the query, best first.""" hits = self.store.similarity search with score query, k=self. num docs best: dict str, float = {} for doc, score in hits: name = doc.metadata "tool" best name = max score, best.get name, float "-inf" ranked = sorted best.items , key=lambda kv: kv 1 , reverse=True return ToolMatch self.tools name , score for name, score in ranked def select self, query: str - list ToolMatch : """Return the top-k relevant tools always at least one plus any always-included tools.""" ranked = self.rank query selected = m for m in ranked : self.top k if m.score = self.min score or ranked :1 chosen = {m.tool.name for m in selected} for name in self.always include: if name not in chosen: selected.append next m for m in ranked if m.tool.name == name return selected if you have a large number of tools and they are semantically similar, and if you are looking to improve tool selection accuracy, hierarchical tool selection technique can be used to achieve the goal at the price of higher latency and complexity. As shown in the example above, a group of tool with same sementic meaning is stored in the lightweight vector database. Upon user query, the query is transformed using embedding model and a group search is performed and from the group top K tool is selected and executed. How AI Agents Choose the Right Tool https://pub.towardsai.net/how-ai-agents-choose-the-right-tool-5c07344a1a9c was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.