{"slug": "aie-2-3-tool-use-how-an-llm-reaches-your-database", "title": "aie_2.3: tool use, how an llm reaches your database", "summary": "A developer's AI engineering series explains how an LLM reaches a database through tool use, building on prior lessons about structured outputs and the messages array. The writeup walks through a customer support scenario for the fictional shop Kora Home, showing how a stateless model's context and token prediction mechanics lead into the need for external tool calls to look up order data.", "body_md": "I want to pick up where we left off. In aie_2.2, we met [Kora Home](https://heymeraki.substack.com/p/aie_22-structured-outputs-getting?r=8ei6qe), a small online shop that sells home goods, and a customer named Amara.\n\n**Amara:*** Hi, this is Amara. My order 4821 was due last week and I’m still waiting. Where is it?*\n\nWe said the app has three jobs to do with that message. Turn it into a record for the support log, find out where order 4821 actually is, and show Amara the answer.\n\nWe handled job one, *[Structured outputs](https://heymeraki.substack.com/p/aie_22-structured-outputs-getting?r=8ei6qe)* turned her message into a row in our log.\n\n```\nname     | order_id | request\n---------|----------|----------------\nJonah    | 3310     | refund\nPriya    | 5127     | change address\nAmara    | 4821     | order status\n```\n\nSo the app now knows, in three values, who wrote in and what they want. This lesson is the next piece of that, finding out where order 4821 is.\n\n### A Quick Refresher\n\nIn [aie_1.0](https://heymeraki.substack.com/p/aie_10-introduction-to-ai-engineering?r=8ei6qe), we learned that an LLM is a prediction machine.\n\n*A **Large Language Model (LLM)** is a prediction machine that is trained on a large amount of text and is capable of generating coherent, contextually appropriate language by predicting likely continuations of any input given.*\n\nThe key word is still ***predict***. The model writes its answer one small piece at a time, and at each step it picks from a range of possibilities, ranked by how likely each one is. Each of those small pieces is called a ***token***.\n\n*A **token** is the smallest unit the model works with. Think of it as a chunk of text. “Hello” is one token. “Unbelievable” might be three.*\n\nWe also learned that LLMs have amnesia.\n\n*A system is **stateless** when each request (message) is handled completely on its own, with zero memory of any request that came before it.*\n\nSo to make the model *“remember”*, the app sends the whole conversation back on every call. That conversation is part of the model’s context, everything the model can see at the moment it writes a response. **Context is the focus of this lesson**, you’d want to keep it front of mind.\n\nIn [aie_2.0](https://heymeraki.substack.com/p/aie_20-inside-the-llm-api-call?r=8ei6qe), we pictured all of this as a phone call where the person on the other end has amnesia. Every time you call, they pick up with zero idea who you are. The only way to have a conversation is to read them the full transcript from the beginning every time. That transcript is the ***messages array***.\n\n*The **messages array** is the list of all that you send to the model on a single API call. It is structured as a sequence of messages, each one labelled with who sent it.*\n\nEach label is a role. `user` is whoever is on the other end of your product, like your customer. `assistant` is the model. Alongside the messages array, you send parameters like `model` and `max_tokens` and what comes back is the response object.\n\n- `response.content` is a list of content blocks, where the model’s reply lives.`response.content[0].text` takes the first block and pulls out the words.\n- `response.usage` holds the token counts,`input_tokens` for what you sent and`output_tokens` for what the model wrote back.\n- `response.stop_reason` tells you why the model stopped writing.`end_turn` means it finished its answer.\n\nTwo more from aie_2.2. JSON is a text format for data where each piece of information is next to a label, the label being a **key** and the information being its **value**. And a **schema** is a written description of the shape you expect, listing each field by name, what type of value goes in it, and which fields must be filled in. We used a schema as a form with labelled boxes. It comes back here doing something slightly different.\n\n### The Model’s Knowledge Limit\n\nKora Home’s database has an orders table, it keep track of all customer’s orders. Here’s what it looks like.\n\n```\norder_id | customer | status               | delivery_date\n---------|----------|----------------------|---------------\n3310     | Jonah    | delivered            | 2026-09-14\n4790     | Amara    | returned             | 2026-09-11\n4821     | Amara    | delayed at courier   | 2026-09-24\n5127     | Priya    | preparing            | 2026-09-30\n```\n\nRow three is the answer Amara wants.\n\nThe model runs on Anthropic’s computers. So let’s be precise about what it can actually see. The model works with exactly two things.\n\n- **What it learned in training.** Training is the process where the model read an enormous amount of text. That process ended on a fixed date, so anything after that date is outside it. So is anything private, like a shop’s customer orders.\n- **What’s in its context.** Everything you put in the request, which is the messages array.\n\nAmara’s order appears in neither one. So watch what happens when the app just asks.\n\n```\nresponse = client.messages.create(\n    model=\"claude-haiku-4-5\",\n    max_tokens=1024,\n    messages=[\n        {\"role\": \"user\", \"content\": \"Where is order 4821?\"}\n    ],\n)\n\nprint(response.content[0].text)\n```\n\nThe reply comes back looking perfectly reasonable.\n\n```\nYour order 4821 shipped on September 19 and is scheduled to arrive \nwithin 3 to 5 business days. You should receive it shortly!\n```\n\nEvery part of that is made up. There's no September 19, no 3 to 5 business days. The model did the only thing it can do, which is predict a likely-sounding continuation *(it guessed)*.\n\n*\"Where is order 4821?\"* is followed, in the enormous amount of text it trained on, by exactly this kind of cheerful shipping update and that's what it wrote.\n\nWhat makes it worse than not responding is that the reply has a real date, a real-sounding timeframe and sounds so confident. Nothing in it looks wrong. If the app sends that to Amara, she waits for a parcel that is sitting with a courier, delayed.\n\nThis behaviour is called ***Hallucination***. \n\n**Hallucination** is when a model says something false with the same confidence that it says something true. It is a result of the model predicting likely text with no way to know which parts of that text are correct.\n\n*This is important because as an AI Engineer, the most dangerous output from an LLM is the one that looks right. In traditional engineering, a bad response typically fails loudly and can be caught in testing. A hallucinated reply from an LLM gets to your end user.*\n\nOne way to prevent this from happening is to connect the model to data so its responses are more grounded. The question we’re answering in this lesson is how.\n\nLet’s look at a couple of approaches:\n\n### Approach One: Put Everything in the Message\n\nThe model works with what’s in its context, and the messages array is the context. Your first thought might be to put the orders in the messages array.\n\n```\norders_text = \"\"\"\norder_id | customer | status               | delivery_date\n3310     | Jonah    | delivered            | 2026-09-14\n4790     | Amara    | returned             | 2026-09-11\n4821     | Amara    | delayed at courier   | 2026-09-24\n5127     | Priya    | preparing            | 2026-09-30\n\"\"\"\n\nresponse = client.messages.create(\n    model=\"claude-haiku-4-5\",\n    max_tokens=1024,\n    messages=[\n        {\"role\": \"user\", \"content\": f\"Here are our orders:\\n{orders_text}\\n\\nWhere is order 4821?\"}\n    ],\n)\n```\n\nYou won’t be wrong to think that, it works fine. The order is in the context now, so the model reads row three where we have Amara and answers from it.\n\n```\nOrder 4821 is currently delayed at the courier. The updated delivery\ndate is September 24.\n```\n\nIt is correct and grounded like we want it to be. With a small number of orders, this is a decent solution but Kora Home happens to have about forty thousand orders. Now remember that every order costs tokens, and you pay for tokens on every call, what this means for our app is this:\n\n```\nOrders in the table | Roughly        | Cost per call\n--------------------|----------------|---------------------------\n4                   | 30 tokens      | basically nothing\n40,000              | 300,000 tokens | adds up fast, every message\n```\n\nIt becomes expensive pretty quickly. There is a second issue too, every model has a maximum amount of tokens that a model can take in a call. Forty thousand orders may simply not fit, and if it does fit today, it stops fitting as the shop grows.\n\nAnd a third issue that has nothing to do with size is that customers ask about all kinds of things.\n\n*Do you still have the linen throw in sage?*\n\n*What’s your returns window?*\n\n*Can I change the address on 5127?*\n\nThese questions touch on things outside of orders, stock levels, policies, and addresses. To cover those, you'd have to paste in the stock table, the policy document and the customer table too, before every call, on the chance that someone asks. You're guessing in advance what each message will need, and there is only so much ground you can cover.\n\nWe can see that Approach One, although a good start, puts you in an awkward spot. Send everything and pay for all of it on every message, or guess what’s needed and be wrong sometimes.\n\nWhat we want is for the model to say *“I need order 4821”*, and for the app to go fetch that one row. The next approach shows us how we can achieve that.\n\n### Approach Two: Let the Model Ask\n\nBack to the phone call, the person on the other end has amnesia, and you read them the transcript on every call.\n\nNow imagine that partway through, they say this.\n\n*“Hold on. Can you look up order 4821 in your records and tell me what it says?”*\n\nYou put the phone down, walk over to your filing cabinet, find the order, and read it. Then you call back, and because they forget every call the moment it ends, you read out the whole transcript again, including their request, and then you add what your records say at the end. Now they have everything they need, and they answer.\n\nNotice *who does the checking*. They never touch your filing cabinet, they ask, and you decide whether to go and look. Everything that follows in this section is built on this idea.\n\n#### What the App Can Already Do\n\nLet's look at what the Kora Home app can do without AI engineering involved. Long before any of this, some engineer at Kora Home wrote a piece of code that looks up an order. Here it is:\n\n``` python\ndef get_order_status(order_id):\n    row = database.query(\n        \"SELECT status, delivery_date FROM orders WHERE order_id = ?\",\n        order_id,\n    )\n    return {\n        \"order_id\": order_id,\n        \"status\": row[\"status\"],\n        \"delivery_date\": row[\"delivery_date\"],\n    }\n```\n\nIt takes an order number and gives back that order's status and delivery date. Call it with `4821` and you get row three of the orders table.\n\n```\nget_order_status(\"4821\")\n{\"order_id\": \"4821\", \"status\": \"delayed at courier\", \"delivery_date\": \"2026-09-24\"}\n```\n\nThis `get_order_status` code block is a ***function***.\n\n*A **function** is a named piece of code that takes some inputs and gives back an output.*\n\nThis function is named `get_order_status`. Its input is an order number, and its output is those three values.\n\nSo the app can already fetch the exact row the model needs, but the problem is the two sides have never been introduced and that is what we are going to remedy.\n\n#### Introducing Both Sides\n\nLet’s put both halves side by side:\n\n```\nThe app                          The model\n---------------------------      ---------------------------\ncan reach the database           writes the reply to Amara\nhas get_order_status             knows zero about the order\nknows zero about what             decides what the reply says\n  Amara is asking for\n```\n\nEach half holds something the other needs. And so far, nothing connects them. The app never told the model that `get_order_status` exists, so as far as the model is concerned, there’s no way to look anything up, which is why it invented a delivery date.\n\nNow think about what actually has to change for that connection to happen.\n\n- *The model needs to know the function is there.* The model only works with what’s in its context, so the function has to be***described*** to it, in text, the same way everything else reaches it.\n- *The model needs a way to say it wants that function run.* Writing text is the only thing it does, so this has to come back as part of a reply.\n- *The app needs to run the function and get the answer back to the model.* And since the model forgot everything the moment the first call ended, that means a second call with the whole conversation in it.\n\nThose three things are what function calling is.\n\n**Function calling** (also called **tool use**, the two names mean the same thing) is a way for the model to ask your code to run one of your functions, and then use the result in its answer.\n\nLet’s take the first of those three things, *describing the function to the model*.\n\nThe model reads text, so the description it expects is text. Notice what it has to cover, the model needs to know the function exists, what it does, so it can tell when the function is worth using, and what inputs it takes, so it can fill them in. What we send is a written description of `get_order_status`, alongside Amara’s message.\n\nThat gives us two different things with similar names:\n\n- `get_order_status` is the function. It’s Python, it runs on Kora Home’s computers, it reaches the database, and the model doesn’t see it.\n- The description is text. It goes to the model, holds the function’s name, an explanation of what it does, and a list of its inputs. The model reads it and nothing else.\n\nThe LLM API has a term for a function that is described to the model, ***a tool***.\n\n*A **tool** is a function your app can run, described to the model in text, so the model knows the function exists and can ask for it.*\n\nHere's the whole flow, matching those three things in order. We'll walk through each step:\n\n```\n1. Your app   →  the model     Amara's question, plus the tool list\n2. The model  →  your app      \"run get_order_status with order_id 4821\"\n3. Your app                    runs get_order_status, gets the row\n4. Your app   →  the model     the whole conversation again, plus the result\n5. The model  →  your app      \"Order 4821 is delayed at the courier\"\n```\n\nTwo things are worth noticing here. There are **two API calls** here, steps 1 and 4, and in step 4, the app sends the whole conversation again, because the model is stateless and forgot step 2 the moment it ended. *Amnesia.*\n\n#### Step 1. Write the Description\n\nThe description has three parts, each one covering something the model needs to know.\n\n```\ntools = [\n    {\n        \"name\": \"get_order_status\",\n        \"description\": \"Look up the current status and delivery date of a Kora Home order, given its order number.\",\n        \"input_schema\": {\n            \"type\": \"object\",\n            \"properties\": {\n                \"order_id\": {\"type\": \"string\", \"description\": \"The order number, for example 4821\"}\n            },\n            \"required\": [\"order_id\"],\n        },\n    }\n]\n```\n\nLine by line, and look out for how each part maps onto the function we looked at earlier.\n\n- `name` is`get_order_status` , spelled like the real function. This is what the model uses to refer to it, so when a request comes back naming`get_order_status` , your code knows which function to run.\n- `description` explains, in plain words, what the function does. The model reads this to decide*whether this function is worth using for the message in front of it* , so a vague description could cause it to get skipped when it was needed.\n- `input_schema` lists the inputs the function needs. The original function receives one input,`order_id` , so there’s one box here.\n\nThat last part should look familiar. It’s a schema, the same kind of form from aie_2.2 being used differently. In aie_2.2, the form described the model’s *answer*, here it describes the *inputs* the model has to fill in when it asks for the function.\n\n`tools` is a list, so Kora Home could describe five functions here and the model would read all five before deciding. We only need one for our use case.\n\n#### Step 2. Send the Question, With the Description Attached\n\nThis is an ordinary call with one new parameter, `tools`.\n\n```\nmessages = [\n    {\"role\": \"user\", \"content\": \"Where is my order 4821? It was due last week.\"}\n]\n\nresponse = client.messages.create(\n    model=\"claude-haiku-4-5\",\n    max_tokens=1024,\n    tools=tools,\n    messages=messages,\n)\n```\n\n#### What Comes Back\n\nThe response object looks different this time, in two places.\n\n- ***The stop reason.*** In aie_2.2, a finished answer came back as`end_turn` . This time it comes back as`tool_use` , which means the model paused partway through to ask for a function. It’s the*“hold on, can you check that for me?”* moment from the phone call.\n- ***The content.*** In aie_2.0, I mentioned that`response.content` is a list because the API is built to hold more than one block, even though there’s usually just one. This is where we see the second block.\n\n```\nstop_reason: \"tool_use\"\n\ncontent: [\n  {\n    \"type\": \"text\",\n    \"text\": \"Let me check on that order for you.\"\n  },\n  {\n    \"type\": \"tool_use\",\n    \"id\": \"toolu_01A...\",\n    \"name\": \"get_order_status\",\n    \"input\": {\"order_id\": \"4821\"}\n  }\n]\n```\n\n The first block is text. The second block, the one with `\"type\": \"tool_use\"` , is the request, it has three parts.\n  - `name` says which function the model wants. Here it is,`get_order_status` , the same name from the description in Step 1.\n  - `input` holds the inputs it filled in, following that description’s form. Here,`order_id` is`4821` . The model read`4821` out of Amara’s message and put it in the box.\n  - `id` is a label for this particular request. When your code sends the result back, it attaches the same label, so the model can match each result to the request that asked for it. It matters when the model asks for more than one function at a time.\n\nThe important thing here is what the model did *not* do. It made zero contact with the database. It wrote a request, in text, and gave it back to you.\n\n#### Step 3. Run the Function\n\nIt starts by finding the request block in the reply and reading the order number out of it. Remember that `response.content` is a list with two blocks in it, a text block and a `tool_use` block. The order number is in the second one, so the app has to pick that block out of the list.\n\nThe straightforward way is to walk through the list and stop at the first block whose type is `tool_use`.\n\n```\ntool_request = None\n\nfor block in response.content:\n    if block.type == \"tool_use\":\n        tool_request = block\n        break\n```\n\nThat reads left to right. Go through the blocks one at a time, and when you find one whose `type` is `\"tool_use\"`, keep it and stop looking.\n\nPython has a shorter way to write that same thing, using a built-in called `next`.\n\n`next` *walks through a collection and hands you back the first item, then stops. Give it a condition and it hands you the first item that matches.*\n\nSo these two do the same job.\n\n```\ntool_request = next(block for block in response.content if block.type == \"tool_use\")\n```\n\nRead it in the same order as the loop. `for block in response.content` walks the list, `if block.type == \"tool_use\"` is the condition each block has to match, and `next` takes the first match and stops. Either version is fine to use. The short one is what you'll see in most code, so it's worth being able to read.\n\nEither way, `tool_request` now holds the request block.\n\n```\n{\"type\": \"tool_use\", \"id\": \"toolu_01A...\", \"name\": \"get_order_status\", \"input\": {\"order_id\": \"4821\"}}\n```\n\nFrom there, the app reaches into `input` and pulls out the value the model wrote in the box.\n\n```\norder_id = tool_request.input[\"order_id\"]   # \"4821\"\n```\n\nThen it runs `get_order_status`, exactly the way the support dashboard runs it.\n\n```\nresult = get_order_status(order_id)\n```\n\n`result` now holds the real row.\n\n```\n{\"order_id\": \"4821\", \"status\": \"delayed at courier\", \"delivery_date\": \"2026-09-24\"}\n```\n\n#### Step 4. Call Back With the Result\n\nThis is the callback from the phone picture. The app adds two messages to the end of the messages array, then makes a second call.\n\n```\nmessages.append({\"role\": \"assistant\", \"content\": response.content})\n\nmessages.append({\n    \"role\": \"user\",\n    \"content\": [\n        {\n            \"type\": \"tool_result\",\n            \"tool_use_id\": tool_request.id,\n            \"content\": json.dumps(result),\n        }\n    ],\n})\n\nfinal = client.messages.create(\n    model=\"claude-haiku-4-5\",\n    max_tokens=1024,\n    tools=tools,\n    messages=messages,\n)\n\nprint(final.content[0].text)\n```\n\nThree details are worth slowing down on.\n\n- **The first new message is the model’s own reply from Step 2** , added back exactly as it came, with the role`assistant` . The model is stateless, so this is how it sees, on the second call, that it asked for order 4821. This is the same move from the aie_2.1 build, where every model reply got added to the array so the conversation held together.\n- **The second new message carries the result** , inside a block with the type`tool_result` , labelled with the same`id` from the request. It goes in with the role`user` , which can feel odd, since Amara typed none of it. In the conversation there’s the model’s side and there’s everyone else’s side. Your customer and your app both sit on the side that talks*to* the model, so both use`user` . And`json.dumps` is the reverse of the`json.loads` from aie_2.2. It turns the Python dictionary back into JSON text so it can travel inside a message.\n- `tools` **gets sent again.** Every call starts fresh, so the model needs the full list of descriptions every time, the same way it needs the full messages array every time.\n\nHere’s what the messages array looks like now, going into that second call.\n\n```\n1. user       \"Where is my order 4821? It was due last week.\"\n2. assistant  text block + tool_use block asking for get_order_status\n3. user       tool_result block holding the real row\n```\n\nThe model reads all three from top to bottom. Now the order details are in its context, which is the whole point. The stop reason comes back as `end_turn`, and the reply reads something like this.\n\n```\nYour order 4821 is currently delayed at the courier. The updated\ndelivery date is September 24. Sorry for the wait!\n```\n\nCompare that to the invented answer at the start of this lesson. Same question, same model, same confident tone but this one is true, because the September 24 came out of row three of the orders table, fetched by a function Kora Home had the entire time.\n\n### Where This Comes Apart\n\nFunction calling solves the problem we started with, but the model is the one deciding *whether* to ask for a function, and that decision is a prediction like any other. There are two situations that should be looked at:\n\n#### The Model Skips a Function It Should Have Used\n\nRemember that the model reads the `description` field to decide whether a function fits the message. With a thin description, it’ll just guess. Here’s the same tool we worked with described badly.\n\n```\n{\n    \"name\": \"get_order_status\",\n    \"description\": \"Gets order info.\",\n    \"input_schema\": { ... }\n}\n```\n\nNow Amara writes in with a less direct message.\n\n*Hi, it’s Amara. Any update on my delivery? It was meant to be here last week.*\n\nShe mentions no order numbers and doesn’t say the words “order status”. With a thin description, the model may decide this looks unrelated and answer from nothing, which puts you right back at the invented shipping update.\n\nA description that says what the function is *for*, including the words customers actually use, makes the match far more reliable.\n\n```\n\"description\": \"Look up the current status and delivery date of a Kora Home order, \n    given its order number. Use this for any question about where an order is,\n    when it will arrive, or whether it has shipped.\"\n```\n\n#### The Model Fills the Form With a Made-Up Value\n\nThe schema guarantees the request has the right *shape*, and *the model still writes the* *values*. That’s the same limit we found in aie_2.2.\n\nTake Amara’s vaguer message again, the one with no order number in it. The model may ask for the function and invent something to put in the box.\n\n```\n{\"type\": \"tool_use\", \"name\": \"get_order_status\", \"input\": {\"order_id\": \"0000\"}}\n```\n\nPerfectly shaped, and `0000` came from nowhere. Your function would then query the database for an order that never existed.\n\nThis is why your code acts as the middleman. Before running anything, check the value.\n\n```\norder_id = tool_request.input[\"order_id\"]\n\nif order_id in known_order_ids:\n    result = get_order_status(order_id)\nelse:\n    result = {\"error\": \"No order found with that number.\"}\n```\n\nAnd you’ll see that the `error` goes back to the model as the tool result, in the same `tool_result` block. The model reads it and writes something sensible to Amara, like asking her for the order number.\n\nOne more connection back to aie_2.2. Adding `\"strict\": True` to a description tells the API to use constrained decoding on the inputs, so the request always matches your `input_schema`. That fixes the shape, you should still check the value.\n\n### When One Function Leads to Another\n\nEverything above was one function, once. In larger apps, you’ll likely need more.\n\nSay Amara’s next message is *“fine, can you cancel it then?”*. Now the model checks the order first, sees it’s already with the courier, and then asks for a second function, `start_return`, because a parcel in transit gets returned instead of cancelled.\n\nSo in real apps, steps 2 through 4 repeat. Each time the stop reason comes back as `tool_use`, the app runs the function and calls back with the result. It ends when the stop reason comes back as `end_turn`.\n\n```\nwhile response.stop_reason == \"tool_use\":\n    # run the function the model asked for\n    # append the request and the result to messages\n    # call the API again\n```\n\nThat repeating loop, where the model keeps asking for functions until it has what it needs, is the core of what people call an ***agent***. Agents get their own lesson later in the series so don’t worry about it now.\n\n### Where We Are\n\nAsking the model directly gave us a confident, made-up delivery date. Pasting the whole database into every message gave us the right answer at a cost that grows with the shop, and made us guess what data to paste. Function calling let the model ask for the row it needed, and kept your code in charge of whether that lookup happens.\n\nBack at Kora Home, Amara’s message is a clean row in the support log, and the app now has a true answer about order 4821, pulled from the real orders table. There’s one job left. That answer takes several seconds to write, and Amara is watching an empty chat window the whole time.\n\nIn the next lesson, we cover streaming, which puts the answer on her screen word by word, starting almost immediately.", "url": "https://wpnews.pro/news/aie-2-3-tool-use-how-an-llm-reaches-your-database", "canonical_source": "https://heymeraki.substack.com/p/aie_23-tool-use-how-an-llm-reaches", "published_at": "2026-09-29 15:25:29+00:00", "updated_at": "2026-09-29 16:21:24.270298+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-tools", "natural-language-processing"], "entities": ["Kora Home", "Amara"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/aie-2-3-tool-use-how-an-llm-reaches-your-database", "markdown": "https://wpnews.pro/news/aie-2-3-tool-use-how-an-llm-reaches-your-database.md", "text": "https://wpnews.pro/news/aie-2-3-tool-use-how-an-llm-reaches-your-database.txt", "jsonld": "https://wpnews.pro/news/aie-2-3-tool-use-how-an-llm-reaches-your-database.jsonld"}}