# Build a production-grade RAG Agent: Part 2

> Source: <https://dev.to/yuvraj2112/build-a-production-grade-rag-agent-part-2-2hbe>
> Published: 2026-11-16 01:45:30+00:00

Let's build the agent and its tools, in TypeScript with the Vercel AI SDK. The ideas carry over to any language or framework, such as Python with Pydantic AI.

The [repo](https://github.com/yuvraj2112/rag-agent) has two folders that run on their own and meet only over HTTP: `harness/`, the measure from Part 1, and `agent/`, which you build here. Its README covers the rest.

This is Part 2 of a two-part series:

**TL;DR**

In Part 2, we build the agent: one model, three tools over a library of tax Acts, and a loop. By the end, we have an agent that beats the first iteration (0.582 → 0.729, with confidently wrong answers down from 5 to 0), and a way to spot a real improvement.

In my main build, the bigger project this tutorial comes from, I first tested which part of the job matters more: finding the law, or reasoning over it. Each version is compared with the floor, one model call with no tools. The repo doesn't include the first two.

| Version | Against the floor | Questions won / lost / tied | 
|---|---|---|
| Perfect passages: the gold's own sources handed to the model | +0.006: a tie | 27 / 28 / 3 | 
| One fixed search: the question searched once, the top passages handed to the model | −0.080 | 17 / 34 / 7 | 
| The agent, searching and reading for itself | +0.082 | 34 / 16 / 8 | 

Handing over the right sources didn't beat the model's own memory, and one search made it worse. Only the agent, searching and reading for itself, beat the floor.

```
cd rag-agent/agent          # the repo you cloned in Part 1
npm i
cp .env.example .env        # add your OPENAI_API_KEY
```

Every setting lives in `agent/.env`, and `.env.example` lists each one with its default. The ones you'll touch:

```
OPENAI_API_KEY=        # the only one you must fill in
MODEL=gpt-6.1-sol      # one model for everything
MODEL_EFFORT=medium
TOOLS=library          # none = the floor; library,web adds the web tools
RUN_ROUNDS=40          # when a run must stop: rounds of tool calls,
RUN_SECONDS=480        # seconds,
RUN_DOLLARS=3          # or dollars
EMBEDDINGS=off         # auto: search by meaning with the library's vectors
```

The model's name is a setting. To change provider, install its package (`npm i @ai-sdk/anthropic`), change `chatModel` in `src/model.ts`, and set the new model's prices in `.env`:

``` js
import { anthropic } from "@ai-sdk/anthropic";
export const chatModel = () => anthropic("claude-sonnet-5-5");
```

The repo ships the three Acts as files in `agent/data/law/`, and a ready-made SQLite library built from them. Copy it in before you start the agent:

```
npm run seed                 # copies the ready-made library: 5,694 passages and their vectors
npm run seed -- --fresh      # downloads the Acts again and embeds every passage: about a minute and $0.20
npm run fetch-law -- <year>/<number>   # fetches another Act from legislation.govt.nz; then npm run seed -- --fresh
```

SQLite's full-text index is searched by default. Set `EMBEDDINGS=auto` to search by meaning instead: each search then embeds its query first.

First, we make one model call and no tools for every answer.

This shows what the model knows on its own. It becomes the baseline: every change after it is measured against this number.

```
# Terminal 1: the floor
cd rag-agent/agent
TOOLS=none PORT=8788 npm start # TOOLS=none turns the agent into the floor

# Terminal 2: ask all 18 questions 3 times, then check and judge every answer
cd rag-agent/harness
npm run eval -- --agent http://localhost:8788 --name floor --runs 3
# eval first says what it will cost and stops; add --yes to go ahead.
```

In code, this could look like:

``` js
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

const { text } = await generateText({
  model: openai("gpt-6.1-sol"),
  instructions: "You are an expert tax adviser… You have no tools. Answer from what you know.",
  messages: [{ role: "user", content: question }],
});
```

Mine, on 14 Nov. The numbers every change had to beat:

| Run | Score | Strict passes | Confidently wrong | Median cost and time an answer | 
|---|---|---|---|---|
| The floor: one call, no tools | 0.582 | 25 of 54 | 5 | $0.023, 59 s | 

Next, tools. They let the agent search and read the law, the "retrieval" in RAG, so it answers from sources instead of memory.

Design tools like APIs: one job each, with a clear set of arguments. Keep the number of tools low and each one simple.

Tool result = data for model; Tool failure = response for the model to act on

A tool essentially looks like this:

```
// agent/src/tools/index.ts, trimmed
passage_search: tool({
  description: "What the tool does, when to use it, and what it returns",
  inputSchema: "The arguments the model can call it with",
  execute: (input) => call(() => searchPassages(library, book, input)),
}),
```

The description is the tool's manual: the model reads it every time.

The repo has these tools:

`library_search`: finds an Act by its title or by what it covers. Nothing it returns can be cited.`passage_search`: searches inside the Acts by a query, exact words or a section code, and returns the first 300 characters of each hit. Nothing here can be cited either.`library_read`: reads a whole section, or an Act's outline. Only what it returns can be cited, as `[extract:L1]`.` TOOLS=library,web`: `web_search` and `web_read`, for law and official guidance the library lacks. Web search needs a Firecrawl key.
The first three are the default, and all the agent needs.

Now the agent itself. It reasons about the question, calls tools, reads what they return, and answers once it has enough. This is a ReAct agent, named after the 2022 paper [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629).

We build this by using the SDK's `generateText`:

```
// agent/src/agent.ts, trimmed: the floor's call, with tools and a way to stop
const result = await generateText({
  model,
  instructions: instructionsFor(names, today),
  messages: [{ role: "user", content: questionMessage(question, country) }],
  tools,                                                   // library_search, passage_search, library_read
  stopWhen: [isStepCount(limits.rounds + 1), overBudget],  // 40 rounds of tools, or $3
  prepareStep,                                             // after the last round: tools off, so it must answer
});
```

That's the whole loop. The SDK calls the model, runs the tools it asks for, sends back the results, and repeats until the model answers without asking for a tool, or a limit stops it.

Run the agent the same way you ran the floor, then compare the two:

```
# Terminal 1: the agent, with its three library tools (the default)
cd rag-agent/agent
npm start

# Terminal 2
cd rag-agent/harness
npm run eval -- --name agent --runs 3 --yes
open results/report.html
```

In the report, pick `agent`, then pick `floor` to compare. It shows the difference, the questions won, lost and tied, and which stamps changed. Ours, on 14 Nov:

|  | The floor | The agent | 
|---|---|---|
| Score | 0.582 | 0.729 | 
| Strict passes | 25 of 54 | 35 of 54 | 
| Confidently wrong | 5 | 0 | 
| Median cost and time an answer | $0.023, 59 s | $0.047, 36 s | 

**What's noise and how to deal with it?**

Run the same agent several times with nothing changed, and see how far the score moves. Three runs of the first version of my main build (58 questions, 3 answers each):

| Run | Score | 
|---|---|
| 1 | 0.742 | 
| 2 | 0.729 | 
| 3 | 0.731 | 

Nothing changed, yet the score moved by 0.013. Compared question by question, the differences between runs stretched from −0.039 to +0.027. So a change had to win by about 0.03 before I believed it.

To try a change of your own, change one thing, restart the agent, and run it under a new name:

```
# for example: edit agent/src/prompt.ts, then restart the agent (Ctrl+C, npm start)
cd rag-agent/harness
npm run eval -- --name agent-my-change --runs 3 --yes
```

In my project where this tutorial is derived from, I tested the following, each time evaluating the results and shedding anything that didn't improve the scores. I suggest you try the same. Do not settle on a methodology just because the industry says it. Test it and verify it for your specific use case.

| What I tried | Against the version before | Kept? | 
|---|---|---|
| The prompt without its expert persona, its "first understand the question" paragraph, or both | +0.002, 0.000 and −0.018: all ties | No change | 
| A calculator tool | −0.008 | Dropped | 
| Web search and reading, official sites only | +0.061: at 3× the cost | Off by default | 
| A cheaper model, about 1/20 the price | −0.091 at medium effort, −0.082 at high | No | 
| One prompt line: search each Act by the issue, not only the sections you know | +0.008 on both question sets; 20–33% fewer key sources missed; 15% more cost | Yes | 
| A separate writer after research, four ways, one a reviser that checks the draft against its sources | −0.019 to +0.008; the best cost 59% more | No | 
| A cheaper research model with the strong writer | −0.006, but only 12% cheaper | No | 

You now have an agent that answers NZ tax questions, and the numbers to show it beats the one you started with.

This simple agentic loop is at the heart of most AI agents. Anthropic puts it plainly: agents "are typically just LLMs using tools based on environmental feedback in a loop" ([Building effective agents](https://www.anthropic.com/engineering/building-effective-agents)). Sub-agents, multi-agent systems, and supervisor or router patterns are built from the same piece: loops that call other loops, or code that picks which loop runs. Not everything is an agent, though: many AI apps are fixed workflows that never loop. Keep building and tweaking: the next improvement might be in the golds, the data or the agent itself.

`library.ts` for it.
Record everything: every answer, step and verdict. You'll use them to see why an answer went wrong, to score again without paying again, and to test one part on its own, like a new writer on research you already saved.

`agent/runs/`. In production, send the same steps to a tracing tool too. The AI SDK speaks OpenTelemetry: register it once at startup, send the traces to Datadog or similar, and alert on cost per answer, limits hit and tool errors (

``` js
// npm i @ai-sdk/otel, plus an OpenTelemetry exporter for your tool
import { registerTelemetry } from "ai";
import { OpenTelemetry } from "@ai-sdk/otel";

registerTelemetry(new OpenTelemetry());
```

Limits are critical: without them, a stuck agent keeps spending. The repo caps each answer at 40 rounds, 8 minutes and $3 ([`stopWhen`](https://github.com/yuvraj2112/rag-agent/blob/main/agent/src/agent.ts#L122) and [a timer](https://github.com/yuvraj2112/rag-agent/blob/main/agent/src/run.ts#L139)). On the last round its tools switch off, so it still answers from what it has read, and after three steps that find nothing new it's nudged to answer or change approach.

Put cost and time next to the score. Web search gave no clear win at 3× the cost, and a separate writer cost 59% more for at most +0.008: neither was worth it.

Treat tool results as data, never instructions. In the repo every result starts with a line saying so, and the prompt repeats it. Set `PLANTS=on` to plant an attack page and check the agent doesn't obey it.

Don't get locked into one provider. Models change price, get retired and go down; a swap is one line in `model.ts`.

Make long runs survive a crash. Durable workflow tools save each step and carry on after a pod restart: try [Temporal](https://temporal.io/), or the AI SDK's [`WorkflowAgent`](https://ai-sdk.dev/docs/agents/workflow-agent), which runs on Vercel's Workflow DevKit.

Put the harness in your CI/CD. `npm test` needs no keys, so run it on every pull request; run the full eval before each release, and stop any release that drops the score by more than the noise. Feature flags let you try a change on a few users first.

Keep checking after release. The code checks need no gold, so run them on live answers. For the judge, write golds for a few real questions each week and add them to your set.

The agent and the harness, with every file this part uses: [github.com/yuvraj2112/rag-agent](https://github.com/yuvraj2112/rag-agent)

Part 1, the evals: [Build a production-grade RAG Agent: Part 1](https://dev.to/yuvraj2112/building-a-production-grade-rag-agent-2eee)
