{"slug": "build-a-production-grade-rag-agent-part-2", "title": "Build a production-grade RAG Agent: Part 2", "summary": "A developer built a production-grade retrieval-augmented generation (RAG) agent in TypeScript using the Vercel AI SDK, combining a single model with three tools over a library of tax Acts and an agentic search-and-read loop. In evaluation, the agent scored 0.729 versus 0.582 for the first iteration, cutting confidently wrong answers from 5 to 0, while a version given perfect gold passages only tied the no-tools baseline (+0.006) and a single fixed search underperformed it (−0.080).", "body_md": "Let's build the agent and its tools, in TypeScript with the Vercel AI SDK. The ideas carry over to any language or framework, such as Python with Pydantic AI.\n\nThe [repo](https://github.com/yuvraj2112/rag-agent) has two folders that run on their own and meet only over HTTP: `harness/`, the measure from Part 1, and `agent/`, which you build here. Its README covers the rest.\n\nThis is Part 2 of a two-part series:\n\n**TL;DR**\n\nIn Part 2, we build the agent: one model, three tools over a library of tax Acts, and a loop. By the end, we have an agent that beats the first iteration (0.582 → 0.729, with confidently wrong answers down from 5 to 0), and a way to spot a real improvement.\n\nIn my main build, the bigger project this tutorial comes from, I first tested which part of the job matters more: finding the law, or reasoning over it. Each version is compared with the floor, one model call with no tools. The repo doesn't include the first two.\n\n| Version | Against the floor | Questions won / lost / tied | \n|---|---|---|\n| Perfect passages: the gold's own sources handed to the model | +0.006: a tie | 27 / 28 / 3 | \n| One fixed search: the question searched once, the top passages handed to the model | −0.080 | 17 / 34 / 7 | \n| The agent, searching and reading for itself | +0.082 | 34 / 16 / 8 | \n\nHanding over the right sources didn't beat the model's own memory, and one search made it worse. Only the agent, searching and reading for itself, beat the floor.\n\n```\ncd rag-agent/agent          # the repo you cloned in Part 1\nnpm i\ncp .env.example .env        # add your OPENAI_API_KEY\n```\n\nEvery setting lives in `agent/.env`, and `.env.example` lists each one with its default. The ones you'll touch:\n\n```\nOPENAI_API_KEY=        # the only one you must fill in\nMODEL=gpt-6.1-sol      # one model for everything\nMODEL_EFFORT=medium\nTOOLS=library          # none = the floor; library,web adds the web tools\nRUN_ROUNDS=40          # when a run must stop: rounds of tool calls,\nRUN_SECONDS=480        # seconds,\nRUN_DOLLARS=3          # or dollars\nEMBEDDINGS=off         # auto: search by meaning with the library's vectors\n```\n\nThe model's name is a setting. To change provider, install its package (`npm i @ai-sdk/anthropic`), change `chatModel` in `src/model.ts`, and set the new model's prices in `.env`:\n\n``` js\nimport { anthropic } from \"@ai-sdk/anthropic\";\nexport const chatModel = () => anthropic(\"claude-sonnet-5-5\");\n```\n\nThe repo ships the three Acts as files in `agent/data/law/`, and a ready-made SQLite library built from them. Copy it in before you start the agent:\n\n```\nnpm run seed                 # copies the ready-made library: 5,694 passages and their vectors\nnpm run seed -- --fresh      # downloads the Acts again and embeds every passage: about a minute and $0.20\nnpm run fetch-law -- <year>/<number>   # fetches another Act from legislation.govt.nz; then npm run seed -- --fresh\n```\n\nSQLite's full-text index is searched by default. Set `EMBEDDINGS=auto` to search by meaning instead: each search then embeds its query first.\n\nFirst, we make one model call and no tools for every answer.\n\nThis shows what the model knows on its own. It becomes the baseline: every change after it is measured against this number.\n\n```\n# Terminal 1: the floor\ncd rag-agent/agent\nTOOLS=none PORT=8788 npm start # TOOLS=none turns the agent into the floor\n\n# Terminal 2: ask all 18 questions 3 times, then check and judge every answer\ncd rag-agent/harness\nnpm run eval -- --agent http://localhost:8788 --name floor --runs 3\n# eval first says what it will cost and stops; add --yes to go ahead.\n```\n\nIn code, this could look like:\n\n``` js\nimport { generateText } from \"ai\";\nimport { openai } from \"@ai-sdk/openai\";\n\nconst { text } = await generateText({\n  model: openai(\"gpt-6.1-sol\"),\n  instructions: \"You are an expert tax adviser… You have no tools. Answer from what you know.\",\n  messages: [{ role: \"user\", content: question }],\n});\n```\n\nMine, on 14 Nov. The numbers every change had to beat:\n\n| Run | Score | Strict passes | Confidently wrong | Median cost and time an answer | \n|---|---|---|---|---|\n| The floor: one call, no tools | 0.582 | 25 of 54 | 5 | $0.023, 59 s | \n\nNext, tools. They let the agent search and read the law, the \"retrieval\" in RAG, so it answers from sources instead of memory.\n\nDesign tools like APIs: one job each, with a clear set of arguments. Keep the number of tools low and each one simple.\n\nTool result = data for model; Tool failure = response for the model to act on\n\nA tool essentially looks like this:\n\n```\n// agent/src/tools/index.ts, trimmed\npassage_search: tool({\n  description: \"What the tool does, when to use it, and what it returns\",\n  inputSchema: \"The arguments the model can call it with\",\n  execute: (input) => call(() => searchPassages(library, book, input)),\n}),\n```\n\nThe description is the tool's manual: the model reads it every time.\n\nThe repo has these tools:\n\n`library_search`: finds an Act by its title or by what it covers. Nothing it returns can be cited.`passage_search`: searches inside the Acts by a query, exact words or a section code, and returns the first 300 characters of each hit. Nothing here can be cited either.`library_read`: reads a whole section, or an Act's outline. Only what it returns can be cited, as `[extract:L1]`.` TOOLS=library,web`: `web_search` and `web_read`, for law and official guidance the library lacks. Web search needs a Firecrawl key.\nThe first three are the default, and all the agent needs.\n\nNow the agent itself. It reasons about the question, calls tools, reads what they return, and answers once it has enough. This is a ReAct agent, named after the 2022 paper [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629).\n\nWe build this by using the SDK's `generateText`:\n\n```\n// agent/src/agent.ts, trimmed: the floor's call, with tools and a way to stop\nconst result = await generateText({\n  model,\n  instructions: instructionsFor(names, today),\n  messages: [{ role: \"user\", content: questionMessage(question, country) }],\n  tools,                                                   // library_search, passage_search, library_read\n  stopWhen: [isStepCount(limits.rounds + 1), overBudget],  // 40 rounds of tools, or $3\n  prepareStep,                                             // after the last round: tools off, so it must answer\n});\n```\n\nThat's the whole loop. The SDK calls the model, runs the tools it asks for, sends back the results, and repeats until the model answers without asking for a tool, or a limit stops it.\n\nRun the agent the same way you ran the floor, then compare the two:\n\n```\n# Terminal 1: the agent, with its three library tools (the default)\ncd rag-agent/agent\nnpm start\n\n# Terminal 2\ncd rag-agent/harness\nnpm run eval -- --name agent --runs 3 --yes\nopen results/report.html\n```\n\nIn the report, pick `agent`, then pick `floor` to compare. It shows the difference, the questions won, lost and tied, and which stamps changed. Ours, on 14 Nov:\n\n|  | The floor | The agent | \n|---|---|---|\n| Score | 0.582 | 0.729 | \n| Strict passes | 25 of 54 | 35 of 54 | \n| Confidently wrong | 5 | 0 | \n| Median cost and time an answer | $0.023, 59 s | $0.047, 36 s | \n\n**What's noise and how to deal with it?**\n\nRun the same agent several times with nothing changed, and see how far the score moves. Three runs of the first version of my main build (58 questions, 3 answers each):\n\n| Run | Score | \n|---|---|\n| 1 | 0.742 | \n| 2 | 0.729 | \n| 3 | 0.731 | \n\nNothing changed, yet the score moved by 0.013. Compared question by question, the differences between runs stretched from −0.039 to +0.027. So a change had to win by about 0.03 before I believed it.\n\nTo try a change of your own, change one thing, restart the agent, and run it under a new name:\n\n```\n# for example: edit agent/src/prompt.ts, then restart the agent (Ctrl+C, npm start)\ncd rag-agent/harness\nnpm run eval -- --name agent-my-change --runs 3 --yes\n```\n\nIn my project where this tutorial is derived from, I tested the following, each time evaluating the results and shedding anything that didn't improve the scores. I suggest you try the same. Do not settle on a methodology just because the industry says it. Test it and verify it for your specific use case.\n\n| What I tried | Against the version before | Kept? | \n|---|---|---|\n| The prompt without its expert persona, its \"first understand the question\" paragraph, or both | +0.002, 0.000 and −0.018: all ties | No change | \n| A calculator tool | −0.008 | Dropped | \n| Web search and reading, official sites only | +0.061: at 3× the cost | Off by default | \n| A cheaper model, about 1/20 the price | −0.091 at medium effort, −0.082 at high | No | \n| One prompt line: search each Act by the issue, not only the sections you know | +0.008 on both question sets; 20–33% fewer key sources missed; 15% more cost | Yes | \n| A separate writer after research, four ways, one a reviser that checks the draft against its sources | −0.019 to +0.008; the best cost 59% more | No | \n| A cheaper research model with the strong writer | −0.006, but only 12% cheaper | No | \n\nYou now have an agent that answers NZ tax questions, and the numbers to show it beats the one you started with.\n\nThis simple agentic loop is at the heart of most AI agents. Anthropic puts it plainly: agents \"are typically just LLMs using tools based on environmental feedback in a loop\" ([Building effective agents](https://www.anthropic.com/engineering/building-effective-agents)). Sub-agents, multi-agent systems, and supervisor or router patterns are built from the same piece: loops that call other loops, or code that picks which loop runs. Not everything is an agent, though: many AI apps are fixed workflows that never loop. Keep building and tweaking: the next improvement might be in the golds, the data or the agent itself.\n\n`library.ts` for it.\nRecord everything: every answer, step and verdict. You'll use them to see why an answer went wrong, to score again without paying again, and to test one part on its own, like a new writer on research you already saved.\n\n`agent/runs/`. In production, send the same steps to a tracing tool too. The AI SDK speaks OpenTelemetry: register it once at startup, send the traces to Datadog or similar, and alert on cost per answer, limits hit and tool errors (\n\n``` js\n// npm i @ai-sdk/otel, plus an OpenTelemetry exporter for your tool\nimport { registerTelemetry } from \"ai\";\nimport { OpenTelemetry } from \"@ai-sdk/otel\";\n\nregisterTelemetry(new OpenTelemetry());\n```\n\nLimits are critical: without them, a stuck agent keeps spending. The repo caps each answer at 40 rounds, 8 minutes and $3 ([`stopWhen`](https://github.com/yuvraj2112/rag-agent/blob/main/agent/src/agent.ts#L122) and [a timer](https://github.com/yuvraj2112/rag-agent/blob/main/agent/src/run.ts#L139)). On the last round its tools switch off, so it still answers from what it has read, and after three steps that find nothing new it's nudged to answer or change approach.\n\nPut cost and time next to the score. Web search gave no clear win at 3× the cost, and a separate writer cost 59% more for at most +0.008: neither was worth it.\n\nTreat tool results as data, never instructions. In the repo every result starts with a line saying so, and the prompt repeats it. Set `PLANTS=on` to plant an attack page and check the agent doesn't obey it.\n\nDon't get locked into one provider. Models change price, get retired and go down; a swap is one line in `model.ts`.\n\nMake long runs survive a crash. Durable workflow tools save each step and carry on after a pod restart: try [Temporal](https://temporal.io/), or the AI SDK's [`WorkflowAgent`](https://ai-sdk.dev/docs/agents/workflow-agent), which runs on Vercel's Workflow DevKit.\n\nPut the harness in your CI/CD. `npm test` needs no keys, so run it on every pull request; run the full eval before each release, and stop any release that drops the score by more than the noise. Feature flags let you try a change on a few users first.\n\nKeep checking after release. The code checks need no gold, so run them on live answers. For the judge, write golds for a few real questions each week and add them to your set.\n\nThe agent and the harness, with every file this part uses: [github.com/yuvraj2112/rag-agent](https://github.com/yuvraj2112/rag-agent)\n\nPart 1, the evals: [Build a production-grade RAG Agent: Part 1](https://dev.to/yuvraj2112/building-a-production-grade-rag-agent-2eee)", "url": "https://wpnews.pro/news/build-a-production-grade-rag-agent-part-2", "canonical_source": "https://dev.to/yuvraj2112/build-a-production-grade-rag-agent-part-2-2hbe", "published_at": "2026-11-16 01:45:30+00:00", "updated_at": "2026-11-16 01:50:58.740612+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "artificial-intelligence", "developer-tools"], "entities": ["Vercel AI SDK", "OpenAI", "Anthropic", "SQLite", "legislation.govt.nz", "Pydantic AI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/build-a-production-grade-rag-agent-part-2", "markdown": "https://wpnews.pro/news/build-a-production-grade-rag-agent-part-2.md", "text": "https://wpnews.pro/news/build-a-production-grade-rag-agent-part-2.txt", "jsonld": "https://wpnews.pro/news/build-a-production-grade-rag-agent-part-2.jsonld"}}