{"slug": "i-built-a-web-search-agent-harness-then-i-checked-if-it-actually-deserved-the", "title": "I Built a Web Search Agent Harness. Then I Checked If It Actually Deserved the Name.", "summary": "A developer built a Perplexity-style web search agent using Bun, React 19, Tavily, OpenRouter, and Postgres, then critically evaluated whether it truly deserved the 'agent harness' label. The engineer concluded that while it is a real but single-tool harness, the code's separation of model decision-making from tool routing is what justifies the term.", "body_md": "*Part 1 of my agent build series*\n\nLet me set the scene.\n\nI wanted a Perplexity-style search assistant — type a question, watch it decide to go search the web mid-answer, get back a real answer with numbered citations you can click. Not a chatbot wearing a search icon. Something that actually reasons about *whether* it needs to look something up before it does.\n\nSo I built it. Bun backend, React 19 frontend, Tavily for search, OpenRouter for the model, Postgres underneath. A few weeks in, it worked. Streaming answers, clickable sources, follow-up questions, the whole thing.\n\nThen I went to write this post, typed the words \"agent harness\" into the title, and stopped.\n\nWas that actually true? Or was I about to publish a buzzword on top of a fetch call with a nice system prompt?\n\nHere's the ambitious version of this post I almost wrote: *\"I built a multi-tool agentic harness with full observability and provider failover.\"*\n\nNone of that is true. There's one tool. One model provider path. No retries if a tool call fails mid-turn.\n\nSo before publishing, I went back through my own `agent-loop.ts`\n\nand `agent-runner.ts`\n\nlike I was reviewing someone else's PR, and asked the boring question: what actually makes something a \"harness\" instead of a script that calls an API?\n\nA harness is the code *around* the model — the part that decides when the model gets to call a tool, manages the back-and-forth, keeps context sane, and turns the mess into something a frontend can render. Not the model itself. The scaffolding.\n\nChecked against that definition, line by line:\n\n`if (needsSearch)`\n\ngate before the LLM sees the query. It gets handed a `web_search`\n\ntool in its schema and chooses whether to use it.`agentLoop()`\n\nkeeps alternating — ask the LLM, execute whatever tool it called, feed the result back, ask again — until the model returns a stop reason instead of a tool-use reason.So — yes, it's a harness. A real one. Just a small one.\n\nThe honest caveat:it's a single-tool harness.`web_search`\n\n, full stop. The multi-tool version — file access, code execution, retries, provider fallback — is a different tier of engineering, and I'm not there yet. I'd rather say that plainly than have someone read the repo and feel oversold by the title.\n\nThis is the part that turns \"an LLM with instructions\" into something that behaves like an agent instead of a script:\n\n``` js\n// backend/src/agent/agent-runner.ts\nconst context: AgentContext = {\n  systemPrompt: getSystemPrompt(),\n  messages: [...(options.history ?? [])],\n  tools: [webSearchTool],\n};\n\nconst stream = agentLoop([userMessage], context, config, signal, getDefaultStreamFn());\n\nfor await (const event of stream) {\n  if (event.type === \"tool_execution_end\" && event.toolName === \"web_search\") {\n    options.onEvent({ type: \"sources\", items: registry.list() });\n  }\n  if (event.type === \"message_update\" && event.assistantMessageEvent.type === \"text_delta\") {\n    options.onEvent({ type: \"delta\", text: event.assistantMessageEvent.delta });\n  }\n}\n```\n\nMy code never decides *whether* to search — it just watches the event stream coming out of the loop and forwards the right pieces downstream. Model decides, harness routes. That split is the entire reason I'm comfortable calling this a harness instead of a script with a search button bolted on.\n\n```\n// backend/src/tools/web-search.ts\nexport function createWebSearchTool(defaultDepth, registerResult) {\n  return {\n    name: \"web_search\",\n    parameters: Type.Object({\n      query: Type.String(),\n      searchDepth: Type.Optional(Type.Union([\n        Type.Literal(\"basic\"),\n        Type.Literal(\"advanced\"),\n      ])),\n    }),\n    async execute(_id, params) {\n      const results = await webSearch(params.query, params.searchDepth ?? defaultDepth);\n      const text = results.map((r) => {\n        const { index, title, url } = registerResult?.(r) ?? r;\n        return `[${index}] ${title}\\nURL: ${url}\\n${r.content}`;\n      }).join(\"\\n\\n\");\n      return { content: [{ type: \"text\", text }], details: { results } };\n    },\n  };\n}\n```\n\nThe Tavily call itself is the boring part. The part that actually bit me is `registerResult`\n\n— every result gets numbered and deduped through a shared registry the instant it comes back, so when the model writes `[1]`\n\nin its answer, that index has to map to a source the frontend already has sitting in its sidebar. Get the timing wrong — number it after the model already referenced it, or dedupe against the wrong run — and citations silently point at nothing. No error, no crash. Just a `[3]`\n\nin the answer with nothing behind it. That one took a full evening to notice, because it doesn't fail loud.\n\nFrontend `searchMode`\n|\nTavily depth |\n|---|---|\n`\"search\"` |\n`basic` |\n`\"research\"` |\n`advanced` |\n\nI want to address this directly because it's the obvious simpler design, and I genuinely started building it that way first.\n\nForce a search before every LLM call. No decision logic, no tool schema, no risk of the model \"forgetting\" to search when it should. Easier to reason about, easier to test.\n\nIt's wrong for a chat agent, and here's the concrete reason: half of what people type into a thread is a follow-up — *\"what about for React 19 instead?\"* — that needs the model to reread its own last answer, not burn a fresh Tavily call and 2+ seconds of latency for a question it can already answer from context.\n\n**Let the model decide when it actually needs fresh information.** That's the whole reason this is tool-first instead of pipeline-first. It's also the design choice that makes the word \"harness\" earn its keep here — the intelligence about *when* to act lives inside the loop, not hardcoded in front of it.\n\n```\nClient (React 19 + assistant-ui)\n        │\n        ▼  POST /ask  +  Bearer JWT\n┌─────────────────────────────────────────────┐\n│           Express 5 · /ask route             │\n│    auth middleware → credit check → runner   │\n└───────┬───────────────────────┬──────────────┘\n        ▼                       ▼\n   agent-runner.ts         Postgres (Prisma 7)\n   systemPrompt + tools     history, credits\n        │\n        ▼\n   agent-loop.ts  ── tool-first loop ──┐\n        │                              │\n        ▼                              ▼\n   OpenRouter (LLM)               Tavily (web_search)\n```\n\n| Layer | Technology |\n|---|---|\n| Runtime | Bun, Express 5 |\n| Agent |\n`@earendil-works/pi-ai` + OpenRouter |\n| Search | Tavily API |\n| Database | PostgreSQL + Prisma 7 |\n| Auth | Supabase (JWT) |\n| Frontend | React 19, Vite 8, assistant-ui, Tailwind CSS 4 |\n\nTwo independent apps, no Docker, no shared infra to babysit. The backend has to know about exactly one thing outside itself at request time: which model to call and which tool to run.\n\n| Plain LLM chat | v0 (this harness) | |\n|---|---|---|\n| Freshness | Frozen at training time | Live web results, only when the model asks for them |\n| Citations | None, or hallucinated | Numbered, deduped, clickable, persisted to the DB |\n| Follow-ups | Re-explains from scratch | Reuses trimmed history, no repeat search |\n| Cost control | None | Per-user credit gate (`creditLimit` default 10), search depth toggle |\n| Delivery | Wait for the full response | NDJSON stream, renders as it's generated |\n\nNone of that is exotic engineering. That's kind of the point — a harness doesn't need to be clever, it needs to be *correct about the loop*. The clever part is knowing which one line-item on that table you're actually solving before you write any code.\n\nThis is v0 on purpose. A few things actually on deck for v1, not just aspirational bullet points:\n\n`search.ts`\n\n, `models.ts`\n\n, and `stream-fn.ts`\n\nare already written as swap points, but I've only ever run this against Tavily + OpenRouter. An abstraction nobody's swapped is just a hope.Code's here if you want to see the whole thing: [github.com/Saurabhsing21/Lumina](https://github.com/Saurabhsing21/Lumina)\n\nLoop internals specifically: `docs/AGENT_LOOP.md`\n\n*If you think one tool doesn't earn the word \"harness,\" or you've hit the same silent-citation bug I did — tell me in the comments. I check daily.*\n\n`#ai`\n\n`#agents`\n\n`#opensource`\n\n`#llm`\n\n`#webdev`\n\n`#buildinpublic`", "url": "https://wpnews.pro/news/i-built-a-web-search-agent-harness-then-i-checked-if-it-actually-deserved-the", "canonical_source": "https://dev.to/saurabh_singh_86cdc588e85/i-built-a-web-search-agent-harness-then-i-checked-if-it-actually-deserved-the-name-4o0g", "published_at": "2026-07-31 10:53:04+00:00", "updated_at": "2026-07-31 11:06:09.227978+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools"], "entities": ["Bun", "React 19", "Tavily", "OpenRouter", "Postgres"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-web-search-agent-harness-then-i-checked-if-it-actually-deserved-the", "markdown": "https://wpnews.pro/news/i-built-a-web-search-agent-harness-then-i-checked-if-it-actually-deserved-the.md", "text": "https://wpnews.pro/news/i-built-a-web-search-agent-harness-then-i-checked-if-it-actually-deserved-the.txt", "jsonld": "https://wpnews.pro/news/i-built-a-web-search-agent-harness-then-i-checked-if-it-actually-deserved-the.jsonld"}}