Part 1 of my agent build series
Let me set the scene.
I wanted a Perplexity-style search assistant — type a question, watch it decide to go search the web mid-answer, get back a real answer with numbered citations you can click. Not a chatbot wearing a search icon. Something that actually reasons about whether it needs to look something up before it does.
So I built it. Bun backend, React 19 frontend, Tavily for search, OpenRouter for the model, Postgres underneath. A few weeks in, it worked. Streaming answers, clickable sources, follow-up questions, the whole thing.
Then I went to write this post, typed the words "agent harness" into the title, and stopped.
Was that actually true? Or was I about to publish a buzzword on top of a fetch call with a nice system prompt?
Here's the ambitious version of this post I almost wrote: "I built a multi-tool agentic harness with full observability and provider failover."
None of that is true. There's one tool. One model provider path. No retries if a tool call fails mid-turn.
So before publishing, I went back through my own agent-loop.ts
and agent-runner.ts
like I was reviewing someone else's PR, and asked the boring question: what actually makes something a "harness" instead of a script that calls an API?
A harness is the code around the model — the part that decides when the model gets to call a tool, manages the back-and-forth, keeps context sane, and turns the mess into something a frontend can render. Not the model itself. The scaffolding.
Checked against that definition, line by line:
if (needsSearch)
gate before the LLM sees the query. It gets handed a web_search
tool in its schema and chooses whether to use it.agentLoop()
keeps alternating — ask the LLM, execute whatever tool it called, feed the result back, ask again — until the model returns a stop reason instead of a tool-use reason.So — yes, it's a harness. A real one. Just a small one.
The honest caveat:it's a single-tool harness.web_search
, full stop. The multi-tool version — file access, code execution, retries, provider fallback — is a different tier of engineering, and I'm not there yet. I'd rather say that plainly than have someone read the repo and feel oversold by the title.
This is the part that turns "an LLM with instructions" into something that behaves like an agent instead of a script:
// backend/src/agent/agent-runner.ts
const context: AgentContext = {
systemPrompt: getSystemPrompt(),
messages: [...(options.history ?? [])],
tools: [webSearchTool],
};
const stream = agentLoop([userMessage], context, config, signal, getDefaultStreamFn());
for await (const event of stream) {
if (event.type === "tool_execution_end" && event.toolName === "web_search") {
options.onEvent({ type: "sources", items: registry.list() });
}
if (event.type === "message_update" && event.assistantMessageEvent.type === "text_delta") {
options.onEvent({ type: "delta", text: event.assistantMessageEvent.delta });
}
}
My code never decides whether to search — it just watches the event stream coming out of the loop and forwards the right pieces downstream. Model decides, harness routes. That split is the entire reason I'm comfortable calling this a harness instead of a script with a search button bolted on.
// backend/src/tools/web-search.ts
export function createWebSearchTool(defaultDepth, registerResult) {
return {
name: "web_search",
parameters: Type.Object({
query: Type.String(),
searchDepth: Type.Optional(Type.Union([
Type.Literal("basic"),
Type.Literal("advanced"),
])),
}),
async execute(_id, params) {
const results = await webSearch(params.query, params.searchDepth ?? defaultDepth);
const text = results.map((r) => {
const { index, title, url } = registerResult?.(r) ?? r;
return `[${index}] ${title}\nURL: ${url}\n${r.content}`;
}).join("\n\n");
return { content: [{ type: "text", text }], details: { results } };
},
};
}
The Tavily call itself is the boring part. The part that actually bit me is registerResult
— every result gets numbered and deduped through a shared registry the instant it comes back, so when the model writes [1]
in its answer, that index has to map to a source the frontend already has sitting in its sidebar. Get the timing wrong — number it after the model already referenced it, or dedupe against the wrong run — and citations silently point at nothing. No error, no crash. Just a [3]
in the answer with nothing behind it. That one took a full evening to notice, because it doesn't fail loud.
Frontend searchMode
|
Tavily depth |
|---|---|
"search" |
basic |
"research" |
advanced |
I want to address this directly because it's the obvious simpler design, and I genuinely started building it that way first.
Force a search before every LLM call. No decision logic, no tool schema, no risk of the model "forgetting" to search when it should. Easier to reason about, easier to test.
It's wrong for a chat agent, and here's the concrete reason: half of what people type into a thread is a follow-up — "what about for React 19 instead?" — that needs the model to reread its own last answer, not burn a fresh Tavily call and 2+ seconds of latency for a question it can already answer from context.
Let the model decide when it actually needs fresh information. That's the whole reason this is tool-first instead of pipeline-first. It's also the design choice that makes the word "harness" earn its keep here — the intelligence about when to act lives inside the loop, not hardcoded in front of it.
Client (React 19 + assistant-ui)
│
▼ POST /ask + Bearer JWT
┌─────────────────────────────────────────────┐
│ Express 5 · /ask route │
│ auth middleware → credit check → runner │
└───────┬───────────────────────┬──────────────┘
▼ ▼
agent-runner.ts Postgres (Prisma 7)
systemPrompt + tools history, credits
│
▼
agent-loop.ts ── tool-first loop ──┐
│ │
▼ ▼
OpenRouter (LLM) Tavily (web_search)
| Layer | Technology |
|---|---|
| Runtime | Bun, Express 5 |
| Agent | |
@earendil-works/pi-ai + OpenRouter |
|
| Search | Tavily API |
| Database | PostgreSQL + Prisma 7 |
| Auth | Supabase (JWT) |
| Frontend | React 19, Vite 8, assistant-ui, Tailwind CSS 4 |
Two independent apps, no Docker, no shared infra to babysit. The backend has to know about exactly one thing outside itself at request time: which model to call and which tool to run.
| Plain LLM chat | v0 (this harness) | |
|---|---|---|
| Freshness | Frozen at training time | Live web results, only when the model asks for them |
| Citations | None, or hallucinated | Numbered, deduped, clickable, persisted to the DB |
| Follow-ups | Re-explains from scratch | Reuses trimmed history, no repeat search |
| Cost control | None | Per-user credit gate (creditLimit default 10), search depth toggle |
| Delivery | Wait for the full response | NDJSON stream, renders as it's generated |
None of that is exotic engineering. That's kind of the point — a harness doesn't need to be clever, it needs to be correct about the loop. The clever part is knowing which one line-item on that table you're actually solving before you write any code.
This is v0 on purpose. A few things actually on deck for v1, not just aspirational bullet points:
search.ts
, models.ts
, and stream-fn.ts
are already written as swap points, but I've only ever run this against Tavily + OpenRouter. An abstraction nobody's swapped is just a hope.Code's here if you want to see the whole thing: github.com/Saurabhsing21/Lumina
Loop internals specifically: docs/AGENT_LOOP.md
If you think one tool doesn't earn the word "harness," or you've hit the same silent-citation bug I did — tell me in the comments. I check daily.
#ai
#agents
#opensource
#llm
#webdev
#buildinpublic