{"slug": "ai-agents-architecture", "title": "AI Agents: Architecture", "summary": "Neciu Dan, technical co-founder of an ed-tech startup and Staff Software Engineer, published a guide on AI agent architecture, explaining that an AI agent is a model choosing steps inside a loop your code runs, enabled by tool calling added to APIs after ChatGPT's launch in late 2022. The article details the four key differences of AI APIs—non-deterministic, hallucination-prone, per-token pricing, and inability to do math—and distinguishes between a single call, a workflow, and an agent.", "body_md": "· [ ai ](/category/ai) · 21 min read\n\n# AI Agents: Architecture\n\nThis series focuses on building AI agents in production. In the first part we're discussing different types of architecture and how to connect multiple agents working together with all the pros and cons.\n\n## Neciu Dan\n\nHi there, it's Dan, a technical co-founder of an ed-tech startup, host of Señors at Scale - a podcast for Senior Engineers, Organizer of ReactJS Barcelona meetup, international speaker and Staff Software Engineer, I'm here to share insights on combining\ntechnology and education to solve real problems.\n\nI write about startup challenges, tech innovations, and the Frontend Development.\nSubscribe to join me on this journey of transforming education through technology. Want to discuss\nTech, Frontend or Startup life? [Let's connect.](https://www.linkedin.com/in/neciudan/)\n\nIn late 2022, ChatGPT launched. For most of us that was our first contact with AI, and back then we had only two ways to use it: a chat box or a simple API call.\n\nThat changed when the providers added tool calling to their APIs. A model can now ask your code to run a function, read the result, and decide what to do next.\n\nThat combination, a model choosing steps inside a loop your code runs, is what people call an AI agent.\n\nLet’s imagine we have a travel product.\n\nOne of our users types: “find us a week in Portugal in September, near a beach, under 1,500 euros, and we’re bringing a toddler.”\n\nOur code sends that request to the model along with four tools: search flights, search hotels, check airport transfers, and draft an itinerary.\n\nThe model asks to search flights first. Our code runs the search and returns the prices. Mid-September comes back at half the price of early September, so the model shifts the dates and asks for hotels near the coast with cribs. It finds three, notices the cheapest one sits an hour from the airport while the flight lands at 11 pm, then checks the transfer options. Out of that come two complete itineraries, with a note asking whether we’d accept a red-eye to save 200 euros.\n\nThe user picks one. Everybody’s happy, including the agent.\n\nThis guide covers what agents are and how to architect this agentic flow, as part of a multi-article series where will discuss how to setup a harness for your agent, how to run evals and how to build a self improvement loop.\n\nLet’s go!\n\n## What an AI agent is\n\nUnderneath every SDK, an AI call is an HTTP request: you POST text to an endpoint, and you get text back.\n\n``` js\nconst res = await fetch('https://api.anthropic.com/v1/messages', {\n  method: 'POST',\n  headers: { 'x-api-key': KEY, 'content-type': 'application/json' },\n  body: JSON.stringify({\n    model: 'claude-opus-5',\n    max_tokens: 1000,\n    messages: [{ role: 'user', content: 'Write a haiku about databases.' }],\n  }),\n});\n```\n\nIt differs from every other API you’ve integrated in four ways:\n\n**Non-deterministic.** Call it twice with identical input and you get two different answers.\n\n**It doesn’t know when it’s wrong.** A false answer arrives with the same confidence as a true one. There’s no error code for “I made this up.” This is called hallucination.\n\n**You pay per token, both directions.** Your text gets chopped into tokens, chunks of roughly three quarters of a word, before the model reads any of it. A long document in your prompt costs money on every send. In a multi-step system that’s every step.\n\n**It can’t do math.** Counting the characters it wrote is a computation it doesn’t perform. Precise arithmetic is almost always wrong unless you hand it a tool.\n\nNow, the word “agent” itself can mean three different things:\n\n**A single call.** You send one request and you get one response back. Our Portugal message goes in, and one word comes out saying whether the traveler wants a new trip or a change to a booking she already has.\n\n**A workflow.** Several calls wired together by your code, in an order fixed in advance. You draft it, you review it, then you fix whatever the review found.\n\n**An independent agent.** The model decides what happens next and when it’s finished. You don’t know the sequence before it runs.\n\n## Picking a model\n\nAt the top sits a frontier model, their smartest and slowest one, which costs the most per call. Small, fast models fill the bottom of that range, answering in under a second for a fraction of a cent.\n\nSo which one does our vacation agent use? The honest answer is several, because an agent is composed of jobs that each need a different level of intelligence.\n\nFor example the agent that is driving the loop.\n\nThis is the model that reads “find us a week in Portugal with a toddler,” decides that searching for flights comes first, reads the prices that come back, realizes the dates are wrong, and needs better information.\n\nDriving takes judgment. Paying the frontier rate per call gives us a model that picks the right tool the first time and recognizes a bad result when it reads one.\n\nIf we put a small model in the driver’s seat, it picks the wrong tool, misreads results, and loops. Sometimes forever.\n\nThe second kind of job is the simple, repetitive one.\n\nSomewhere in the pipeline, something reads each incoming message and answers one small question about it. Is this email a booking request or a complaint? That question, picking one label from a fixed list, is called **classification**.\n\nWhat’s the traveler’s budget in this message? Pulling a specific value out of free text like that is **extraction**.\n\nAnd once the message has a label, sending it to the right handler, complaints to one prompt, bookings to another, is called **routing**.\n\nAll three go to the cheapest model in the range, since every answer is short and easy to check against the message it came from.\n\nAnother job a model can do is reviewing.\n\nDoes this itinerary make sense for a family with a toddler? Would I send this reply to an angry customer? Nothing below our strongest model answers those.\n\nA judge who’s weaker than the writer approves whatever the writer produces, turning the review step into a rubber stamp.\n\nThere’s one more thing that determines which models we pick: how long the user is willing to wait.\n\nWhen we build something new, we start every job with the strongest model, because our first question is whether the feature works at all. A weak model makes that question unanswerable: when the output is bad, we can’t tell if the idea failed or the model did.\n\nThen we can downgrade later, one job at a time, after we have evals and we can measure the result of the cheaper models vs the frontier models.\n\n## Architecture types\n\nSo far, we’ve decided that our feature needs a model and have picked which models to use. The next decision is how the calls fit together: does one call do everything, do several calls run in a fixed order, or does one model check another’s work?\n\nThose arrangements repeat across products. After you build a few of these, the same ten patterns keep showing up.\n\nThe code examples use a small `ask()`\n\nhelper that wraps the SDK call from the first section: you give it a system prompt, the user content, and optionally tools.\n\n### 1. Single call\n\n```\n  input ──▶ [model] ──▶ output\n```\n\nThis diagram describes any feature that transforms one input into one output.\n\nBefore any agent runs, something has to read each incoming message and decide what the user wants:\n\n```\nasync function classifyRequest(message) {\n  return ask({\n    system: 'Classify this travel message. Reply with JSON: ' +\n            '{ \"intent\": \"new_trip\" | \"change_booking\" | \"question\" }',\n    user: message,\n  });\n}\n```\n\nThis is how we were building agents 3 years ago. And it works, but you will hit the limits pretty fast. As people add more requests to a single message, you end up adding more instructions to your prompt.\n\nYou will start adding: “and if the message mentions a booked trip, also…”, then another clause, then another. The accuracy drops with each one, because the model juggles too many rules at once.\n\n### 2. Chain\n\n```\n  input ──▶ [call 1] ──▶ [call 2] ──▶ [call 3] ──▶ output\n```\n\nA chain runs a fixed sequence where each output feeds the next call. For example, every itinerary the agent produces gets turned into a friendly confirmation email, in the user’s language.\n\n``` js\nasync function confirmationEmail(itinerary, userLang) {\n  const draft  = await ask({ system: WRITE_CONFIRMATION_PROMPT, user: itinerary });\n  const local  = await ask({ system: `Translate to ${userLang}.`, user: draft });\n  const report = await ask({ system: CHECK_NUMBERS_PROMPT,\n                             user: JSON.stringify({ itinerary, email: local }) });\n  if (report.mismatches.length === 0) return local;\n  return ask({ system: FIX_NUMBERS_PROMPT, user: JSON.stringify({ local, report }) });\n}\n```\n\nThe steps never change order; no matter what the input says, that means our code owns the sequence.\n\nBut be aware that errors compound down a chain, because every step inherits the previous step’s mistakes and adds its own. If we chain five steps that each succeed 95% of the time, the arithmetic leaves us a run that succeeds about 77% of the time. (60% of the time, it works everytime)\n\n### 3. Router\n\n```\n             ┌──▶ [refund handler]\n  input ──▶ [classify] ──▶ [bug handler]\n             └──▶ [sales handler]\n```\n\nA router classifies first, then dispatches to a specialist prompt. Our new-trip handler knows how to gather requirements for the agent, our change-booking handler knows the rebooking rules, and our question handler knows the product FAQ, without any of them carrying the others’ rules.\n\n``` js\nconst HANDLERS = {\n  new_trip:       { system: NEW_TRIP_PROMPT },\n  change_booking: { system: CHANGE_BOOKING_PROMPT },\n  question:       { system: QUESTION_PROMPT },\n};\n\nasync function respond(message) {\n  const { intent } = await classifyRequest(message);  // shape 1, reuse\n  return ask({ ...HANDLERS[intent], user: message.body });\n}\n```\n\nBut it can easily go wrong, first you need all the options beforehand and then when the classifier sends a change-booking request to the question handler, the question handler doesn’t complain; it produces a confident, well-written FAQ answer to someone whose flight leaves tomorrow.\n\n### 4. Fan-out and voting\n\n```\n  input ──┬──▶ [chapter 1] ──┐\n          ├──▶ [chapter 2] ──┼──▶ combine ──▶ output\n          └──▶ [chapter 3] ──┘\n```\n\nFan-out splits independent work and runs it in parallel.\n\n```\nasync function compareDestinations(requirements, cities) {   \n// ['Lisbon', 'Faro', 'Madeira']\n  const briefs = await Promise.all(\n    cities.map(c => ask({ system: DESTINATION_BRIEF_PROMPT,\n                          user: JSON.stringify({ city: c, requirements }) }))\n  );\n  return ask({ system: COMPARE_PROMPT, user: briefs.join('\\n---\\n') });\n}\n```\n\nEach brief was written blind to the others, so if Faro’s brief doesn’t know Lisbon’s flights cost half as much, the final comparison rests on three documents that dont know anything about each other.\n\nVoting runs the same task several times and aggregates the answers.\n\n``` js\nasync function readFareTotal(farePage) {\n  const reads = await Promise.all([1, 2, 3].map(() =>\n    ask({ system: EXTRACT_FARE_PROMPT, user: farePage })\n  ));\n  const counts = tally(reads.map(r => r.total));\n  const [winner, votes] = counts[0];\n  return votes >= 2 ? { total: winner } : { needsHuman: true, reads };\n}\n```\n\n### 5. Generator plus reviewer\n\n```\n  input ──▶ [writer] ──▶ draft ──▶ [reviewer] ──┬─▶ good ──▶ output\n                 ▲                              │\n                 └────── \"fix these\" ───────────┘\n```\n\nA second model critiques whatever the first one writes, so the writer revises against that critique until it passes or until we run out of rounds.\n\n``` js\nasync function draftWithReview(input) {\n  let draft = await ask({ system: WRITER_PROMPT, user: input });\n\n  for (let round = 0; round < 2; round++) {           // the bound\n    const review = await ask({ system: REVIEWER_PROMPT, user: draft });\n    if (review.approved) return draft;\n    draft = await ask({\n      system: WRITER_PROMPT,\n      user: `${input}\\n\\nRevise this draft. Fix: ${review.issues.join('; ')}`,\n    });\n  }\n  return draft;   // out of rounds; ships with the last revision\n}\n```\n\nThis architecture also has some issues, like a reviewer approving everything because its prompt told it to, a reviewer judging a property it was never shown examples of, and a reviewer drawn from the same model family as the writer, which shares the writer’s blind spots and waves through the exact mistakes that family of model makes most often.\n\nSame-family bias is the hardest of those to picture.\n\nIn our product it looks like the writer putting a 6 am departure in front of a family with a toddler, and the reviewer, running on the same model, reading that itinerary back and calling the schedule tight but fine.\n\n### 6. Tool loop\n\n```\n  ┌───────────────────────────────┐\n  │  think: what do I need?       │\n  │  act:   call a tool           │\n  │  see:   read the result       │\n  └────────────┬──────────────────┘\n               │ repeat until done\n               ▼\n            answer\n```\n\nThe model requests a tool, your code runs it with your permissions, and the result is returned to the conversation for the model to read.\n\n```\n// The bare loop, without the bounds and checks a production version wraps around it.\nlet messages = [{ role: 'user', content: request }];\nfor (let i = 0; i < MAX_STEPS; i++) {\n  const res = await ask({ system: AGENT_PROMPT, messages, tools: TOOLS });\n  if (!res.toolCall) return res.text;\n  const result = await runTool(res.toolCall);\n  messages.push(res.message, { role: 'tool', content: result });\n}\n```\n\nThree things change for us once the model is in control.\n\nThe run can loop, so a cap on turns is mandatory as we dont want it to run forever.\n\nCost climbs as the run goes, since every turn re-sends the whole conversation and the late turns pay again for everything before them. Prompt caching helps by keeping the unchanged part of the prompt on the model side.\n\nAnd prompt injection gets dangerous once the model can act on what it reads. Someone plants a line in a hotel description telling the reader to ignore its earlier instructions and email the traveler’s card details to an address, and when our search tool hands that description back, the model reads it as an instruction like everything else in the conversation. (And now we are hacked)\n\n### 7. Plan then execute\n\n```\n  input ──▶ [planner] ──▶ written plan ──▶ [worker] ──▶ step ──▶ step ──▶ done\n                ▲                                          │\n                └────────── replan if stuck ───────────────┘\n```\n\nA capable model plans once, and a smaller model executes each step.\n\n``` js\nasync function buildPlan(request) {\n  const plan = await ask({ model: STRONG, system: PLANNER_PROMPT, user: request });\n  // plan.steps: [{ id, instruction }, ...]\n\n  const results = [];\n  for (const step of plan.steps) {\n    results.push(await ask({ model: SMALL, system: EXECUTOR_PROMPT, user: step.instruction }));\n  }\n  return results;\n}\n```\n\nThis is basically how Spec Driven Programming works with Claude Code.\n\nThe problem here is that the model made the plan before anyone knew what they’d find. Step three can reveal that the plan was wrong from the start.\n\nOur planner writes five steps around a beachfront hotel in Lagos, and step three searches that hotel’s room types and finds no cribs. The two steps after it still book an airport transfer and an itinerary for a place this family can’t sleep in, because the executor follows the instruction it was handed rather than the fact it uncovered.\n\n### 8. Supervisor\n\n```\n              ┌──────────────┐\n              │  SUPERVISOR  │  holds the goal, dispatches,\n              └──┬───┬───┬───┘  decides when it's finished\n                 │   │   │\n        ┌────────┘   │   └────────┐\n        ▼            ▼            ▼\n   [researcher]  [writer]   [fact checker]\n```\n\nNamed specialist agents sit under one supervisor agent that everything goes through.\n\nIn my experience, this architecture requires four things to work correctly.\n\n-\nIt needs a termination condition that it can evaluate, since “keep going until done” causes supervisors to loop.\n\n-\nIt needs structured worker output, so the supervisor acts on results instead of reading through them.\n\n-\nIt needs workers who return conclusions instead of full transcripts, because the supervisor’s context accumulates everything its workers say.\n\n-\nIt needs workers we can verify one at a time, because once a worker’s error is folded into the supervisor’s aggregated answer, nothing downstream can tell us which worker got it wrong.\n\nThe basic implementation is a loop where the supervisor’s output is a dispatch decision:\n\n``` js\nlet state = { requirements, findings: [] };\nwhile (!state.done) {\n  const decision = await ask({ model: STRONG, system: SUPERVISOR_PROMPT,\n                               user: JSON.stringify(state) });\n  // decision: { worker: 'flights' | 'hotels' | 'activities' | 'finish', task }\n  if (decision.worker === 'finish') break;\n  const result = await WORKERS[decision.worker](decision.task);\n  state.findings.push({ worker: decision.worker, result });   // conclusions, never transcripts\n}\n```\n\n### 9. Handoff\n\n```\n   [triage] ──▶ [billing] ──▶ [refunds]\n                    ▲            │\n                    └────────────┘\n```\n\nPeers pass control directly, with no coordinator above them.\n\nWe see this on our travel product’s support side: a triage agent hands the conversation to the booking-changes agent. Halfway through, the user mentions the airline already canceled the flight, which makes it a refund case for a different specialist.\n\n``` js\nlet current = 'triage';\nfor (let turn = 0; turn < GLOBAL_BUDGET; turn++) {   // nobody owns \"done\", so the budget does\n  const res = await AGENTS[current].respond(conversation);\n  conversation.push(res.message);\n  if (res.handoffTo) current = res.handoffTo;        // changes passes to refunds mid-thread\n  else if (res.finished) return conversation;\n}\n```\n\nHandoffs fail in a specific, almost comic way: no single agent owns “we’re done,” so control bounces between changes and refunds indefinitely, with each one politely handing the customer back to the other.\n\n### 10. Human in the loop\n\n```\n  [agent] ──▶ proposal ──▶ [you] ──┬─▶ approve ──▶ do it\n                                   ├─▶ edit ──▶ do the edited version\n                                   └─▶ reject\n```\n\nIn this shape, a person checks the agent’s work before anything irreversible happens. The agent drafts the email and you press send, or it stops at a proposed migration until you approve.\n\nThe implementation is a pause: the agent writes a proposal and stops, and a separate handler handles the response when the human responds.\n\n```\nasync function proposeBooking(runId, itinerary) {\n  await db.saveProposal(runId, { itinerary, status: 'awaiting_approval' });\n  await notifyUser(runId);          // the run now sits idle, costing nothing\n}\n\nasync function onUserDecision(runId, decision) {      // approve | edit | reject\n  await db.recordDecision(runId, decision);           // the training data, one row\n  if (decision.action !== 'reject') {\n    await bookTrip(decision.itinerary);               // the edited version, if edited\n  }\n}\n```\n\nThis implementation produces the best training data you will ever collect, because approve, edit, and reject are graded labels generated by users in the course of their work.\n\n## Picking one architecture\n\n| Your situation | Architecture | Number |\n|---|---|---|\n| One well-defined transformation | Single call | 1 |\n| Stages you can name in advance | Chain | 2 |\n| Your prompt fills up with “if the user asks about X…” | Router | 3 |\n| Independent pieces of one big job, or a wrong answer that costs more than three model calls | Fan-out and voting | 4 |\n| Quality improves when told what’s wrong | Generator + reviewer | 5 |\n| The next step depends on what it finds | Tool loop | 6 |\n| Many simple steps, cost matters | Plan then execute | 7 |\n| Named specialists, one accountable place | Supervisor | 8 |\n| Specialist unknowable until it unfolds | Handoff | 9 |\n| Anything irreversible | Human in the loop | 10 |\n\nAs you can imagine, its not as simple as it looks. There are all sort of issues and problems when building multi-agent systems. Here are some that I encountered:\n\n-\nMultiple agents multiply what we spend.\n\n-\nEvery extra worker multiplies the chance of being wrong, without us knowing\n\n-\nPassing information between agents takes longer than it looks.\n\n-\nWhen agents are expecting something concrete and they receive something else, they dont complain and do what they are told, which usually means bad results\n\nWhat you should do before committing to multiple agents: describe each agent’s job in one sentence, then ask whether another engineer would independently agree which agent handles a given input.\n\n## Our architecture for the vacation agent\n\nEverything so far was theory, so let’s run it against our original example. A user lands on our website and types this into our one chat interface:\n\n“My husband and I want to go on vacation, find us a week in Portugal in September, near a beach, under 1,500 euros, and we’re bringing a toddler.”\n\nHow would we architect this to optimize results and costs?\n\nWe can trace the request through the architectures above. At every stage we ask the same two questions: which architecture fits this piece of work, and which model tier does the job deserve.\n\nThe first thing that touches the message is a classifier, our first architecture, using a small, dumb, cheaper model.\n\nIt answers one question with one label, picked from the three intents `classifyRequest`\n\nalready knows about, and this message comes back with `new_trip`\n\n. We learned earlier that classification has a checkable answer, so the cheap model handles it for a fraction of a cent.\n\nThat label feeds our router form architecture 3.\n\nA `new_trip`\n\ngoes to the trip planner, a `change_booking`\n\ngoes to the rebooking handler, and a `question`\n\ngets the FAQ prompt. Our user’s message routes to the trip planner.\n\nThe trip planner is the tool loop, architecture 6, and here is where we need the frontier model because dates can change, we need to consider transfer to the hotel, different types of rooms. We need judgement.\n\nWhen the loop produces an itinerary, we do a plain code check against the given budget (if there is any). We extract the budget with an extraction agent and call this function:\n\n```\nfunction checkBudget(itinerary, budgetEur) {\n  if (budgetEur == null) {\n    return { skipped: 'no budget stated' };   // recorded, so a silent skip\n  }                                           // never masquerades as a pass\n  const total = itinerary.items.reduce((sum, i) => sum + i.priceEur, 0);\n  return total > budgetEur\n    ? { violations: [`Total ${total} EUR exceeds ${budgetEur} EUR. Cut ${total - budgetEur} EUR.`] }\n    : { violations: [] };\n}\n```\n\nThen an itinerary that passes the code check meets the reviewer, architecture 5, which also runs on the frontier model.\n\nIt judges what code can’t: does this trip make sense with a toddler, and is an 11 pm landing followed by an hour of transfer something we’d propose to this family?\n\nWe run the same flow two times (second time it has the first itenerary in context).\n\nThen we hand the user the two options and let her pick one. This is the human-in-the-loop approach. Her approval, edit, or rejection lands in `recordDecision`\n\nas a labeled example of what she wanted. (And can be used to fine-tune the model in the future)\n\nAfter she approves and selects one itenerary, we trigger one more run just to confirm that all prices have not changed just to be extra safe that we dont charge the user more than she agreed to.\n\nAnd after the booking is confirmed, the chain from architecture 2 takes over on the small model: it drafts her personalised confirmation email.\n\nWe can put the whole flow on one diagram:\n\n```\n  message\n     │\n     ▼\n  [classify]        architecture 1, small model\n     │  new_trip\n     ▼\n  [router]          architecture 3\n     │\n     ▼\n  [extract]         architecture 1 again, small model\n     │  { budgetEur, month, nights, toddler, near }\n     ▼\n  [tool loop]       architecture 6, frontier model\n     │  itinerary        search_flights / search_hotels / check_transfers\n     ▼\n  [budget check]    plain code, zero tokens\n     │\n     ▼\n  [reviewer]        architecture 5, frontier model, two rounds max\n     │  proposal\n     ▼\n  [the user]        architecture 10: approve / edit / reject\n     │  approve\n     ▼\n  [fare voting]     architecture 4, three reads, small model\n     │\n     ▼\n  [book] ──▶ [confirmation chain]   architecture 2, small model\n```\n\nAn example of how it looks put together:\n\n```\nasync function handleMessage(runId, message) {\n  const { intent } = await classifyRequest(message);        // architecture 1, small model\n\n  if (intent !== 'new_trip') {\n    return HANDLERS[intent](message);                       // architecture 3\n  }\n\n  const req = await extractRequirements(message);           // small model, nulls for gaps\n  if (req.budgetEur == null) {\n    return askUser(runId, 'What budget should I plan against?');\n  }\n\n  const itinerary = await runTripLoop(message, req);        // the architecture 6 loop from above,\n                                                            // re-running until checkBudget\n                                                            // on its exit passes\n  const proposal  = await draftWithReview(itinerary);       // architecture 5\n\n  await proposeBooking(runId, proposal);                    // architecture 10; we stop and wait\n}\n\n// onUserDecision from architecture 10 calls this when the decision is an approval\nasync function bookApprovedTrip(runId, decision) {\n  const fare = await readFareTotal(decision.itinerary);     // architecture 4, three reads\n  if (fare.needsHuman) return escalate(runId, fare);        // a human queue, not a retry\n\n  const booking = await bookTrip(decision.itinerary);\n  return confirmationEmail(booking, decision.userLang);     // architecture 2, small model\n}\n```\n\n## Why not one call to one powerful model?\n\nEverything above looks complicated, so why not push the whole job into one frontier model and have it handle all of this?\n\nThe frontier model could handle the whole job on its own. We hand it the 4 tools and one big prompt.\n\nIt classifies the intent itself, identifies the 1,500 budget in the text, plans, evaluates its own plan, and writes the confirmation email at the end.\n\nIn a demo, it produces the same two itineraries as our pipeline, with a tenth of the code.\n\nBut at scale, our pipeline has at least four benefits:\n\n-\nMoney. Every token the solo version touches, input and output, is billed at frontier rates, where the frontier model costs 5 to 10 times more per token than the small one. Especially because the growing transcript adds an extra charge, because the loop re-sends the conversation on every turn. Then the classification, extraction, and confirmation email still run as frontier tokens.\n\n-\nSeams. Meaning places where our code can step in. Our\n\n`checkBudget`\n\nruns between the loop and the reviewer, reads a plain JSON itinerary, and feeds exact numbers back. If we had only the frontier model do the whole thing then the budget arithmetic happens inside the model, which is the one place we’ve established it sometimes fails. -\nVisibility. In our pipeline when a run goes wrong, we can see which part failed. When the frontier model proposes a hotel with no crib, we’re left staring at one long transcript, asking whether the model missed the toddler while reading, ignored it while planning, or failed to catch it while reviewing itself. Our pipeline leaves an artifact at every stage: a label, a requirements object, an itinerary, a review verdict.\n\n-\nOptimisations. With separate jobs, we can move one job at a time to a cheaper model and measure whether anything got worse and fine tune each model as we see fit.\n\nAlso keep in mind that the architecture is still missing everything that keeps an agent alive in production: records, keeping the model alive, shared memory, etc.\n\nEverything that wraps the model calls is called the harness.\n\nWe’re gonna discuss it in the next part.\n\n## References\n\n[Effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)- Anthropic[Building effective agents](https://www.anthropic.com/engineering/building-effective-agents)- Anthropic[Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)- Claude Platform docs[Don’t Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks](https://arxiv.org/abs/2601.06007)- the 41 to 80% measurement across three providers\n\n### Discover more from The Neciu Dan Newsletter\n\nA weekly column on Tech & Education, startup building and occasional hot takes.\n\nOver 1,000 subscribers", "url": "https://wpnews.pro/news/ai-agents-architecture", "canonical_source": "https://neciudan.dev/ai-agents-architecture", "published_at": "2026-08-12 00:00:00+00:00", "updated_at": "2026-08-12 09:36:21.585739+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-infrastructure"], "entities": ["Neciu Dan", "ChatGPT", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/ai-agents-architecture", "markdown": "https://wpnews.pro/news/ai-agents-architecture.md", "text": "https://wpnews.pro/news/ai-agents-architecture.txt", "jsonld": "https://wpnews.pro/news/ai-agents-architecture.jsonld"}}