# Lessons from building production agentic systems — what actually matters beyond the LLM > Source: > Published: 2026-08-14 07:22:21+00:00 I’ve spent the last year and a half moving projects from “chatbot with a system prompt” to actual agentic systems — ones that plan, call tools, and complete multi-step tasks without a human babysitting every turn. Most tutorials make this look like a weekend project. It isn’t. Here’s the architecture and the failure modes nobody warns you about. **What “agentic” actually requires** Strip away the hype and an agent is four components in a loop: **A reasoning core** — the LLM itself. We’ve used both hosted APIs and open models (Llama 3.x and Qwen 2.5 fine-tunes served via TGI/vLLM). For tool-heavy workloads, a smaller model with good function-calling accuracy beats a bigger model that hallucinates arguments. Benchmark on *your* tool schemas, not leaderboards. **Tool interface layer** — where 70% of real engineering lives. Every tool needs a strict schema, input validation, timeouts, and a machine-readable error format the model can recover from. Our biggest early mistake: returning raw stack traces as tool output. The model would “apologize” and retry the identical broken call forever. Now every tool returns structured errors with a `retryable` flag and a hint field, and loop behavior improved dramatically. **Planning + control flow** — we started with free-form ReAct-style loops. They work in demos and drift in production. What stabilized things: an explicit state machine around the model. The LLM proposes actions; deterministic code decides what’s allowed from the current state. Think of the model as a proposal engine, not the executor. Max-iteration caps and budget ceilings are non-negotiable — an agent that can loop is an agent that *will* loop. **Memory** — three separate problems people conflate: (a) in-context working memory (summarize aggressively — after ~15 tool calls, transcripts poison the context), (b) episodic memory across sessions (we use embedding retrieval over past task outcomes), (c) long-term structured state (just use a database; not everything needs vectors). **The eval problem** This is where most agent projects quietly die. Single-turn evals tell you almost nothing about multi-step behavior. What we landed on: **Trajectory evals**: record full action sequences on a fixed task suite, score terminal state (“did the refund actually get issued?”), not the politeness of the final message. **Regression traps**: every production failure becomes a permanent test case. **LLM-as-judge for intermediate steps**, but only after we calibrated the judge against human labels — uncalibrated judges gave us confident garbage. Rough numbers from our experience: a task with 90% per-step reliability across 5 chained steps lands around 59% end-to-end. That math is the whole reason agent architecture matters. You claw reliability back through validation layers, retries with modified prompts, and checkpoints where a human can intervene — not through prompt wizardry. **Things I’d tell past-me** - Start with **one narrow workflow**, fully instrumented, before generalizing. “General-purpose agent” as v1 is how projects die. - Deterministic guardrails > prompt instructions. If an action is dangerous, gate it in code. “Please don’t delete records” in a system prompt is a wish, not a control. - Log every model call with full context. Agent debugging without traces is archaeology. - Human-in-the-loop isn’t a compromise — for anything irreversible (payments, data mutation, external comms), approval gates are what make deployment survivable. - Fine-tuning helped us far less than better tool design. We assumed we’d need custom models; mostly we needed better schemas and error messages. **Where I’m still unsure** Multi-agent setups. We’ve experimented with planner/executor splits and reviewer agents, and honestly the added coordination overhead ate most of the gains for our use cases. Single well-instrumented agent + good tools has beaten our multi-agent prototypes almost every time. But I see strong claims in the other direction from people doing research-style workloads. So, question for this community: **for those running agents in production — has anyone found a multi-agent architecture that consistently beats a single-agent loop on reliability, not just on benchmark tasks? And what’s your eval setup for multi-step trajectories?** Genuinely curious what’s working outside my bubble.