{"slug": "context-engineering-for-agents-a-practical-guide", "title": "Context Engineering for Agents: A Practical Guide", "summary": "Context engineering, the practice of filling an LLM's context window with precisely the right information for each step, has replaced prompt engineering as the critical skill for building and using AI agents, according to a practical guide from Malt. The guide explains that LLM APIs are stateless, so every token sent is paid for and re-read at every turn, and that agents burn through far more tokens than simple conversations due to their think-act-observe loops. It provides toolkits for both building agents, where developers control context, and using agents like Claude Code, where users steer context via commands.", "body_md": "# Don’t Take This Out of Context: Feeding Your LLM Exactly What It Needs\n\nIn 2022, prompt engineering was the skill everyone talked about. You tweaked a few words, added a “you are an expert…” or “think step by step”, and hoped for the best. Four years later, the game has changed. We’ve moved from simple chatbots to autonomous, multi-step agents surrounded by a whole ecosystem, RAG, MCP, memory, tools, skills, all feeding more and more tokens into our models. Feed too little, and the model hallucinates. Feed too much, and you get skyrocketing costs, higher latency, or a counterintuitive drop in quality that’s hard to debug.\n\nThis tension has a name now: **context engineering**, the “delicate art and science of filling the context window with just the right information for the next step.” And it applies to both situations that matter at Malt:\n\n- When you’re\n**building agents**, you have full control over the context: you decide exactly what goes in, and when. - When you’re\n**using agents**, like Claude Code or any coding assistant, each tool gives you commands and techniques to steer that context yourself.\n\nMy goal is to give you both toolkits. So buckle up, and let’s feed our LLMs exactly what they need. No more, no less.\n\n## It all starts with a stateless model\n\nLet’s start at the bottom. LLM APIs are stateless. The model remembers nothing between calls. Every single time you talk to it, you resend the entire conversation.\n\nTurn one: you send a message, the model replies. Turn two: to ask a follow-up, you resend your first message, the model’s first reply, and your new message. Turn three: all of that again, plus the new bit. It grows and grows.\n\nNote that some newer APIs, such as [Google’s Interactions API](https://ai.google.dev/gemini-api/docs/interactions-overview), can give the impression that the LLM is stateful, but it isn’t: the API is a stateful interface that calls a stateless model underneath.\n\nThis one fact drives everything else in this article. Keep it in mind: **whatever you put in the context gets paid for, and re-read, at every turn.**\n\n## But wait. What is “context”, exactly?\n\nWe already talked about it but here’s a precise definition: the context is **all the tokens available to the model when it generates a response**, everything you send in, plus the tokens it has already generated in its current answer.\n\nAnd it’s **bounded**. That ceiling is the context window, a hard technical limit of the model. The good news is that it has grown a lot. GPT-3.5 sat below 10,000 tokens; today’s flagship models cluster around one million (note the logarithmic y-axis on the chart below), with a few claiming far more.\n\nThat’s a lot of room, but is it enough to dump everything in? Not quite…\n\n## From conversations to agents\n\nOver the last couple of years, a shift drove a token growth. Everything I described so far, question, answer, question, answer, is a simple multi-turn conversation. But that’s not how we build any more. We build **agents**.\n\nAn agent doesn’t just answer once. Given a prompt, it starts to iterate, in a loop of three phases:\n\n**Think:** plan the next action, using whatever state or knowledge you’ve given it.**Act:** call a tool to take that action, fetch a document, check the weather, query an API, and get a result back. For this it needs the tool definitions.**Observe:** read the tool’s result, maybe update its state, and go around again.\n\nThink, act, observe. Think, act, observe. With today’s models, that loop can run for minutes, sometimes hours. And here’s the catch: every step feeds more back into the context, tool definitions, the calls the model makes, their responses, and any business data pulled in along the way. So even with a huge window, agents burn through far more tokens than a plain conversation ever did.\n\n## Opening the opaque box\n\nIf we open up the context of a modern agent, there’s a lot more in there than the user’s question:\n\n**System prompt and instructions**, the classic base layer.** Previous messages**, the conversation history. This also includes the thinking tokens (if the model has this capability).** Tools and MCP server specs**, plus every** tool call**and every** tool response**.** Structured output schemas**, if you ask the model for a specific JSON shape, that schema lives in the context too.** General and contextual knowledge**, a catch-all layer for everything the agent can pull in from outside the conversation.\n\nThis last layer deserves its own zoom-in, because it’s a grab-bag that shares one idea: information brought in from beyond the message history: Skills, parts of a codebase (if the agent can read your repo), a record fetched from a database (say, a freelancer profile you want to summarize), web-search results (close to a tool call), a knowledge base, intermediate artifacts the agent wrote to a file and reads back later, memory files, like the ones Claude keeps, and so on…\n\nThe point is simple: **all of this fills the context, and most of it isn’t the user’s actual question. Stacked up, it can strain even the biggest context windows.** But is size the only problem?\n\n## Why bother controlling it?\n\nBeyond size, there are other downsides when you don’t control the context:\n\n**Cost.** More tokens, bigger bill, and it*compounds*. Remember, we resend everything every turn. If you dropped the entire codebase into the context on turn one when two files would have done, you pay for that codebase again on every follow-up. (There are caching mechanisms that soften this, but the problem is still there; more on those later.)**Latency.** Bigger payloads mean a slower time-to-first-token: the model has to read everything before it can start generating.**Relevance.** The subtle and counterintuitive one:**a bigger context doesn’t mean a better context, or a better answer.**[Chroma’s 2025 “Context Rot” study](https://www.trychroma.com/research/context-rot)tested a batch of frontier models and found that every one degraded as the input grew, even on simple tasks, well before hitting the advertised limit. It echoes the older “lost in the middle” finding: models don’t process context uniformly. Drown your signal in noise, and quality drops.\n\n[In his excellent post “How Long Contexts Fail”](https://www.dbreunig.com/), Drew Breunig lays out a taxonomy of the ways a model goes bad when its context does:\n\n**Poisoning:** a hallucination or an error slips into the context. Because we resend the context every turn, that error sticks around, and the model keeps building on it, diverging further and further.**Distraction:** there’s so much in the context that the model over-focuses on it and stops drawing on what it learned in training. It stops generating novel plans and just parrots the context.**Confusion:** too much data, or too many tools that look alike, and the model’s judgment degrades. It doesn’t know which tool to call. This one is very well known to anyone who has added a ton of MCP servers to their agent.**Clash:** contradictory information within the context, a stale entry in a knowledge base next to a fresh one, for example (very challenging when you build a RAG system), and the model doesn’t know which version to trust.\n\n## Here comes context engineering\n\nAll of this leads neatly to how Andrej Karpathy (previously director of AI at Tesla, founding team at OpenAI) defined the field:\n\nContext engineering is the delicate art and science of filling the context window with just the right information for the next step.\n\nThree things in that quote are worth pausing on:\n\n- First, it’s the natural evolution of prompt engineering. Prompt engineering was for the conversational world. Now that we build agents, it’s no longer just about the prompt, it’s about the whole structure of the system: tools, memory, schemas, and how they fit together.\n- Second,\n**art and science**. Art, because it’s a young field, the techniques below aren’t silver bullets, and people are still figuring out what works. But science too, because you can measure it. Cost, latency, and relevance are all quantifiable, which means you can optimize them empirically. There’s no closed-form answer, but there is a rigorous way to approach the problem. - Third,\n**for the next step.** Because you resend everything each turn, every step is a fresh opportunity to reshape the context. You’re not stuck with the past; you can rewrite it.\n\nWith the vocabulary in place, let’s get practical. I group the techniques into three families, inspired by the [LangChain context engineering post](https://www.langchain.com/blog/context-engineering-for-agents): **Selecting**, **Editing**, and **Architecting**.\n\n## 1. Selecting\n\nThe best way to avoid something bad in your context is to never let it in. Selecting is about curating information before it enters. Two angles here: a retrieval phase, and progressive disclosure.\n\n**Retrieval**\n\nYou’ve all heard of RAG, retrieval-augmented generation. I want to be careful with that word, though, because RAG is one retrieval technique, and retrieval is much bigger than RAG. Two broad cases:\n\n**Direct retrieval**, when you already have a reference to what you need: a primary key, a file path. If I want to summarize a freelancer profile and I already have its ID, I don’t need a vector database or a graph, I just fetch the record. But even then, one rule: include only the relevant fields. For a summary I need the title, the description, the past experience, not the phone number or the address. That’s better for performance, cost, and security, you’re not shipping personal data to the model for no reason. If you’re in Claude Code, you can pull specific files in with*@*, which saves the agent a round of*grep*and*ls*. And sometimes a simple heuristic beats any lookup: the file the user currently has open, the closest file in the working tree, the most recently edited one, for a coding agent.**Search-based retrieval**, when you don’t know where the relevant content lives. RAG with a vector database is one option, but it’s not the only one, keyword search, graph traversal, and hybrids all have their place. (At Malt we lean heavily on this for freelancer/project matching, over 1m+ freelancers behind a retriever-ranker architecture backed by a vector database;[I’ve written about that separately](https://medium.com/nerds-malt/super-powering-our-freelancer-recommendation-system-using-a-vector-database-add643fcfd23).)\n\nNow, the part you may miss: retrieval isn’t only for memory or knowledge bases. **You can apply it to your tools and MCP servers too**.\n\n## Get Nicolas Mauti’s stories in your inbox\n\nJoin Medium for free to get updates from this writer.\n\nSay I’m working on a task, “set up OpenTelemetry to send traces to Langfuse”. If I connect every MCP server to my agent up front (Figma, Jira, Playwright, Context7, …), the agent carries the full description of each server in its context, on every turn. That’s a confusion risk and a token tax. Instead, I add a first search step: given the question, find the relevant server, here it’s Context7, which fetches library documentation, and hand the agent only that one.\n\nSame idea applies dynamically during a run. If at some step the agent can’t possibly call a tool yet (it’s missing a required input, say), don’t even offer that tool on that turn. **You can reshape the tool list at every step** (at the cost of invalidating your cache, later on this later).\n\n### Progressive disclosure\n\nProgressive disclosure is a different flavor of selecting: adding context *only as the model dives deeper* into a task. You start lean, and reveal more when the model asks for it.\n\n**Skills** are the cleanest example. Following the Agent Skills specification (that Claude Code implements), a skill is a file with a name, a short description, and a set of instructions for a task. The spec says: don’t load all skills and all their instructions into the context up front. Do it in three phases.\n\n**Discovery:** at startup, the agent loads only each skill’s*name and description*, just enough to know when it might be relevant.**Activation:** when a task matches a skill’s description, the agent reads the full SKILL.md into the context.**Execution:** the agent follows those instructions, loading any further referenced files or running bundled code only as needed.\n\nNothing heavy enters the context until it’s actually needed, and only for the skills that were activated.\n\n**Meta-tools** are the same trick applied to tools, and it’s a pattern I’m seeing more and more. Instead of exposing every tool up front, group them under “meta-tools” that reveal sub-tools on demand.\n\nA deliberately tiny example. I have four tools: read and update a freelancer profile, read and update a client project. I group them under a *FreelancerTool* and a *ClientProjectTool*. Now watch the flow when a user asks, *“Give me the daily rate for freelancer 1234.”*\n\n- The agent calls the model with only the two meta-tools available.\n- The model reasons that this is a freelancer question, and “calls”\n*FreelancerTool*. - That meta-tool doesn’t really do anything except reveal its children: the agent calls the model again, now exposing\n*ReadFreelancer*and*UpdateFreelancer*. - The model calls\n*ReadFreelancer*. This one is real, so the agent executes it, gets the data, and passes it back. - The model crafts the final answer, and the agent returns the daily rate.\n\nNotice what never happened: *ReadClientProject* and *UpdateClientProject* were never loaded. Their descriptions never touched the context, so they could never cause confusion or fill up the context. With four tools it’s probably overkill. With a hundred, grouped into a handful of categories, it could make the difference between a focused agent and a lost one.\n\nAnd some SDKs do this for you. On our JVM side at Malt, we build agents with [Embabel](https://hub.embabel.com/), and this [progressive tool](https://hub.embabel.com/reference/tools#progressive-tools) disclosure is handled natively: you define your meta-tool and its sub-tools, and the back-and-forth on the model side happens automatically.\n\n## 2. Editing\n\nSelecting is about what you let in. **Editing** is what you do once information is already inside, and it’s too late to un-see it. Maybe it was relevant early in the conversation but isn’t any more. Three moves here.\n\n**Summarization:** replace a chunk of message history with a digest. You can call an LLM with the history and a “please summarize this conversation” prompt, then swap the messages for the summary. There are also models tuned specifically for high-quality summarization, which matters, because a naive summary can quietly drop the one detail you needed. In Claude Code this is */compact*, and it kicks in automatically as you approach the window limit.\n\n**Pruning:** remove low-value messages outright. A crude heuristic like “drop the oldest” works but is maybe not the best possible approach; there are also models trained to spot low-value messages. Claude Code doesn’t prune surgically, its */clear* just wipes the whole context, which is a blunt version of the same idea.\n\n**Compression:** keep only the meaningful tokens from a message. This is what tools like RTK do, even if, rather than rewriting your history, they compress content at the source, before it ever enters the context. RTK installs aliases on your machine, so *rtk docker ps* returns a far more compact version of what *docker ps* would print, same information, a fraction of the tokens.\n\nThese techniques are great, but not totally free. Three caveats:\n\n**Loss of information:** cut too aggressively and you lose something the model needed. Summarization especially: if the summarizer misjudges what mattered, that detail is gone.**Prediction drift.** This one is more subtle, and it’s the old ML engineer in me talking. Models are trained on certain distributions of text. Compression, RTK-style especially, produces output the model never saw during training (it has seen plenty of normal docker ps output, but very little of the RTK format). Over a long run, that mismatch between training and inference can hurt performance. I haven’t benchmarked this, and I haven’t found solid published papers, so it’s more of an intuition; if you have references, I’d genuinely love to hear about it.**Caching.** LLM providers cache to save you money and time, but caching only works on an**exact prefix match**. The cached portion has to be byte-identical.\n\nHere’s why that matters for editing. If I just append a new user message, the whole prefix is unchanged, cache hit, cheap and fast (on the order of a 90% saving on the cached tokens). But if I summarize, I’ve rewritten the middle of the context. Now only the instructions and tools are still cached; the summary and everything after it are fresh tokens the model has to reprocess, and re-cache. So there’s a real trade-off: summarizing shrinks future turns, but it costs you a cache write today. Sometimes it’s worth it, sometimes it isn’t.\n\nA nice illustration of this trade-off is [an internal behavior of Claude Code that Anthropic revealed in a recent article](https://www.anthropic.com/engineering/april-23-postmortem): in an active session, it keeps Claude’s reasoning, the *thinking tokens*, in context, so Claude can still see why it made its past responses and tool calls. They’re cheap, they’re already cached, so there’s no point evicting them and busting the cache. But leave the session idle for a couple of hours, come back, and it strips that old reasoning, keeping the conversation but not the thinking behind it. Why? Because the cache has a TTL (commonly a few minutes to an hour), and once it expires you’d have to resend everything anyway, at which point those verbose thinking tokens no longer earn their place. That’s a deliberate design decision, and exactly the kind of call you’ll make yourself when building agents.\n\n## 3. Architecting\n\nThe third family is structural: instead of massaging one context, you design the system so that contexts stay isolated by construction. Two approaches: branching and subagents.\n\n**Branching** means splitting a workflow into specialized paths that share a common ancestor. Say I’ve used my coding agent to build a feature; the generated code and the feature details are now in the context. Next I want to do two independent things: write the tests, and draft a Slack message announcing the feature 🎉🚀. Both need the “feature built” history, but they don’t need each other, the test task doesn’t care about the Slack message, and vice versa. So I branch: one branch writes tests, the other drafts the message, each forking from the shared history with its own clean context. In Claude Code that’s */branch*.\n\nThere’s a close friend, */rewind*, which jumps back to an earlier state instead of forking. I can write the tests, then rewind to just after the code was generated and draft the Slack message from there. The difference: rewind drops the messages you rewound past, so once you’ve rewound to do the Slack message, you can’t go back and follow up on the tests, that branch is gone. If you want to keep both alive, prefer */branch*.\n\n**Subagents** go all the way to full isolation. Each agent has its own instructions, skills, tools, MCP servers, and so on. A main agent delegates a task to a subagent by sending it a request; the subagent works in its own isolated context and returns a single response, which the main agent appends to its context and continues.\n\nHere I want to raise a flag, because subagents are somewhat “trendy” and it’s tempting to reach for them immediately. I’m not saying they aren’t useful, they are. But they don’t make the context problem disappear; they multiply it. Think about what you actually pass from the main agent to the subagent. Just a single instruction (“do this task”)? A summary of the conversation so far, in case something earlier matters? The entire conversation? **That decision is itself a context-engineering problem**, and now you have three contexts to reason about instead of one: the main agent’s, the subagent’s, and what flows between them.\n\nSo my advice: start with a **single** agent, and only split into subagents once your metrics say you need to. Otherwise you’ve just signed up to do context engineering in three places at once.\n\n## A final note on guardrails and security\n\nOne question that could comes up: *where do guardrails and input sanitization fit?* Mostly **in the selecting phase**. When you send any user-derived data into the model, sanitizing what you include is part of building a clean context.\n\nBut there’s an important difference between LLM guardrails and something like SQL-injection defense. With SQL injection, sending the malicious input is already the problem. With a plain text-in/text-out LLM, it’s usually fine to send a potentially “poisoned” input to the model (this stops being true if the model API isn’t fully stateless and an action could be triggered server-side automatically), a prompt injection isn’t dangerous in itself, because you can inspect the model’s response and apply your guardrails after the call, not only before it. So you have two lines of defense: sanitize what goes in, and check what comes out. Keep both in mind.\n\n## Where should you start?\n\nIf this feels like a lot of techniques, don’t try to apply all of them at once. Here’s the pragmatic path.\n\n**Start small and simple.** First validate that your product does the thing at all. Don’t over-engineer the context before you’ve proven the use case.**Evaluate that first version with the right metrics.** There’s no magic metric here; it depends entirely on your use case. For a real-time feature (recommendations as a user browses), I’ll trade a little quality for latency. For an offline batch job, latency is irrelevant and I optimize purely for quality. It’s always a balance between cost, quality, and latency.**Iterate, and measure the impact of each change against those metrics.**\n\nYes, that’s a **feedback loop**, the same one we’ve relied on in machine learning for years. It’s just as relevant when building agents, if not more, because there are so many more moving parts to get wrong.\n\nAnd a feedback loop only works if you can actually see what’s happening. This is where observability comes in: at Malt we lean on Langfuse, which builds on OpenTelemetry, so if your SDK is OTel-compatible (most are), you get traces of everything, tool calls, subagent calls, exactly what was sent and what came back, and you can layer trace-based evaluations on top. (I’ve written separately about [instrumenting GenAI apps with Spring AI, OpenTelemetry, and Datadog](https://medium.com/nerds-malt/how-to-monitor-genai-applications-with-spring-ai-opentelemetry-and-datadog-24dbb2085c06) if you want the engineering details.)\n\n## Wrapping up\n\nContext engineering isn’t a single trick; it’s a discipline. LLMs are stateless and their windows, however large, degrade when you overfill them, so every token you add is a token competing for the model’s attention, and for your budget. The good news is that you have a real toolkit: select what goes in, edit what’s already there, and architect your system so contexts stay clean by design. And whichever techniques you reach for, close the loop, measure, iterate, repeat.\n\nAre you building internal agents, or steering ones like Claude Code day to day? I’d love to hear your techniques, which ones have (and haven’t) worked for you, and how you’re measuring the impact. Drop a comment.\n\n*This article is the written, detailed version of a talk I gave recently, you can find it there*", "url": "https://wpnews.pro/news/context-engineering-for-agents-a-practical-guide", "canonical_source": "https://blog.malt.engineering/dont-take-this-out-of-context-feeding-your-llm-exactly-what-it-needs-0db8a86d2151", "published_at": "2026-08-03 20:12:22+00:00", "updated_at": "2026-08-03 20:22:37.427766+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-tools"], "entities": ["Malt", "Claude Code", "Google", "GPT-3.5"], "alternates": {"html": "https://wpnews.pro/news/context-engineering-for-agents-a-practical-guide", "markdown": "https://wpnews.pro/news/context-engineering-for-agents-a-practical-guide.md", "text": "https://wpnews.pro/news/context-engineering-for-agents-a-practical-guide.txt", "jsonld": "https://wpnews.pro/news/context-engineering-for-agents-a-practical-guide.jsonld"}}