Welcome back to the Harness Engineering series — a 10-part journey from raw language model to production-ready agentic system. Made by builders. For builders.
In Part 4, we looked at the Tools — the set of functions the model can call. But there's still one big open question hanging over every turn of the Loop: what does the model actually see when the Loop calls it?
The answer is: whatever the harness put in the payload. That payload — the entire package of instructions, history, retrieved documents, tool definitions, and everything else — is called the Context.
Some readers know this territory under an older name: prompt engineering. That name isn't wrong, but it's narrow. A prompt sounds like something you write once and ship. The reality of running an agent is that the payload changes every turn, and designing what goes in it is an ongoing discipline. Hence the newer, more accurate term: context engineering.
What's ahead:
By the end of this article, you'll know what the Context actually is, why every turn forces you to answer "what should the model know right now?" from scratch, and the three moving pieces (system prompt, history, retrieval) that make up a well-designed context.
Let's get started.
📚 Want to go deeper than the articles? #
While you follow along with this series, I've put together two hands-on resources that go further than any single article can: — A self-paced course where I walk you through building a production-grade agentic harness from the ground up, in code.[Build a Harness from Scratch — Udemy Course]— A live, cohort-based workshop for builders who want direct feedback, Q&A, and to work through the material with peers.[Harness Engineering for AI Agents — Live Maven Workshop]Both are optional — the series stands on its own. But if you want the full studio-quality version, that's where it lives.
The Context is everything fed into the model on a given API call. All of it.
Concretely, on any single turn, the payload sent to the model typically includes:
That entire package goes over the wire. The model reads all of it. Then it produces one response. Then, on the next iteration of the Loop, the harness assembles a new package — probably with the previous response added, maybe with new tool results appended, maybe with different retrieved content — and sends that.
Every turn is a new context. Every turn.
Because the model is stateless. We noted this back in Part 1: every API call is independent on the model's side. Nothing persists between calls unless something outside the model puts it there.
Which means the question "what should the model know right now?" turns into a very different question:
"What do we put in the context?"
And that question has to be answered on every single turn.
There is no shortcut. You can't tell the model "remember what I said five minutes ago." You can't say "reference the file we discussed earlier." Every relevant thing — every fact, every prior message, every result, every document — has to physically be in the payload for the current call, or the model doesn't know about it.
This is why context engineering is arguably the deepest engineering discipline in the whole harness. Deciding what to include, what to compress, what to leave out, and when — that's the work. And it's ongoing. The context is a moving target, different every turn. There is no version to "ship." There's only the version being assembled right now, for this call.
There are three moving pieces in any context, and each is worth being deliberate about: the system prompt, the history, and any retrieved knowledge.
The system prompt is the top layer of the context — the part that gets prepended to every turn and stays roughly the same throughout the conversation. It tells the model who it is, what tools it has, and what good behavior looks like.
Think of the system prompt as the control plane of the agent. It's set by the agent's builder — the person designing the harness — and it defines the model's role, constraints, and preferred style.
There's a related but distinct pattern worth noting: files like AGENTS.md
(or .cursorrules
, or CLAUDE.md
) that let the user — not the agent builder — inject their own instructions into the context. These are user-controlled additions layered on top of the built-in system prompt. Same mechanism, different owner. When you're designing a harness, you have to decide whether to support this and how the two layers interact.
The conversation history — every message the user and the model have exchanged — is the second big chunk of the context. And here's where things get interesting: you can't just send all of it forever.
Context windows are finite. Every token you spend re-sending old messages is a token you can't spend on the current turn. So a history strategy is a decision about how much of the past to include, and in what shape.
The common strategies:
The right strategy depends on the task's time horizon — what the model needs to remember for the sub-step it's currently on, versus what can safely be forgotten. A quick lookup doesn't need much history. A long-running refactor needs a strategy that keeps the model coherent across hundreds of turns.
The third piece is retrieval — pulling in relevant outside knowledge and injecting it into the context. Documentation, code snippets, database records, prior conversations, whatever the task requires.
The design principle here is simple to state and hard to do: bring in just enough to be useful, without burying the model.
Too little retrieval, and the model is missing information it needs. Too much retrieval, and the important stuff is drowning in noise. The model has to find the relevant part of what you injected, which uses attention budget it could have spent actually answering the question.
Good retrieval systems rank aggressively and are willing to send less than the context window can hold. If your retrieval is filling half the context on every call, it probably isn't retrieval — it's a hedge.
Cursor's context engineering is most of why it feels different from using raw ChatGPT — even when the underlying model is the same.
Look at what Cursor is quietly assembling into the context on every keystroke that triggers a completion:
That's five distinct context sources, all being packaged into a single API call, every time you trigger a completion. And the API call itself is going to a foundation model that anyone with an API key can call.
The model is the same model anyone can call. The context is the product.
That's the punchline of this whole article. When you compare two agents built on the same foundation model, the differences you feel between them are almost always context-engineering differences. What each one puts in the context, how much of it, in what order, at what moments — that's the design surface where agents differentiate themselves.
The Context is what the model gets to see on any given call. The Loop drives the cycle. The Tools define what the model can request. The Context defines what state the model has when it decides what to request. Three components, tightly coupled, each depending on the others.
But context engineering has one dependency the other components don't: it depends on something being available to inject. The system prompt is easy — you wrote it. Recent history is easy — the harness holds it. But retrieval? Memory? Files that came from somewhere? Those all need a place to live. That "somewhere" is what the next few articles are about: the Environment (Part 6) that files and tool outputs live in, and the Memory Layer (Part 7) that lets the harness carry state forward across turns and sessions.
Remember that this article is part of a longer 10-part series that walks you through every component of an agentic harness.
Here's the roadmap:
See you in the next one.
Happy coding :)