For a while I treated prompt engineering, context engineering, harness engineering, and the newer talk about loops and graphs as separate disciplines. Each arrived with its own vocabulary, and for a few months each felt like the thing that finally mattered. The distinction started to break down for me once I stopped asking what each discipline was called and started asking what each one actually controls in a running system. Underneath the different names, they all change the same thing. They move the boundary between what the model decides and what deterministic software decides.
The harder engineering problem is deciding where that boundary belongs. Prompt, context, tools, graphs, harnesses, and guards are different ways of placing it. The prompt controls how the model reasons. The context controls what evidence it reasons over. Tools control what actions exist. The harness controls the execution environment around each call. Guards and evaluators control whether a model decision is allowed to become a real action. The graph controls the topology of decision making, which transitions remain deterministic, and where decisions are allowed to occur. None of these is a generation that replaced the one before it. In a real agent you use all of them at once and none of them retires. They are different levers, and each one moves the model versus software boundary at a different point in the system.
When people argue about whether context engineering or harness engineering matters more, they are often arguing about which lever to reach for. The useful question is which decisions in this system should belong to the model at all. I have come to think of that question as discretion engineering, and I think it is the center the other terms have been circling.
The obvious way to picture the boundary is a single slider, model on one end and deterministic code on the other, and you decide how far to push it. That picture is what led me, for a while, to assume that pushing everything toward code was the safe default. It is not, and the reason is that “model or code” is the wrong question to ask first. Two things actually decide where a given decision belongs, and they are independent of each other.
The first is ambiguity. How much interpretation does this decision require. Reformulating a vague search query is highly ambiguous. Checking whether a previous write committed is not ambiguous at all.
The second is consequence. What happens if this decision is wrong. Picking a slightly worse retrieval path is cheap to be wrong about. Authorizing a payment that should not have happened is not. I use consequence broadly here. It includes blast radius, reversibility, external side effects, and the cost of correcting a bad decision after it has already executed.
Once you separate those two axes, the design rule falls out of them rather than out of taste:
The point is that you are no longer asking whether the model or the code should handle a decision. You are asking how ambiguous the decision is and what it costs to get it wrong, and the quadrant tells you the shape of the answer.
Placing decisions on that map only helps if you have enumerated the decisions in the first place, and that is an easy step to skip when an agent design starts with prompts, tools, or graph structure. This is not an industry taxonomy and I would not present it as one, but in the systems I have worked on I usually find roughly eight decision points where the boundary gets placed, well or badly: interpretation, planning, tool selection, authorization, execution, recovery, verification, and termination. The exact list matters less than the habit of writing the decisions down before choosing any technique.
Three of them sit in very different places on the map, and walking those three is enough to show how the framework behaves.
Interpretation and planning are the clearest case for model discretion, though not for identical reasons. Interpretation, understanding what an upstream signal is really asking for, is high ambiguity and usually low consequence on its own, which is exactly the judgment you reached for a model to get. Trying to make it deterministic usually means rebuilding the brittle intent classifier you adopted an LLM to retire. Planning is also highly ambiguous, but its consequence is not fixed. A plan inherits its consequence from what its steps can actually cause. A research plan that only reads and summarizes can tolerate broad model discretion. A plan that can move money or change production infrastructure cannot, because the ambiguity is now attached to actions that are expensive to get wrong. So planning does not sit in one quadrant. Where it lands depends on the blast radius of the actions the plan is allowed to trigger, which is the framework doing its job rather than a category being filed once.
Authorization is the opposite corner, and it is the one I feel most strongly about. In the payment and money movement work I have done, with very high reliability expectations, authorization is low ambiguity and high consequence, which puts it firmly in deterministic territory. Whether an action is permitted, on whose behalf, and within what limits is not a judgment call that benefits from the model’s flexibility. It is a place where flexibility is the risk. The model can propose an action, but whether that action becomes real should be a deterministic decision made by a guard the model cannot talk its way past. I have seen the argument that a well prompted model can respect limits, and I do not trust it, because the failure mode is not a wrong answer a reviewer catches later. It is an unauthorized action that already happened. Execution belongs in the same deterministic corner for a related reason: idempotency, duplicate suppression, and retries live in the harness, and one of the more instructive failures I have written about before is the duplicate write an agent cannot see, where a retry at the model level quietly becomes two real actions downstream. A better prompt does not solve that class of failure. The fix belongs in execution state, idempotency, and retry handling.
Verification and termination are the dangerous middle, high enough in consequence that letting the model self grade and self stop is where I see the most quiet failures. Agents loop too long, stop too early, or declare success on output a simple deterministic check would have rejected. I would rarely leave either of these entirely to the model, and I would rarely fully automate them either, because both genuinely need some judgment about whether a result is good enough in context. That is the high ambiguity, high consequence quadrant in practice: model proposes, policy bounds, independent evaluator verifies.
Recovery is the decision that changed my thinking the most, and it is the clearest example that these decision points are not atomic. Any one of them can contain its own discretion boundary, and recovery is where that is most obvious.
The intuitive conclusion from the authorization discussion is that determinism is safety, so a maximally harness heavy design should be the most reliable. In my experience that holds right up until something fails in a way the rules did not anticipate. Recovery is the decision that needs to understand why something failed, and why is exactly the kind of open ended judgment that fixed policies are bad at. A bounded retry with fixed backoff handles the failures you predicted and does nothing useful for the failure you did not. So the instinct to strip judgment out of recovery is really the instinct to remove the model from the one place its judgment was most needed.
The mistake in my earlier thinking was treating recovery as a single decision to assign. It is not. It decomposes:
Failure detected -> Classify the failure (timeout, validation error, stale data, or semantic mismatch?) -> Propose a recovery strategy (reformulate the query? try a different retrieval path? change the requested item?) -> Check the strategy is permitted (can I retry this payment at all?) -> Confirm current state (was the previous write actually committed?) -> Execute the recovery (bounded, in the harness) -> Verify the resulting state
Once it is broken out like this, the assignment is not model or code, it is per step, and each step maps cleanly onto the two axes. Classifying a novel failure is high ambiguity, so I am comfortable giving the model that call, supported by signals. I am comfortable letting the model classify the failure only if that classification cannot trigger a consequential action by itself. If the model labels something a timeout and the system immediately retries a payment on that label, the classification has quietly become part of a consequential action. So the classification is an input to policy, not the policy. The model can say what it thinks failed, and deterministic policy still decides whether the recovery that label implies is permitted. Proposing a recovery strategy works the same way, model owned as a proposal. Whether a payment can be retried is low ambiguity and high consequence, so it is a deterministic policy, not a model opinion. Whether the previous write committed is a deterministic state check, not something to ask the model to remember. Retry limits come from deterministic policy rather than model judgment. Changing the requested item is a model proposal that still has to pass policy or approval. Whether to escalate is genuinely hybrid.
So the honest version of my recovery position is not that recovery stays with the model. It is that recovery is where I preserve the most model discretion, and even there the model proposes while deterministic controls still govern what is permitted and how it executes. Even a single decision point contains the same ambiguity and consequence tradeoff at a finer grain.
My experience suggests reliability is not monotonic with determinism. Removing model discretion helps around authorization and execution, but can make recovery brittle, because recovery is the one place the model’s judgment was doing real work. I have not tested that as a controlled benchmark, so I treat it as an operating hypothesis rather than a measured result. A useful experiment would hold the task fixed and move the discretion boundary across versions, measuring wrong actions, failed recoveries, and unnecessary retries.
This reframing changed how I start a design. Instead of opening with what the prompt should say or whether I need a multi agent graph, I now start by writing down the decisions the system has to make, and for each one I ask the two questions: how ambiguous is it, and what happens if it is wrong. That places each decision in a quadrant, which tells me the shape of the answer before I have written a line of prompt. Authorization comes out deterministic. Execution comes out deterministic with the model proposing. Interpretation stays mostly with the model. Planning gets as much discretion as its downstream blast radius permits. Verification and termination come out as constrained discretion with independent checks. Recovery gets decomposed and assigned step by step.
Only after that do the familiar techniques come back in, and now they have clear jobs. Prompt and context are how I shape the decisions I decided the model should own. Tools and graph structure are how I constrain the decisions that are shared. Harness and guards are how I enforce the ones I decided the model should not own at all. The techniques stop competing for the title of most important discipline and become implementation of a boundary I have already placed deliberately.
The vocabulary will keep changing, and there will be another engineering after this one, named for whatever decision the field notices it has been handling badly. Underneath the new name the work is the same. Before I choose a prompt strategy, a graph, or a harness, I now write down the decisions the system has to make, and for each one I ask how much judgment it requires and what happens if it is wrong. Once those two answers are clear, where the model should have discretion becomes much easier to reason about.
Discretion Engineering: Deciding What Your AI Agent Is Allowed to Decide was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.