A functional taxonomy for LLM inference in agentic tasks

A new functional taxonomy classifies each large language model inference call in an agentic task into one of four roles — Initialization, Orchestration, Reasoning, and Synthesis — to separate inference that resolves task-relevant uncertainty from inference that merely copies data between requests. The taxonomy's author defines Reasoning functionally, as any call used to resolve uncertainty that matters to task completion, without taking a position on whether a neural network "really reasons." The framework matters because aggregate token bills — calls, input tokens, output tokens, latency, and reasoning tokens — cannot distinguish valuable inference from waste.

Measuring the cost of an agentic cost is straightforward. Distinguishing value from waste is not. Our token bills collapse model inference into a few aggregate numbers: calls, input tokens, output tokens, latency, perhaps reasoning tokens if we're lucky. Those numbers are useful, but they say very little about what the model was being asked to do. Two inference calls with similar token counts can play completely different roles in task completion. One might be deciding which source of information is relevant to a task; the other might simply be copying a cursor from one response into the next request. Both consume model inference, but only one is resolving meaningful uncertainty. I think there is a useful way to describe this distinction. Given some task T and the sequence of model invocations used to complete it, each invocation can be classified according to the functional role that inference plays in T. I use four categories: Initialization, Orchestration, Reasoning, and Synthesis. This is deliberately a taxonomy of inference calls<sup 1</sup rather than a theory of what happens inside the model. In particular, I use “reasoning” in a functional sense: an inference call is Reasoning when it is being used to resolve uncertainty that matters to completion of the task. Nothing about the taxonomy requires a position on whether a neural network “really reasons.” Initialization initialization This is the easiest category to distinguish. Before an agent begins making task-specific decisions, the model usually has to ingest some amount of task-independent context: system prompts, tool descriptions, etc. Basically, any context that defines the environment in which the task will be performed. A software development agent might be told that it can call read file , search code , edit file , and run command ; a more general agent might receive hundreds of function definitions and substantial policy text. This work is primarily an input or prefill cost. The model may produce almost nothing in response, but it still has to process the environment that has been placed into context. As agent environments become larger, Initialization can become significant even before any task-specific inference takes place. There is an important boundary here between the model and the agent harness. If the harness parses JSON, increments a retry counter, loads a schema from disk, executes a subprocess, or dispatches an HTTP request without invoking the model, none of that belongs in this taxonomy because none of it is inference. If the harness then places a 20,000-token schema or tool catalog into the model context, however, consuming that context does contribute to inference cost. The distinction is not between “agent work” and “non-agent work;” it is specifically about the role played by model inference within the broader agent runtime. Examples examples Reasoning reasoning The most interesting distinction begins once the task is underway. Suppose a user asks an agent why checkout latency increased after yesterday’s deployment. The agent has access to deployment history, logs, metrics, incident records, and runbooks. If the model decides that the first useful step is to inspect recent deployments for the checkout service, that inference is doing more than formatting a tool call. It is deciding what evidence is relevant, which operation should retrieve it, and what parts of the system are relevant. That is Reasoning. The same is true when a coding agent decides which implementation to inspect after reading a stack trace, when a research agent decides that conflicting secondary sources require consultation of the original regulation, or when a browser agent discovers that the expected navigation path is missing and has to determine where the relevant information lives instead. In each case, the model is being used because something task-relevant remains unresolved. Reasoning also includes judgments about whether execution should continue. An agent that has gathered several sources may need to determine whether they are sufficient to support an answer, whether an apparent contradiction matters, or whether another search is warranted. A model invocation that concludes “we have enough evidence” can therefore be Reasoning even if the resulting output is extremely short. Ask whether the invocation had to resolve a meaningful question about how the task should proceed; if the answer is "yes," then this inference call is filed under Reasoning. Examples examples Orchestration orchestration Now consider a different situation. The agent has already determined that it needs every page of a result set. The first response contains a cursor: { "next cursor": "cD0y" } The next request is therefore already semantically determined: GET /records?cursor=cD0y If the model is invoked simply to produce that request, no meaningful task uncertainty has been resolved. The decision to continue pagination has already been made; the cursor itself has already been supplied by the preceding response. The model is advancing an execution path whose semantics are known. I deem this Orchestration. This shows up in a variety of forms, but they have the same underlying structure. A previous call may return customer id: "cus 82ca91" after the agent has already decided that the next step is to fetch that customer’s invoices; another inference call then emits list invoices customer id="cus 82ca91" . An agent could decide that the same metric must be collected for every service in a known list and then invoke the model repeatedly to substitute checkout , payments , inventory , and shipping into otherwise identical requests. After that let's say the jobs API returns 202 Accepted and a status URL, with the agent subsequently invoking the model each time it needs to poll that URL. Our retry policy specifies that a 503 should be retried up to three times, yet every 503 is returned to the model so that it can decide to retry the exact same operation. None of this is unnecessary work. The essential question is why those operations require us to dispatch those calls to a model once the decisions governing them have already been made. Ordinary software is very good at loops, parameter substitution, state transitions, retries, and protocol progression. Here's an example that exposes a key boundary in the taxonomy: command construction. Imagine that two model calls both produce: grep -R "refreshToken" packages/auth In one trace, the preceding state already says “search packages/auth for refreshToken .” The model is simply rendering an established operation in shell syntax. That is Orchestration. In another trace, the model has been shown a bug report explaining that authentication works until the access token expires, considers several plausible causes, decides that refresh-token handling is the most promising place to investigate, and emits the same grep command. That is Reasoning. The output is identical, but the function of the inference call is different. This is why predetermination is the key criterion, not syntax. You cannot reliably classify an inference call by looking only at what it emits. Any tool invocation or RPC may be the result of a task-relevant decision that had to be made inside the call. Conversely, Orchestration does not need to produce machine-readable syntax at all. A model that emits “Now inspect the next page,” “Proceed to the second repository,” or “Retry the previous operation” may still be spending inference on execution that was already determined. Query construction has exactly the same ambiguity. Suppose an agent is asked to find enterprise customers who renewed last quarter but reduced their seat count. If the model has to inspect a schema, determine which entities and fields represent customers, renewals, and seat history, and decide how those pieces need to be combined, the inference that produces the eventual query is Reasoning. If an earlier call has already established the relevant entity, filters, comparison, and requested fields, and a later call merely turns that representation into the syntax required by the environment, the latter call is Orchestration. “The model generated a query” tells us almost nothing by itself. I would classify whole inference calls rather than trying to divide the work inside a single invocation. If a model both decides that refresh-token handling should be inspected and emits the corresponding shell command, there is little practical value in trying to determine which output tokens belong to reasoning and which belong to rendering. The useful question is what was known when the invocation began. If the call had to resolve what operation should be performed, it is Reasoning. If the relevant operation was already determined and the model was used to carry it forward into execution, it is Orchestration. Examples examples Synthesis synthesis This is the terminal inference that turns the accumulated task state into the response delivered to the user. Imagine that an incident investigation has established that checkout-api 4.18.2 was deployed at 14:03, p95 latency increased from 220ms to 810ms four minutes later, logs show repeated inventory RPC deadline failures, the release changed that RPC timeout from 1500ms to 300ms, and a rollback restored latency to baseline. The inference that converts those facts into a concise explanation of the incident is Synthesis. Synthesis is not necessarily simple. In some tasks it may require more judgment than any individual step that preceded it. It gets its own category because its functional role is different. Reasoning inference determines how the task should proceed; Synthesis produces the terminal response from the state produced by that process. Examples examples Discussion discussion Taken together, the four categories give us a different way to read an agent trace<sup 2</sup . Consider an agent asked to find the three most recent failed CI runs for a repository, determine whether they share a cause, and recommend a fix. The repository instructions, tool schemas, and coding policies consumed before task execution are Initialization. The call that decides to retrieve the latest failed runs is Reasoning. If the resulting API response contains three run IDs and the agent has already decided that all three logs must be inspected, model calls whose only job is to propagate those IDs into three get run logs operations are Orchestration. The call that compares those logs, notices that they all fail during dependency installation, and decides to inspect recent lockfile changes is Reasoning. If that same call produces git log -10 -- pnpm-lock.yaml package.json , it remains Reasoning because selection of that operation had not been predetermined when the call began. Once the relevant commit has been identified, a separate model call whose only job is to render git show 72cc1ab -- package.json pnpm-lock.yaml may be Orchestration. The eventual explanation of the common failure and proposed fix is Synthesis. A new metric: Reasoning Yield a-new-metric-reasoning-yield The economic distinction between Reasoning and Orchestration is the part I find most useful. Model inference is expensive machinery. It adds latency and token cost, and because model outputs are probabilistic it can also introduce another opportunity for execution to drift from an otherwise valid plan. That cost is often justified when the task calls for judgment, if you will. It is harder to justify when the next operation is already represented in the state of the system and another model call exists principally to translate that state into the next mechanical step. A mature agent architecture should therefore be evaluated in part by how effectively it reserves model inference for work that benefits from model judgment. Your goal isn't necessarily to minimize orchestration; every nontrivial task requires orchestration. The optimization target is the amount of inference spent performing orchestration after the relevant choices have already been made. An architecture in which the model chooses a plan and ordinary software executes the deterministic portions of that plan is different, in an economically meaningful way, from one that returns to the model after every state transition simply because the agent loop is implemented as model - tool - model - tool . One possible way to capture part of this distinction is a metric I call Reasoning Yield: Reasoning Yield measures the share of task-execution inference spent resolving task-relevant uncertainty rather than mechanically advancing already-determined execution. If an agent spends 8,000 tokens on Reasoning and 2,000 on Orchestration, its Reasoning Yield is 80%. If another agent reaches the same result using the same 8,000 Reasoning tokens but another 12,000 tokens of Orchestration, its yield is 40%. All else equal, the former is making more efficient use of model inference. Reasoning Yield is not a complete quality or cost metric, and a higher number does not automatically imply a better system. An agent could achieve a very high yield while reasoning inefficiently, making bad decisions, taking longer, or failing the task altogether. Nor should Initialization and Synthesis simply be ignored because they fall outside the ratio. Initialization can be a major prefill expense, particularly in environments that expose large schemas or tool catalogs, while Synthesis may itself be computationally substantial. I would want all four categories represented in any serious trace analysis. Reasoning Yield isolates one narrower question: within the inference being spent on task execution, how much is being used to resolve uncertainty versus advance execution that has already been determined? There will inevitably be edge cases because predetermination is not always binary. A loop that appears mechanical may encounter one item that requires interpretation. A retry that normally follows a fixed policy may produce a new authentication error that requires a decision about whether to refresh credentials, change identity, or stop. A browser agent may mechanically click “Next” for five pages and then reach a page whose layout has changed. At that point the character of the next inference changes because task-relevant uncertainty has reappeared. That is not a weakness of the taxonomy so much as the property it is trying to expose: model judgment becomes useful again when the runtime can no longer determine the next meaningful operation on its own. The framework also says nothing about whether an individual reasoning call is good reasoning. Reasoning inference can reach the wrong conclusion, and Orchestration inference can be a perfectly reasonable engineering tradeoff when it keeps an implementation simple. There is no claim here that every orchestrational use of a model is wasteful or that an ideal agent would contain none of it. The narrower claim is that these uses of inference are functionally different, and treating them as one undifferentiated bucket makes it harder to understand why one agent architecture consumes more inference than another. That distinction becomes especially visible in benchmark traces. A benchmark might tell us that an agent completed a task successfully using 84,291 tokens across 37 model calls. What it does not tell us is why 37 calls were required. Perhaps most of them involved evaluating evidence, choosing between plausible approaches, recovering from surprises, and deciding when the investigation was complete. Or perhaps the agent made only a handful of meaningful decisions and spent the remaining calls moving identifiers between tools, formatting commands, stepping through known collections, following pagination cursors, and polling operations whose continuation conditions were already defined. Those are very different architectures despite producing the same success flag. The four categories give us a vocabulary for describing that difference. Initialization accounts for the inference required to place the model inside the operating environment. Reasoning covers inference used because some task-relevant question still has to be resolved. Orchestration covers inference used after that question has already been answered but execution still needs to move forward. Synthesis covers the final transformation of the completed task state into the response delivered to the user. Of those boundaries, the one between Reasoning and Orchestration is the important one: was the semantic operation already determined before the inference call began? If not, the model is being used to resolve task-relevant uncertainty. If so, the model is participating in execution of a decision that already exists. Agents will continue to need both reasoning and orchestration. This framework may help practitioners evaluate and tune their agentic workloads accordingly.