Spring AI Token Usage: Measure Cost Before You Pick a Model — LLM Cost Control 1/4

Cutting LLM costs in Spring AI starts with two choices: which model answers a request, and what defaults your ChatClient adds to every one it sends.

Neither is worth changing until you can see where the tokens go. That is why this article starts with measurement.

This is Part 1 of four, and it covers the first three of ten cost drivers. Driver #0 tells you where the money actually goes; #1 and #2 are the two decisions that shape every request your application sends. The remaining seven attach to what you build here.

A note on the numbers: where a price ratio matters for the argument (input vs. output, cache read vs. write), this series quotes real July 2026 list prices with a link. All other examples use a flat rate of $1 per million input tokens, so you can redo the calculation with your own provider's price sheet. You should do that, because these prices change every few months.

Provider invoices and usage dashboards usually show your spending by model and by token type — input, output, and cached. That is useful, but it is not enough. The numbers cannot tell you which feature, client, or advisor inside your application was responsible for that usage.

Spring AI integrates with Spring Boot's Micrometer-based observability to fill this gap. Its core AI components automatically emit that data. ChatModel,

EmbeddingModel

ImageModel

ChatClient

VectorStore

Each metric includes built-in tags, such as the model name and token type. These tags separate models and providers, but not callers: every request to the same model carries the same tag values, so they cannot tell two features apart on their own. Spring AI marks tags as low- or high-cardinality: low-cardinality tags go on both metrics and traces, high-cardinality tags go on traces only. To attribute usage to a specific feature or client, add your own low-cardinality tags (https://javadoc.io/doc/io.micrometer/micrometer-observation/latest/io/micrometer/observation/ObservationConvention.html). Keep those values few and stable, not one per user or request. For more on how tags work, and how to keep their number under control, see Micrometer's documentation on tags and naming.

With this data in place, you can answer the questions the later drivers raise. Is history growth (Part 2) really increasing costs? Watch prompt token usage over time. Did the tool-search advisor (Part 3) reduce the amount of context sent to the model? Compare prompt token usage before and after enabling it. Is a reasoning model becoming more expensive than expected (Part 2)? Output token trends can reveal unexpectedly long responses and rising generation costs.

Two more steps make this setup safer, not just more informative. First, create an alert for sudden increases in token usage or request volume. A bug that puts an LLM call inside a loop can burn through your budget fast, and an alert catches it within minutes instead of at the end of the month. Second, configure provider-side spending controls where available, such as budgets, quotas, or usage limits. This is a final safeguard that no application framework can enforce.

Treat observability as step zero. Set it up before optimising anything else, so every later change has a measurable before-and-after comparison.

The fastest way to overspend on an LLM application is to send every request to the most capable model you can find. Flagship and budget models on the same provider's price sheet can differ tenfold in price per token, or more. But a large share of typical workloads — classification, extraction, routing, short summaries — passes the quality bar on the smaller model already. Sending those requests to a flagship model increases cost without improving the outcome.

Price is only one factor, though. Models also differ in what they can do: some accept images or audio, some are built for tool calling or strict structured output, some reason step by step while others just answer directly. The rule is the same either way: pick the smallest model that covers what the task actually needs. Do not pay for image support on a text-only pipeline, and do not pay for a reasoning model just to reformat JSON. Requirements change over time, so treat your model choice as temporary, not final. Providers release cheaper and better models every few months, and a task that needed the flagship model yesterday often works fine on the mini model next quarter.

This only pays off if switching models is easy. Spring AI's answer is that the model is configuration, not architecture. ChatClient

does not depend on any one provider or model: your service code does not need to know which model answers, portable options like maxTokens

and temperature

carry over between models, and provider-specific settings stay isolated in one options object. Spring AI 2.0 makes this explicit: default options set on a ChatClient

act as a partial "delta" over the model's start-up configuration. The framework signals this same idea in its own defaults: the OpenAI integration in 2.0 defaults to gpt-5-mini, not the flagship model. A good pattern is one client per cost tier:

@Configuration
public class ChatClients {

    @Bean
    @Primary
    ChatClient cheapClient(ChatClient.Builder builder) {
        return builder
              .defaultOptions(ChatOptions.builder()
                  .model("gpt-5-mini"))
              .build();
    }

    @Bean
    ChatClient flagshipClient(ChatClient.Builder builder) {
        return builder
              .defaultOptions(ChatOptions.builder()
                  .model("gpt-5"))
              .build();
    }
}

A ticket classifier feature could inject @Qualifier("cheapClient") ChatClient

, while a contract-analysis feature injects the flagship one. Two details matter here. First, build clients from the auto-configured builder, not from a raw ChatClient.builder(chatModel)

. The auto-configured builder already has observability wired in, and the token metrics from Driver #0 depend on that. Second, this example covers two tiers on one provider. If you mix vendors in the same application, create a separate ChatClient

from each auto-configured ChatModel

instead. Either way, the size of the change stays small: swapping models is a property change, and swapping vendors just adds a starter dependency plus properties. No code changes, no prompt rewrites, no new SDK to learn.

A ChatClient

default is not free. Everything attached to the client — system prompt, tools, memory advisor — is included in every request that client sends. One shared "do-everything" client means a simple 10-token call (e.g. a classifier feature) carries the same 2,000-token system prompt, tool schemas, and conversation history as your most complex feature. If the classifier runs 50,000 times a month, that is 100 million input tokens of extra content it never actually needs — $100 a month at the example rate, for nothing.

Spring AI's unit of cost control is the client, not the whole application. ChatClient.Builder is auto-configured as a prototype bean, so each task can build its own client with only the defaults that task needs:

.defaultSystem()

for the prompt, .defaultOptions()

for model and token settings, .defaultAdvisors()

and .defaultTools()

for the rest. Every later driver in this series — memory windows, RAG advisors, tool search — is attached at exactly this level. This is why building per-task clients should come before the drivers that follow.

@Configuration
class TaskClients {

    @Bean
    ChatClient classifierClient(ChatClient.Builder builder) {
        return builder
                .defaultSystem("Classify the ticket. Reply with one word: BILLING, TECH, or OTHER.")
                .defaultOptions(ChatOptions.builder()
                        .model("gpt-5-mini")
                        .maxTokens(5)
                        .temperature(0.0))
                .build();
    }

    @Bean
    ChatClient supportChatClient(ChatClient.Builder builder, ChatMemory memory) {
        return builder
                .defaultSystem(new ClassPathResource("prompts/support.st"))
                .defaultOptions(ChatOptions.builder()
                        .model("gpt-5")
                        .maxTokens(800))
                .defaultAdvisors(MessageChatMemoryAdvisor.builder(memory).build())
                .build();
    }
}

Note the[Spring AI 2.0 change]:.defaultOptions()

and the per-call.options()

now take aChatOptions.Builder

, not a fully built instance. This builder is merged with the model's defaults before the first advisor runs, and only the fields you explicitly set override them. That merge lets you override only the options you need without replacing the model's configured defaults:

classifierClient.prompt(text)
    .options(ChatOptions.builder().maxTokens(1)) // override for a yes/no case
    .call()
    .content();

The result: each request pays only for the defaults its own task actually declared. That is a requirement for every control that follows.

Drivers #1 and #2 decide which model answers a request and which client sends it; Driver #0 shows you what those decisions cost. You now have one client per task, each carrying only the defaults that task needs, and the token metrics to prove it.

None of that limits the size of a single request. A reasoning model can bill more hidden thinking than visible answer. A support conversation resends its whole history on every turn. The same system prompt goes out thousands of times a day at full price, unless you structure it so the provider can cache it.

Part 2 covers those three. Each one is a limit or an advisor you attach to the clients you have just built.

Part 2 arrives in August 2026.