# Your Spring AI tests are slow, flaky, and cost money. Here's how to make them deterministic.

> Source: <https://dev.to/rifatcakir/your-spring-ai-tests-are-slow-flaky-and-cost-money-heres-how-to-make-them-deterministic-1og7>
> Published: 2026-08-11 10:07:48+00:00

You wire up Spring AI, the `ChatClient`

fluent API feels great, your feature works. Then you sit down to write a test — and every good option is bad.

A test that calls a real model is:

The usual workarounds all hurt: **Mockito** means hand-building Spring AI's nested `ChatResponse → Generation → AssistantMessage`

graph and asserting against a response *you* wrote; **WireMock/MockWebServer** means owning each provider's exact wire JSON, SSE frames, and tool-call envelopes, and rewriting it all when you switch providers; **the real model** is the four problems above, accepted rather than solved.

There's a well-worn answer from the HTTP world — Ruby's VCR, Python's `vcrpy`

: record the real interaction once, replay it deterministically after. The catch is those work at the HTTP layer, and Spring AI's value is the abstraction *above* HTTP. So I built the same idea where Spring AI actually lives.

One dependency:

```
<dependency>
    <groupId>io.github.rifatcakir</groupId>
    <artifactId>spring-ai-test-tools</artifactId>
    <version>0.1.0</version>
    <scope>test</scope>
</dependency>
```

One property (`src/test/resources/application-test.yml`

):

```
spring:
  ai:
    test:
      vcr:
        enabled: true
        mode: RECORD_OR_REPLAY   # REPLAY_ONLY in CI
```

Your test doesn't change at all — you write it exactly as you would against a real model:

```
@SpringBootTest
class OrderStatusTest {

    @Autowired ChatClient.Builder chatClientBuilder;

    @Test
    void answersAQuestionAboutTheOrder() {
        String answer = chatClientBuilder.build().prompt()
            .user("What's the status of order ORD-4471?")
            .call().content();

        assertThat(answer).contains("shipped");
    }
}
```

First run reaches a real model and writes `src/test/resources/llm-cache/{sha256}.json`

— **you commit that file.** Every run after replays it in under a millisecond, offline.

```
FIRST RUN          slow · costs tokens · needs network
  Your test ──▶ ChatClient ──▶ Real LLM  ──writes──▶  cassette.json  (committed)

EVERY RUN AFTER    instant · $0 · fully offline
  Your test ──▶ ChatClient ◀──reads──  cassette.json                 (~0.8 ms)
```

The advisor attaches to every `ChatClient.Builder`

in the context via `ChatClientBuilderCustomizer`

— so **nothing under test, and nothing in production, knows the cache exists.** In CI you seal it with `mode: REPLAY_ONLY`

: now a cache miss is a *loud failure*, not a silent call to a live model. The cache key is an exact SHA-256 over the canonical request; there is no fuzzy matching, ever. (This is why Spring AI's *production* semantic cache doesn't solve the testing problem — it matches on similarity thresholds, which is exactly backwards for a test.)

The point isn't a benchmark number — it's what disappears:

And yes, replay is ~0.8 ms (median over 200 timed iterations in a real Spring context) versus a warm hosted call of ~1–2 s or a local cold call of ~47 s — but treat that as a side effect. The real win is that the network, the cost, and the rate limits are simply gone.

Up front, because senior engineers rightly distrust silver bullets — this sits *above* the HTTP layer, so it cannot test that layer:

`Retry-After`

, connection pooling, a body arriving malformed mid-stream → that's Each of these is verified against a real model, not assumed:

`@Tool`

call's name and arguments are part of the cache key, and on replay the recorded tool result is injected `Flux<ChatResponse>`

replays chunk-for-chunk — not a single-chunk fake — tool-call fragments included.`.entity(MyDto.class)`

call's target schema is part of the cache key, so two output types with the same prompt never collide.`EmbeddingModel`

calls cache independently of chat; a replayed vector is exactly, not approximately, what was recorded.`RelevancyEvaluator`

/ `FactCheckingEvaluator`

run deterministically in CI (the judge call itself is recorded), or live on demand for a drift check.Independent, community project (not affiliated with Spring/Broadcom), Apache-2.0, currently `0.1.0`

and early — tested against Java 21 · Spring Boot 4.0.0 · Spring AI 2.0.0. If you try it, issues and feedback are genuinely wanted.