You wire up Spring AI, the ChatClient
fluent API feels great, your feature works. Then you sit down to write a test — and every good option is bad.
A test that calls a real model is:
The usual workarounds all hurt: Mockito means hand-building Spring AI's nested ChatResponse → Generation → AssistantMessage
graph and asserting against a response you wrote; WireMock/MockWebServer means owning each provider's exact wire JSON, SSE frames, and tool-call envelopes, and rewriting it all when you switch providers; the real model is the four problems above, accepted rather than solved.
There's a well-worn answer from the HTTP world — Ruby's VCR, Python's vcrpy
: record the real interaction once, replay it deterministically after. The catch is those work at the HTTP layer, and Spring AI's value is the abstraction above HTTP. So I built the same idea where Spring AI actually lives.
One dependency:
<dependency>
<groupId>io.github.rifatcakir</groupId>
<artifactId>spring-ai-test-tools</artifactId>
<version>0.1.0</version>
<scope>test</scope>
</dependency>
One property (src/test/resources/application-test.yml
):
spring:
ai:
test:
vcr:
enabled: true
mode: RECORD_OR_REPLAY # REPLAY_ONLY in CI
Your test doesn't change at all — you write it exactly as you would against a real model:
@SpringBootTest
class OrderStatusTest {
@Autowired ChatClient.Builder chatClientBuilder;
@Test
void answersAQuestionAboutTheOrder() {
String answer = chatClientBuilder.build().prompt()
.user("What's the status of order ORD-4471?")
.call().content();
assertThat(answer).contains("shipped");
}
}
First run reaches a real model and writes src/test/resources/llm-cache/{sha256}.json
— you commit that file. Every run after replays it in under a millisecond, offline.
FIRST RUN slow · costs tokens · needs network
Your test ──▶ ChatClient ──▶ Real LLM ──writes──▶ cassette.json (committed)
EVERY RUN AFTER instant · $0 · fully offline
Your test ──▶ ChatClient ◀──reads── cassette.json (~0.8 ms)
The advisor attaches to every ChatClient.Builder
in the context via ChatClientBuilderCustomizer
— so nothing under test, and nothing in production, knows the cache exists. In CI you seal it with mode: REPLAY_ONLY
: now a cache miss is a loud failure, not a silent call to a live model. The cache key is an exact SHA-256 over the canonical request; there is no fuzzy matching, ever. (This is why Spring AI's production semantic cache doesn't solve the testing problem — it matches on similarity thresholds, which is exactly backwards for a test.)
The point isn't a benchmark number — it's what disappears:
And yes, replay is ~0.8 ms (median over 200 timed iterations in a real Spring context) versus a warm hosted call of ~1–2 s or a local cold call of ~47 s — but treat that as a side effect. The real win is that the network, the cost, and the rate limits are simply gone.
Up front, because senior engineers rightly distrust silver bullets — this sits above the HTTP layer, so it cannot test that layer:
Retry-After
, connection pooling, a body arriving malformed mid-stream → that's Each of these is verified against a real model, not assumed:
@Tool
call's name and arguments are part of the cache key, and on replay the recorded tool result is injected Flux<ChatResponse>
replays chunk-for-chunk — not a single-chunk fake — tool-call fragments included..entity(MyDto.class)
call's target schema is part of the cache key, so two output types with the same prompt never collide.EmbeddingModel
calls cache independently of chat; a replayed vector is exactly, not approximately, what was recorded.RelevancyEvaluator
/ FactCheckingEvaluator
run deterministically in CI (the judge call itself is recorded), or live on demand for a drift check.Independent, community project (not affiliated with Spring/Broadcom), Apache-2.0, currently 0.1.0
and early — tested against Java 21 · Spring Boot 4.0.0 · Spring AI 2.0.0. If you try it, issues and feedback are genuinely wanted.