Your Spring AI tests are slow, flaky, and cost money. Here's how to make them deterministic.

Rifat Cakir released spring-ai-test-tools, a VCR-style library that records real Spring AI interactions and replays them deterministically in tests, eliminating slow, flaky, and costly live model calls. The library attaches to ChatClient.Builder via ChatClientBuilderCustomizer, caches exact SHA-256 request hashes, and supports REPLAY_ONLY mode for CI, making cache misses loud failures. Replay runs in about 0.8 ms versus seconds for live calls, and it supports tool calls, streaming, entity mapping, embeddings, and evaluators.

You wire up Spring AI, the ChatClient fluent API feels great, your feature works. Then you sit down to write a test — and every good option is bad. A test that calls a real model is: The usual workarounds all hurt: Mockito means hand-building Spring AI's nested ChatResponse → Generation → AssistantMessage graph and asserting against a response you wrote; WireMock/MockWebServer means owning each provider's exact wire JSON, SSE frames, and tool-call envelopes, and rewriting it all when you switch providers; the real model is the four problems above, accepted rather than solved. There's a well-worn answer from the HTTP world — Ruby's VCR, Python's vcrpy : record the real interaction once, replay it deterministically after. The catch is those work at the HTTP layer, and Spring AI's value is the abstraction above HTTP. So I built the same idea where Spring AI actually lives. One dependency: <dependency <groupId io.github.rifatcakir</groupId <artifactId spring-ai-test-tools</artifactId <version 0.1.0</version <scope test</scope </dependency One property src/test/resources/application-test.yml : spring: ai: test: vcr: enabled: true mode: RECORD OR REPLAY REPLAY ONLY in CI Your test doesn't change at all — you write it exactly as you would against a real model: @SpringBootTest class OrderStatusTest { @Autowired ChatClient.Builder chatClientBuilder; @Test void answersAQuestionAboutTheOrder { String answer = chatClientBuilder.build .prompt .user "What's the status of order ORD-4471?" .call .content ; assertThat answer .contains "shipped" ; } } First run reaches a real model and writes src/test/resources/llm-cache/{sha256}.json — you commit that file. Every run after replays it in under a millisecond, offline. FIRST RUN slow · costs tokens · needs network Your test ──▶ ChatClient ──▶ Real LLM ──writes──▶ cassette.json committed EVERY RUN AFTER instant · $0 · fully offline Your test ──▶ ChatClient ◀──reads── cassette.json ~0.8 ms The advisor attaches to every ChatClient.Builder in the context via ChatClientBuilderCustomizer — so nothing under test, and nothing in production, knows the cache exists. In CI you seal it with mode: REPLAY ONLY : now a cache miss is a loud failure , not a silent call to a live model. The cache key is an exact SHA-256 over the canonical request; there is no fuzzy matching, ever. This is why Spring AI's production semantic cache doesn't solve the testing problem — it matches on similarity thresholds, which is exactly backwards for a test. The point isn't a benchmark number — it's what disappears: And yes, replay is ~0.8 ms median over 200 timed iterations in a real Spring context versus a warm hosted call of ~1–2 s or a local cold call of ~47 s — but treat that as a side effect. The real win is that the network, the cost, and the rate limits are simply gone. Up front, because senior engineers rightly distrust silver bullets — this sits above the HTTP layer, so it cannot test that layer: Retry-After , connection pooling, a body arriving malformed mid-stream → that's Each of these is verified against a real model, not assumed: @Tool call's name and arguments are part of the cache key, and on replay the recorded tool result is injected Flux<ChatResponse replays chunk-for-chunk — not a single-chunk fake — tool-call fragments included. .entity MyDto.class call's target schema is part of the cache key, so two output types with the same prompt never collide. EmbeddingModel calls cache independently of chat; a replayed vector is exactly, not approximately, what was recorded. RelevancyEvaluator / FactCheckingEvaluator run deterministically in CI the judge call itself is recorded , or live on demand for a drift check.Independent, community project not affiliated with Spring/Broadcom , Apache-2.0, currently 0.1.0 and early — tested against Java 21 · Spring Boot 4.0.0 · Spring AI 2.0.0. If you try it, issues and feedback are genuinely wanted.