Parallelization Workflow with Spring AI: Fan-Out LLM Calls in Java A developer walkthrough examines Spring AI's parallelization workflow example, which fans out independent LLM calls across a configurable number of worker threads using a ChatClient and returns responses in input order. The pattern, drawn from Anthropic's "Building Effective Agents" guidance, supports sectioning (splitting a task into independent subtasks) and voting (repeating one prompt for consensus), while flagging costs around rate limits, per-call billing, and partial failures. When people first start building with LLMs in Java, the usual shape of the code is simple: build a prompt, call the model, read the answer. That works well for a single question. It starts to hurt when you need to ask the same model the same kind of question a hundred times, or when one request is really five smaller questions that happen to be asked together. Each call waits for the network, the model generates tokens at its own pace, and your thread sits idle for most of that time. Anthropic's article "Building Effective Agents" describes a set of workflow patterns for composing LLM calls. One of them is the Parallelization Workflow : instead of running model calls one after another, you run them at the same time and then combine the results. The Spring AI team has published a small, focused example of this pattern in the spring-ai-examples repository, under agentic-patterns/parallelization-workflow . It is a compact class built on top of Spring AI's ChatClient , and it is a good starting point for understanding how to fan out LLM work in a Spring application. In this article I walk through the pattern, the two variations it supports sectioning and voting , the code in the example, and the practical issues you will face when you use it for real workloads: ordering, failures, rate limits, cost, and testing. I will keep the code close to the example so you can follow along with the repository open. A typical chat completion can take anywhere from a second to tens of seconds, depending on the model, the size of the prompt, and the length of the answer. Those numbers matter when you multiply them. Imagine you need to analyze ten product reviews, one at a time, with a call that takes four seconds. That is forty seconds of wall-clock time, most of it spent waiting. If the calls are independent of one another, there is no reason to wait for review number three before starting review number four. You can send them all at once and collect the answers as they arrive. Parallelization helps in three ways: It also has costs. You pay for every call, not for every batch. You can overload a provider's rate limits. And your application has to handle partial failures, which are more complicated when several calls are in flight at once. The rest of this article is about getting the benefits without ignoring those costs. The example README names two variations of the pattern. They solve different problems, so it helps to keep them separate in your head. Sectioning breaks a larger task into independent subtasks and processes each one in parallel. The subtasks do not depend on each other, so the order of execution does not matter. Each call gets a different input and, often, a slightly different instruction. A typical case is analyzing several stakeholder groups, several document sections, or several customer segments with the same analytical prompt. Voting runs the same prompt several times, usually against the same input, and then combines the results. The goal is diversity of perspective or a consensus. If you ask a model whether a transaction looks fraudulent five times and four answers say "review," you have a more robust signal than a single answer. The aggregation step can be a majority vote, a rule that requires a minimum agreement, or another model call that summarizes the candidates. The distinction is practical. Sectioning is about speed on a set of different problems. Voting is about confidence on one problem. You can combine them, but it is clearer to start with one and then add the other. The core of the example is one class: ParallelizationlWorkflow , in the package com.example.agentic . It takes a ChatClient in its constructor and exposes a single method: public List