AI Intro Part 4: Workflow Patterns (Part 2)

Anthropic's whitepaper 'Building Effective Agents' outlines five workflow patterns for using large language models, and this post covers the final three: Parallelization, Orchestrator-Workers, and Evaluator-Optimizer. Parallelization splits tasks into simultaneous calls, either by sectioning (e.g., guardrails) or voting (e.g., content moderation), and Anthropic recommends using the simplest solution, not necessarily building agentic systems.

In the last post, I covered an important difference between workflows and agents: Workflows, or workflow patterns, are systems where LLMs are exercised used through predefined code paths. Agents, or agentic systems, are systems in which LLMs direct their own processes and tool use. Anthropic reminds us clearly that you might not need an agentic system: “We recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all.” Non-agentic systems that use LLMs to do some of most of their work are called workflows . They are workflow applications that leverage LLMs to perform specific tasks but aren’t actually driven by the LLM itself. Much of this post has been adapted from the public whitepaper “ Building Effective Agents https://www.anthropic.com/engineering/building-effective-agents ,” with more detailed explanations and code examples in Python provided by me. In the last post, I covered Prompt Chaining and Routing, the first two out of five workflow patterns. In this post we will patterns 3, 4, and 5: Parallelization including section & versioning Orchestrator-Workers Evaluator-Optimizer Parallelization Parallelization is like the Routing workflow, but we send things to multiple LLMs at the same time—in “parallel.” Parallelization is effective when the subtasks can be divided up cleanly. In particular, because the operation happens in parallel, it will happen much faster than if each step were sequential. Anthropic says: “For complex tasks with multiple considerations, LLMs generally perform better when each consideration is handled by a separate LLM call, allowing focused attention on each specific aspect.” There are two kinds of parallelization: Sectioning and Voting. With sectioning, you split one job into different pieces. A common use is guardrails. One call answers the user’s question. A second call, running at the same time, checks that same input for anything you don’t want to respond to. The two calls never see each other’s work. If the screener flags the input, you throw away the answer before it reaches the user. Anthropic notes this beats asking a single call to do both jobs, and the reason is straightforward: a prompt that says “answer this, and also police it” splits the model’s attention. A prompt that says “screen this” does one thing. Evals work the same way. Say you want to score generated copy on tone, factual accuracy, and length. Three calls, one per criterion, all at once. Each prompt is short and specific. You get three scores back and combine them in your code. With voting, every call gets the same input and answers the same question. What changes is the prompt, or nothing at all. Then you count the answers. Run five prompts against the same function looking for security problems. If three of the five flag something, you flag it. You set the threshold. One vote catches nearly everything, including many false alarms. Unanimous only surfaces the obvious. Content moderation lives on that dial. Sectioning and voting can look similar in code. Both fan out, both wait, both collect. The difference is whether the calls could disagree. In sectioning, they can’t, because they’re answering different questions. In voting, they can, and the disagreement is the heart of what you’re measuring. Sectioning: the guardrail python import anthropic from concurrent.futures import ThreadPoolExecutor client = anthropic.Anthropic def ask prompt : response = client.messages.create model="claude-sonnet-4-6", max tokens=1024, messages= {"role": "user", "content": prompt} , return response.content 0 .text def answer query : return ask f"You are a helpful support agent. Answer: {query}" def screen query : return ask f"Does this message ask for anything harmful, off-topic, or " f"against policy? Reply with one word: SAFE or BLOCK.\n\n{query}" .strip .upper def handle query : with ThreadPoolExecutor as pool: answer future = pool.submit answer, query screen future = pool.submit screen, query if screen future.result == "BLOCK": return "Sorry, I can't help with that." return answer future.result print handle "How do I reset my password?" Step through handle. ThreadPoolExecutor is Python’s thread pool. pool.submit hands it a function and its argument, then returns immediately with a future. The call is now running on a background thread. Submit twice and you have two calls in flight. The with block waits for both threads to finish before it exits. After that, .result pulls the return value out of each future. By the time you read them, the work is done, so neither .result blocks. Then you check the screener first. If it says BLOCK , you return the canned refusal and never touch the answer. The answer already came back and you throw it away. That waste is deliberate. You paid for tokens you didn’t use. What you bought is time. The user waits for the slower of the two calls, not the sum of both. Run them in sequence and you’d save the tokens but double the wait on every clean request, which is most of them. Voting: the security review REVIEWERS = "Look for SQL injection. Reply VULNERABLE or SAFE, then one line why.", "Look for missing authentication or authorization checks. Reply VULNERABLE or SAFE, then one line why.", "Look for unvalidated user input. Reply VULNERABLE or SAFE, then one line why.", "Look for secrets or credentials in the code. Reply VULNERABLE or SAFE, then one line why.", "You are a senior security engineer. What is wrong with this code? Reply VULNERABLE or SAFE, then one line why.", def review instruction, code : return ask f"{instruction}\n\n python\n{code}\n " def audit code, threshold=2 : with ThreadPoolExecutor as pool: results = list pool.map lambda r: review r, code , REVIEWERS votes = r for r in results if r.strip .upper .startswith "VULNERABLE" return { "flagged": len votes = threshold, "votes": len votes , "findings": votes, } code = """ def get user request : uid = request.args.get "id" return db.execute "SELECT FROM users WHERE id = " + uid """ print audit code Step through audit. REVIEWERS is a list of five instruction strings. pool.map takes a function and that list, calls the function once per item, and runs all five at the same time. The lambda is there to pin the second argument, since map only passes one thing per call: each reviewer instruction goes in, code stays the same every time. map returns the results in the order of the input list, not the order the calls finish. Wrapping it in list waits for all five and collects them. Then we count it. The filter keeps any result whose text starts with VULNERABLE . len votes is the tally. Compare that against threshold and you get a boolean. Note what is returned: the function returns the flag, the tally, and the actual findings, so the consumer can see which reviewers objected and why. Orchestrator-Workers Orchestrator-workers looks similar to parallelization. Both fan work out to multiple LLM calls and collect the results, but with the orchestrator-workers pattern, there’s an orchestrator LLM who decides what work gets done. In parallelization, your code decides how to split the job into pieces before anything runs. In orchestrator-workers, an LLM decides. A central “orchestrator” call looks at the input, breaks it into subtasks on the fly, and hands each one to a worker call. A final step pulls the worker outputs back together. Anthropic frames the distinction this way: parallelization subtasks are “pre-defined,” while orchestrator-workers subtasks are “determined by the orchestrator based on the specific input” Building Effective Agents . That’s the whole difference in one sentence. You lose the guarantee of parallelization’s fixed shape, but you gain the ability to handle a task where you can’t know the shape in advance. Research and search tasks work this way. You don’t know how many sources are worth pulling until you’ve started pulling them. The orchestrator can spin up more workers, or fewer, depending on what the first few turn up. python import json import anthropic from concurrent.futures import ThreadPoolExecutor client = anthropic.Anthropic def ask prompt, system=None : response = client.messages.create model="claude-sonnet-4-6", max tokens=1500, system=system, messages= {"role": "user", "content": prompt} , return response.content 0 .text def plan topic : prompt = f"""You're planning research for a briefing document. Topic: {topic} Decide which subtopics need separate research to cover this well. Reply as JSON: a list of objects with "subtopic" and "angle" keys, where "angle" is one sentence on what to focus on for that subtopic. Use 3 to 6 subtopics, no more than needed.""" raw = ask prompt, system="Reply with JSON only, no markdown fences." return json.loads raw def research subtopic, angle, topic : prompt = f"""You're researching one piece of a larger briefing on: {topic} Your subtopic: {subtopic} Your angle: {angle} Write three or four sentences covering this subtopic. Be specific. If you're not confident about a fact, say so instead of guessing.""" return ask prompt def synthesize topic, findings : combined = "\n\n".join f"{f 'subtopic' }:\n{f 'content' }" for f in findings prompt = f"""Topic: {topic} Here's research gathered on separate subtopics: {combined} Write a briefing document. Open with a two sentence summary, then cover each subtopic in its own short section. Flag anywhere the research disagreed or left gaps.""" return ask prompt def run topic : subtopics = plan topic with ThreadPoolExecutor as pool: futures = { pool.submit research, s "subtopic" , s "angle" , topic : s for s in subtopics } findings = for future, subtopic in futures.items : findings.append { "subtopic": subtopic "subtopic" , "content": future.result , } return synthesize topic, findings briefing = run "Our main competitor's recent product launch" print briefing Let’s look at the run function above. The plan call which is doing the orchestration , gets one input: the topic. It returns a list of subtopics and an angle for each. Ask about a product launch, and you might get pricing, feature set, market reception, and positioning against your own product. Ask about a different topic, and you get a different list. Each subtopic becomes a research call, and those run in parallel through the thread pool, same as parallelization. Each worker only sees its own subtopic and angle. It doesn’t see what the other workers are finding, so there’s no risk of one worker’s guess contaminating another’s. synthesize is the piece that makes this pattern more than parallelization with extra steps. It concatenates the findings, then reads across them and writes something coherent, including calling out where two pieces of research contradict each other or where a subtopic came back thin. Since that’s a judgment call and not a formatting task, it gets an LLM call instead of a template. When you don’t know how many angles a topic needs until you’ve looked at the topic, this pattern fits research better than a fixed pipeline. The orchestrator decides how much research the topic needs. A minor update gets two workers. A full launch gets six. You don’t pick that number in advance; you let the orchestrator decide. Evaluator-Optimizer The last pattern is Evaluator-Optimizer. It’s a loop. One LLM generates a result. A second LLM evaluates it and either accepts it or sends it back with feedback. This pattern works well when you have a clear standard to check against, something closer to a pass or fail than a judgment call. When the evaluator rejects a result, it sends the generator all the previous attempts along with specific feedback on what was wrong. The generator uses both as input for the next attempt. That’s the refinement loop. Anthropic uses the analogy of an author and an editor. The author sends a chapter. The editor reads it, marks it up, and sends it back. The author revises. That cycle repeats until the editor is satisfied. The pattern also fits search well. A single search might not fully answer a real question. In this pattern, you run a query, look at what came back, and decide if you have enough to work with or if to try another pass. An evaluator makes that call instead of you hardcoding a fixed number of search rounds. Some queries might resolve in one search. Others need three or four, each one narrower than the last based on what the previous round turned up. python import json import anthropic client = anthropic.Anthropic def ask prompt, system=None : response = client.messages.create model="claude-sonnet-4-6", max tokens=1500, system=system, messages= {"role": "user", "content": prompt} , return response.content 0 .text def search query : stand-in for a real search call return f" search results for: {query} " def generate query topic, history : if not history: prompt = f"Write a search query to research: {topic}" else: last = history -1 prompt = f"""Topic: {topic} Previous query: {last 'query' } Previous results: {last 'results' } Evaluator feedback: {last 'feedback' } Write a better search query that addresses the feedback.""" return ask prompt, system="Reply with just the search query, nothing else." def evaluate topic, history : combined = "\n\n".join f"Query: {h 'query' }\nResults: {h 'results' }" for h in history prompt = f"""Topic: {topic} Search history so far: {combined} Do we have enough information to write a complete, well-supported answer on this topic? Reply as JSON with keys "sufficient" true or false and "feedback" one or two sentences: if sufficient, say why; if not, say specifically what's missing or what to search next .""" raw = ask prompt, system="Reply with JSON only, no markdown fences." return json.loads raw def synthesize topic, history : combined = "\n\n".join f"Query: {h 'query' }\nResults: {h 'results' }" for h in history prompt = f"""Topic: {topic} Search history: {combined} Write a complete answer to the topic, drawing on all the search results above.""" return ask prompt def run topic, max rounds=4 : history = for round num in range max rounds : query = generate query topic, history results = search query history.append {"query": query, "results": results, "feedback": None} verdict = evaluate topic, history history -1 "feedback" = verdict "feedback" if verdict "sufficient" : break return synthesize topic, history answer = run "What caused the 2024 slowdown in EV sales growth?" print answer Each pass through the loop does three things: generate a query, run the search, and evaluate the results. generate query behaves differently depending on whether history is empty. On the first round it just writes a starting query from the topic. On every round after that, it sees the previous query, what came back, and the evaluator’s feedback, and writes a query meant to close that specific gap. evaluate is the core of the machine: It instructs the LLM to respond with ‘sufficient’ or ‘feedback’ These can be configured to respond directly in the JSON result of what the LLM returns, since unlike web LLM interfaces that most peolpe are used to, LLM API can be defined with specific, structured responses – you tell the AI specific data points. The entire response is in a JSON JavaScript object so it can be easily parsed consumed by your code. The Evaluator looks at everything gathered so far, not just the latest result, and decides if it’s enough to answer the topic well. It returns a boolean and a reason. The boolean controls the loop. The reason becomes the feedback the next generate query call reads. max rounds is there so a topic that never quite satisfies the evaluator doesn’t loop forever. Four rounds is arbitrary, but some ceiling is necessary any time an LLM controls the exit condition, to avoid creating an LLM infinite loop. Once the loop ends, either because the evaluator was satisfied or because it ran out of rounds, synthesize writes the final answer from the full search history. Early rounds often surface context that later rounds build on, and the final answer should read like it used all of it, not just the most recent search. One downside of this pattern is variable runtime. A well-scoped topic might resolve in one round. A broad or ambiguous one could burn through all four. That’s the trade-off: you give up a predictable number of calls in exchange for not stopping short on a question that needed more digging. —- That’s all five workflow patterns: Prompt Chaining , Routing , Parallelization , Orchestrator-Workers , and Evaluator-Optimizer . They share a common trait. In every case, your code controls the structure. The LLM does work inside that structure, but it doesn’t choose the structure itself. That’s what makes these workflows and not agents. The patterns also build on each other logically. Chaining is sequential. Routing adds a decision point. Parallelization removes the sequence. Orchestrator-workers let the LLM decide the shape of the parallelization. Evaluator-optimizer adds a feedback loop. Each one trades simplicity for flexibility, and you should only make that trade when the simpler pattern can’t do the job.