When I started this project, I was not trying to build the most advanced multi-agent system. I wanted to understand what actually changes when a research workflow moves from one agent to several specialized agents — and whether adding orchestration makes the system more reliable.
I built three research workflows with LangGraph and connected them to a web-search tool. The first used a single ReAct-style agent. The second separated research, writing, and critique into a fixed pipeline. The third added a supervisor that selected which role should act next.
I evaluated each system across the same five research topics, running each topic three times per system. That produced 45 runs in total. I tracked completion and failure rates, execution behavior, token usage, tool calls, and citation-related measurements.
The main lesson was not that more agents are always better. It was that separating responsibilities changes the system’s behavior — and changes the kinds of failures that appear.
System A was a single-agent ReAct-style workflow. The agent reasoned about the topic, called the web-search tool, inspected results, decided whether another search was needed, and wrote the final report. The implementation is in system_a_baseline.py.
This design is straightforward: one main conversation controls the research and writing process. But tool calls, search results, retry messages, and writing instructions accumulate in the same context. In this implementation, that became a source of tool-call errors.
System A completed 8 of 15 runs and failed 7. All seven failures contained the same type of error: “Tool choice is none, but model called a tool.” Some invalid arguments contained fields such as {“cursor”: 0, “id”: 0} where the search tool expected a query.
System B divided the workflow into three roles: a researcher, a writer, and a critic. The researcher gathered notes using a bounded search loop. The writer turned those notes into an article. The critic reviewed the draft for accuracy, specificity, and clarity. If the draft was not approved, the writer received the feedback and revised it.
This separation meant that the writer did not directly control the search tool. It received research notes instead of continuing the full tool-use conversation. System B completed 13 of 15 runs. Its two failures had different causes: one critic returned invalid JSON, and one run failed because of a rate-limit error.
The improvement was not that the underlying model became more capable. The architecture reduced the number of responsibilities handled inside one conversation. Separating tool use from writing removed the specific malformed search-call failure observed in System A.
System B also exposed a citation-preservation problem. The researcher collected source URLs, but the writer usually omitted them from the final draft. The writer prompt asked for a clear, factual article based on the notes, but did not explicitly require source URLs to be preserved. This was a handoff and prompt-design problem, not evidence that the researcher had failed to retrieve sources.
System C added a supervisor above the researcher, writer, and critic. At each step, the supervisor received a compact description of the workflow state and proposed one of four actions: researcher, writer, critic, or finish.
For example, the supervisor could select the researcher when no research notes existed, the writer when notes were available but no draft existed, the critic when a draft needed review, or finish when the draft was approved. The supervisor did not have unrestricted control. The program applied deterministic safeguards after receiving its decision. The system could not finish without a draft; the critic could not run before a draft existed; research could not be repeated after notes were available; and revision, token, and recursion limits were enforced. Retryable errors could be retried, while invalid worker output was recorded as a failure.
This led to one of the most useful design principles in the project: the language model proposes a route, but the program enforces the rules.
System C completed 11 of 15 runs and failed 4. Three failures occurred in the researcher stage because findings did not contain valid URLs. One run reached the graph recursion limit. Its stricter requirements could protect downstream stages from poorly structured research output, but they could also reject output that a less strict workflow might have accepted.
I used the same five research topics for all three systems: Photosystem II in oxygenic photosynthesis; quantum-computing architectures in 2026; conflicting evidence about caffeine and cardiovascular outcomes; recent global semiconductor trade policies; and the development and dialectal features of the Elfdalian language.
Each system was run three times on each topic, giving 15 runs per system and 45 runs overall. The evaluation ran in a Python 3.12.3 virtual environment. The main versions were LangGraph 0.2.70, LangChain Core 0.3.36, LangChain Groq 0.2.4, DuckDuckGo Search 7.3.0, and Pydantic 2.14.1.
The model was openai/gpt-oss-20b through Groq. Temperatures differed by role: System A used 0.0 for normal calls and 0.3 for its retry model; System B used 0.0 for the researcher, 0.7 for the writer, and 0.0 for the critic; System C used 0.0 for the supervisor and inherited the System B role temperatures.
Search used DuckDuckGo through the DDGS interface. Each search returned up to three results. Results were cached locally, and each formatted result was truncated to approximately 2,500 characters before being passed to the model.
The systems also had different workflow limits. System A allowed up to four search-tool executions and used a LangGraph recursion limit of 20. System B allowed up to three tool iterations per researcher pass and up to two writer revisions. System C used a maximum of two revisions, a 20,000-token per-run budget, a recursion limit of 20, up to three research notes per researcher pass, and up to three retry attempts for retryable failures. These were the settings of the implementations tested, not universal recommendations.
A run counted as failed if the system raised an exception, explicitly returned a failure state, produced an error message, or returned an empty final report. In simplified form, the evaluator used this rule:
failed = bool(error or result.get(“failed”, False) or len(report.strip()) == 0) Completion rate was calculated as completed runs divided by total runs. Failure rate was calculated as failed runs divided by total runs. Since each system had 15 runs, one run represented approximately 6.7 percentage points.
System A — 8/15 completed · 53.3% completion · 46.7% failure
System B — 13/15 completed · 86.7% completion · 13.3% failure
System C — 11/15 completed · 73.3% completion · 26.7% failure
System B achieved the highest completion rate in this evaluation. That does not make it universally superior. The systems had different routing behavior, validation rules, and failure modes. System A was flexible but vulnerable to malformed tool calls. System B was more predictable and completed more runs, but lost citations during the researcher-to-writer handoff. System C added routing, validation, error tracking, and budget control, but its stricter validation produced researcher failures and one recursion-limit failure.
These figures describe these particular implementations and this experiment. They should not be interpreted as universal performance guarantees for ReAct, fixed pipelines, or supervisory systems.
The project also showed why citation presence and evidence quality should not be treated as the same thing. The evaluator compared URLs in the final report with URLs found in the research notes. If the final report contained no URLs, the citation-validity score was recorded as 0.0.
Only one System B report visibly contained URLs, which explains its mean citation-validity score of approximately 0.067. In most runs, the researcher notes contained search URLs, but the writer did not copy them into the final report. The low score therefore primarily reflects a citation-preservation problem in the handoff, not proof of poor source retrieval.
There is a second limitation: the metric measures whether a cited URL appeared in the research notes. It does not establish whether the source supports the exact claim. The evaluation also relied on search snippets rather than full source pages. A relevant search result may not provide enough information to verify a specific number, date, or named entity.
For that reason, I treat these results as URL-preservation and citation-validity measurements, not as a complete evaluation of factual grounding. A stronger evaluation would distinguish citation presence, URL retrieval, support for the general claim, and support for exact details. System A could be tested with a cleaner final synthesis step. One option is to make the final writing call without the earlier tool-call history and provide the collected evidence as plain text. I would keep the original and improved versions separate so the comparison remains transparent.
System B should pass structured evidence records rather than unstructured strings. Each record could contain a claim, supporting evidence, and source URL. The writer should then be explicitly instructed to preserve citations in the final report.
{ “claim”: “…”, “evidence”: “…”, “url”: “https://example.com/source" } The current URL metric does not prove that a source supports a claim. A future evaluation should inspect retrieved source text and assess whether it supports the general statement and any exact details. Fetching full source pages would be a useful next step, but it has not yet been implemented as part of these results.
The most important question about System C is whether its supervisor makes useful decisions that System B would not. If it mostly follows researcher → writer → critic → writer → critic, it may add tokens and latency without changing the work. If it requests more research for difficult topics, skips unnecessary work, or stops safely when evidence is insufficient, then supervision may provide a measurable benefit.
The corrected aggregate for System C reports a mean execution time of 53.67 seconds, mean total token usage of 5,793, mean LLM calls of 7.2, and mean tool calls of 6.8 across 15 runs. The four failures were three researcher failures caused by findings without valid URLs and one graph recursion-limit failure. An earlier aggregate incorrectly showed 100% failure and zero tokens; I regenerated the report from the raw metrics and used the corrected values here.
This project did not show that adding more agents automatically improves research quality. It showed that different orchestration designs produce different behavior and different failure modes.
The single-agent ReAct system was flexible but experienced repeated invalid tool-call failures. The fixed role-based pipeline completed the most runs in this evaluation, but its citation handoff needs improvement and its claim-level evidence quality remains unverified. The supervisor-controlled workflow added explicit routing, validation, error tracking, and budget control, while also introducing additional complexity and stricter failure conditions.
The lesson I am taking from this project is simple: multi-agent orchestration should not be judged by how many agents it contains. It should be judged by whether the added structure improves reliability, evidence grounding, and decision-making enough to justify its cost.
I started this project to learn how to build a research agent. I ended up learning that orchestration is not only about adding roles. It is also about deciding which responsibilities should be separated, which decisions should be left to the model, and which rules should remain under program control.
GITHUB REPOSITORY: https://github.com/GulrezQayyum/research-team-agents ABOUT THE AUTHOR: Gulrez Qayyum is a final-year Software Engineering student interested in AI engineering, Large Language Models, RAG systems, and agentic AI. She enjoys building practical software projects, experimenting with emerging AI technologies, and documenting the engineering lessons behind them.
Connect with me:
Building and Evaluating Three Multi-Agent Research Systems with LangGraph was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.