{"slug": "build-a-multi-agent-research-pipeline-with-crewai-and-ollama", "title": "Build a Multi-Agent Research Pipeline with CrewAI and Ollama", "summary": "A three-agent CrewAI pipeline backed by a locally running Llama 3.1 model via Ollama autonomously produces structured, cited research reports without requiring an OpenAI key. The pipeline uses a Researcher, Analyst, and Writer agent to gather facts, synthesize insights, and format the final document.", "body_md": "# Build a Multi-Agent Research Pipeline with CrewAI and Ollama\n\nAssemble a three-agent CrewAI crew backed by a locally running Llama 3.1 model to autonomously produce structured, cited research reports — no OpenAI key required.\n\n[Mariana Souza](https://www.devclubhouse.com/u/mariana_souza)\n\n## What You'll Build\n\nA three-agent CrewAI pipeline that takes a research topic and produces a formatted Markdown report with sourced findings. A Researcher gathers facts via web search, an Analyst synthesizes them, and a Writer produces the final document. All inference runs locally through Ollama.\n\n## Prerequisites\n\n- Python 3.10 or 3.11 (3.12 works; 3.9 does not)\n[Ollama](https://ollama.com/download)0.1.x or later installed- At least 8 GB of free RAM; 16 GB is comfortable for\n`llama3.1:8b`\n\n- macOS or Linux. On Windows, use WSL2.\n- A virtual environment tool (\n`venv`\n\n,`conda`\n\n, etc.)\n\n## Step 1: Get Ollama Running with Llama 3.1\n\nIf Ollama isn't already running as a background service, start it first:\n\n```\n# Only needed on Linux or if you didn't install the macOS .dmg app\nollama serve &\n```\n\nOn macOS with the app installed, Ollama starts at login automatically. Then pull the model:\n\n```\n# Downloads ~4.7 GB (Q4 quantization)\nollama pull llama3.1:8b\n```\n\nConfirm it's listening before continuing:\n\n```\ncurl http://localhost:11434/api/tags\n```\n\nYou should get JSON listing your local models. If you get `Connection refused`\n\n, Ollama isn't running yet.\n\n## Step 2: Install Python Dependencies\n\n```\npython3 -m venv .venv\nsource .venv/bin/activate\n\npip install \"crewai>=0.28.0\" \"langchain-ollama>=0.1.0\" \"langchain-community>=0.2.0\" duckduckgo-search\n```\n\n`langchain-ollama`\n\nis the standalone adapter split from `langchain-community`\n\nin LangChain 0.2.x. Prefer it over the older `langchain_community.chat_models.ChatOllama`\n\nimport path. CrewAI requires Pydantic v2, so verify with `pip show pydantic`\n\nif you're in a shared environment.\n\n## Step 3: Configure the LLM and Search Tool\n\nCreate `research_pipeline.py`\n\n. The LLM setup is a single object you'll hand to every agent:\n\n``` python\nfrom crewai import Agent, Task, Crew, Process\nfrom langchain_ollama import ChatOllama\nfrom langchain_community.tools import DuckDuckGoSearchRun\n\n# base_url defaults to http://localhost:11434\n# Only override if Ollama is on a different host or port\nllm = ChatOllama(model=\"llama3.1:8b\", temperature=0.1)\nsearch_tool = DuckDuckGoSearchRun()\n```\n\nLow temperature keeps the researcher and analyst factual. Bump it to 0.3 on the writer if you want less-dry prose.\n\n## Step 4: Define the Three Agents\n\n```\nresearcher = Agent(\n    role=\"Research Specialist\",\n    goal=\"Find accurate, recent information on the given topic and collect key facts with sources.\",\n    backstory=(\n        \"You are a meticulous researcher with a talent for locating credible sources \"\n        \"and summarizing them without losing detail.\"\n    ),\n    llm=llm,\n    tools=[search_tool],\n    allow_delegation=False,\n    verbose=True,\n)\n\nanalyst = Agent(\n    role=\"Data Analyst\",\n    goal=\"Identify patterns, gaps, and key insights from the research findings.\",\n    backstory=(\n        \"You specialize in transforming raw research into structured analysis, \"\n        \"separating signal from noise.\"\n    ),\n    llm=llm,\n    tools=[],\n    allow_delegation=False,\n    verbose=True,\n)\n\nwriter = Agent(\n    role=\"Technical Writer\",\n    goal=\"Produce a well-structured, cited research report suitable for a technical audience.\",\n    backstory=(\n        \"You write clear, authoritative reports. You cite sources precisely \"\n        \"and never pad content with filler.\"\n    ),\n    llm=llm,\n    tools=[],\n    allow_delegation=False,\n    verbose=True,\n)\n```\n\n`allow_delegation=False`\n\nprevents agents from spontaneously reassigning work mid-run. In a sequential pipeline it mostly avoids confusion rather than wasted inference.\n\n## Step 5: Define Tasks with Explicit Context\n\n```\nresearch_task = Task(\n    description=(\n        \"Search for recent developments, key players, and real-world use cases for: {topic}. \"\n        \"Collect at least five distinct facts and note the source URL for each.\"\n    ),\n    expected_output=(\n        \"A bullet-point list of findings, each with a source URL. \"\n        \"Minimum five items, no speculation.\"\n    ),\n    agent=researcher,\n)\n\nanalysis_task = Task(\n    description=(\n        \"Review the research findings and identify: (1) the three most significant trends, \"\n        \"(2) any contradictions or gaps, and (3) practical implications.\"\n    ),\n    expected_output=(\n        \"A structured analysis in three labelled sections: Trends, Gaps, Implications. \"\n        \"Each section contains 2-3 concise paragraphs.\"\n    ),\n    agent=analyst,\n    context=[research_task],  # analyst receives researcher's full output\n)\n\nwriting_task = Task(\n    description=(\n        \"Write a research report on {topic} using the findings and analysis provided. \"\n        \"Include: Executive Summary, Findings, Analysis, and Conclusion. \"\n        \"Cite sources inline.\"\n    ),\n    expected_output=(\n        \"A formatted Markdown report with H2 headings for each section, \"\n        \"inline citations, and a References section at the end.\"\n    ),\n    agent=writer,\n    context=[research_task, analysis_task],\n)\n```\n\nThe `context`\n\nlist is the key wiring. Without it, the analyst and writer only see their own task description, not the upstream output.\n\n## Step 6: Assemble the Crew and Run\n\n```\ncrew = Crew(\n    agents=[researcher, analyst, writer],\n    tasks=[research_task, analysis_task, writing_task],\n    process=Process.sequential,\n    verbose=True,\n)\n\nif __name__ == \"__main__\":\n    result = crew.kickoff(inputs={\"topic\": \"post-quantum cryptography standardization\"})\n    print(\"\\n=== FINAL REPORT ===\\n\")\n    print(result)\n```\n\n`kickoff(inputs=...)`\n\ninterpolates `{topic}`\n\ninto every task description at runtime. Change the topic string without touching the pipeline definition.\n\n```\npython research_pipeline.py\n```\n\nExpect 5-15 minutes depending on hardware. Each agent's reasoning steps scroll by as it works.\n\n## Verify It Works\n\nA successful run prints each agent's inner monologue, then `=== FINAL REPORT ===`\n\nfollowed by a Markdown document with H2 sections and a References block. If you see `Action: duckduckgo_search`\n\nlines in the researcher's output, tool use is working correctly.\n\nQuick check: search the report for at least one `http`\n\nURL in the References section. If there are none, the researcher completed without invoking the search tool — see the first troubleshooting item below.\n\n## Troubleshooting\n\n**Agent loops without producing a Final Answer**\nLocal models sometimes fail to emit the `Final Answer:`\n\ntoken in the ReAct format CrewAI expects. Add `max_iter=5`\n\nto the offending agent. If it persists, try `mistral:7b`\n\n— different quantizations handle ReAct prompting differently, and some work significantly better than others out of the box.\n\n**Researcher never calls the search tool**\nThe model generated an answer from its weights rather than invoking the tool. Make the task description more directive: add \"You MUST use the search tool for every fact.\" It's a prompt engineering issue, not a code bug.\n\n**DuckDuckGo raises an exception or returns empty results**\nDuckDuckGo rate-limits aggressive scrapers. If you hit it repeatedly during testing, add `time.sleep(2)`\n\nbetween runs. For production workloads, swap in `from crewai_tools import SerperDevTool`\n\n(requires a free Serper API key) which is more reliable under repeated load.\n\n**Pydantic validation errors on Agent or Task construction**\nCrewAI requires Pydantic v2. Run `pip show pydantic`\n\nand confirm `2.x`\n\n. If something in your environment pins v1, create a fresh virtual environment rather than trying to coerce compatibility.\n\n## Next Steps\n\n- Add\n`memory=True`\n\nto`Crew`\n\nwith a local embeddings model to give agents persistent memory across runs. - Swap\n`Process.sequential`\n\nfor`Process.hierarchical`\n\nand pass`manager_llm=llm`\n\nto let CrewAI dynamically assign tasks. - Explore\n`crewai_tools`\n\nfor`WebsiteSearchTool`\n\n,`PDFSearchTool`\n\n, and`FileWriterTool`\n\nto write reports directly to disk. - Profile throughput with\n`ollama ps`\n\nwhile the pipeline runs and compare`llama3.1:8b`\n\nagainst`mistral:7b`\n\non your hardware for the best speed/quality tradeoff.\n\n[Mariana Souza](https://www.devclubhouse.com/u/mariana_souza)· Senior Editor\n\nMariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon.\n\n## Discussion 0\n\nNo comments yet\n\nBe the first to weigh in.", "url": "https://wpnews.pro/news/build-a-multi-agent-research-pipeline-with-crewai-and-ollama", "canonical_source": "https://www.devclubhouse.com/a/build-a-multi-agent-research-pipeline-with-crewai-and-ollama", "published_at": "2026-06-26 07:44:55+00:00", "updated_at": "2026-06-26 08:08:21.985370+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "developer-tools"], "entities": ["CrewAI", "Ollama", "Llama 3.1", "LangChain", "DuckDuckGo"], "alternates": {"html": "https://wpnews.pro/news/build-a-multi-agent-research-pipeline-with-crewai-and-ollama", "markdown": "https://wpnews.pro/news/build-a-multi-agent-research-pipeline-with-crewai-and-ollama.md", "text": "https://wpnews.pro/news/build-a-multi-agent-research-pipeline-with-crewai-and-ollama.txt", "jsonld": "https://wpnews.pro/news/build-a-multi-agent-research-pipeline-with-crewai-and-ollama.jsonld"}}