{"slug": "multi-agent-implementing-the-orchestrator-worker-pattern", "title": "Multi-Agent — Implementing the Orchestrator Worker Pattern", "summary": "A developer implemented a multi-agent architecture using the Orchestrator Worker pattern, where an orchestrator delegates tasks to specialized workers for search and quality checking. The approach, inspired by Anthropic research showing a 90% improvement in task completion rates, replaces a single-agent system with role-specific agents to handle complex tasks more effectively.", "body_md": "Through [Chapter 6 (Fine-tuning)](https://dev.to/hiroki-kameyama/fine-tuning-domain-specializing-models-with-lora-180g), we focused on improving a single AI system. This chapter introduces **multi-agent** design, where multiple Agents collaborate on complex tasks.\n\nOur previous Agent implementations used a \"one Agent does everything\" approach. For complex tasks, putting all responsibility on a single Agent has limits.\n\n```\n[Single Agent (before)]\nUser → Agent → Tools → Answer\nOne Agent makes all decisions and executes everything\n\n[Multi-Agent (now)]\nUser → Orchestrator → Search Agent\n                    → Answer Generation Agent\n                    → Quality Check Agent\n                 → Final Answer\nEach role is specialized and they collaborate\n```\n\n**Multi-agent is effective when:**\n\n2026 Trend:Anthropic research shows multi-agent architectures improve task completion rates by 90%. The Orchestrator × Worker pattern has become the production standard.\n\n```\nUser's question\n    ↓\n[Orchestrator]\n  Analyzes task and delegates to two workers\n    ↓               ↓\n[Search Worker]    [Quality Check Worker]\nSearches pgvector  Evaluates answer quality\n    ↓               ↓\n[Orchestrator]\n  Integrates results and generates final answer\n    ↓\nFinal Answer\npgvector-tutorial/\n├── existing files\n└── multiagent/\n    ├── search_worker.py      # ★ Search specialist worker\n    ├── quality_worker.py     # ★ Quality check specialist worker\n    ├── orchestrator.py       # ★ Orchestrator\n    └── 14_multiagent.py      # ★ Execution script\n```\n\n`multiagent/search_worker.py`\n\nA worker focused exclusively on pgvector search. Single responsibility (search only) keeps the prompt simple and improves accuracy.\n\n```\n# multiagent/search_worker.py\n\"\"\"\nSearch specialist worker\n\nRole: Search and return documents from pgvector\nResponsibility: Search only (no answer generation)\n\"\"\"\nimport sys\nimport os\nsys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))\n\nimport psycopg2\nfrom google import genai\nfrom google.genai import types\nfrom dotenv import load_dotenv\nimport time\n\nload_dotenv()\n\nclient = genai.Client(api_key=os.getenv(\"GEMINI_API_KEY\"))\n\nconn = psycopg2.connect(\n    host=os.getenv(\"DB_HOST\"),\n    port=os.getenv(\"DB_PORT\"),\n    dbname=os.getenv(\"DB_NAME\"),\n    user=os.getenv(\"DB_USER\"),\n    password=os.getenv(\"DB_PASSWORD\"),\n)\ncur = conn.cursor()\n\nSEARCH_WORKER_SYSTEM = \"\"\"You are a document search specialist.\nSearch for documents related to the given question from pgvector and return the results as-is.\n\nConstraints:\n- Do not generate answers (just return search results)\n- If no results found, return \"No relevant documents found\"\n- Use category filtering when the category is clear\n\"\"\"\n\ndef get_embedding(text: str) -> list[float]:\n    result = client.models.embed_content(\n        model=\"gemini-embedding-001\",\n        contents=text,\n        config=types.EmbedContentConfig(\n            task_type=\"RETRIEVAL_QUERY\",\n            output_dimensionality=768,\n        ),\n    )\n    return result.embeddings[0].values\n\ndef search_documents(query: str, top_k: int = 3) -> list[dict]:\n    \"\"\"Search all categories.\"\"\"\n    query_embedding = get_embedding(query)\n    cur.execute(\"\"\"\n        SELECT title, body, category,\n               1 - (embedding <=> %s::vector) AS similarity\n        FROM documents\n        ORDER BY embedding <=> %s::vector\n        LIMIT %s;\n    \"\"\", (query_embedding, query_embedding, top_k))\n    rows = cur.fetchall()\n    return [\n        {\"title\": r[0], \"body\": r[1], \"category\": r[2], \"similarity\": round(r[3], 4)}\n        for r in rows\n    ]\n\ndef search_by_category(query: str, category: str, top_k: int = 3) -> list[dict]:\n    \"\"\"Category-filtered search.\"\"\"\n    query_embedding = get_embedding(query)\n    cur.execute(\"\"\"\n        SELECT title, body, category,\n               1 - (embedding <=> %s::vector) AS similarity\n        FROM documents\n        WHERE category = %s\n        ORDER BY embedding <=> %s::vector\n        LIMIT %s;\n    \"\"\", (query_embedding, category, query_embedding, top_k))\n    rows = cur.fetchall()\n    return [\n        {\"title\": r[0], \"body\": r[1], \"category\": r[2], \"similarity\": round(r[3], 4)}\n        for r in rows\n    ]\n\ndef list_categories() -> list[dict]:\n    \"\"\"Get category list.\"\"\"\n    cur.execute(\"\"\"\n        SELECT category, COUNT(*) as count\n        FROM documents\n        GROUP BY category ORDER BY count DESC;\n    \"\"\")\n    rows = cur.fetchall()\n    return [{\"category\": r[0], \"count\": r[1]} for r in rows]\n\nSEARCH_TOOLS = types.Tool(\n    function_declarations=[\n        types.FunctionDeclaration(\n            name=\"search_documents\",\n            description=\"Search all categories for documents related to a query.\",\n            parameters=types.Schema(\n                type=types.Type.OBJECT,\n                properties={\n                    \"query\": types.Schema(type=types.Type.STRING, description=\"Search query\"),\n                    \"top_k\": types.Schema(type=types.Type.INTEGER, description=\"Number of results\"),\n                },\n                required=[\"query\"],\n            ),\n        ),\n        types.FunctionDeclaration(\n            name=\"search_by_category\",\n            description=\"Search documents in a specific category only.\",\n            parameters=types.Schema(\n                type=types.Type.OBJECT,\n                properties={\n                    \"query\": types.Schema(type=types.Type.STRING),\n                    \"category\": types.Schema(type=types.Type.STRING, description=\"ML / Python / Cloud\"),\n                    \"top_k\": types.Schema(type=types.Type.INTEGER),\n                },\n                required=[\"query\", \"category\"],\n            ),\n        ),\n        types.FunctionDeclaration(\n            name=\"list_categories\",\n            description=\"Return the list of categories in the DB.\",\n            parameters=types.Schema(type=types.Type.OBJECT, properties={}),\n        ),\n    ]\n)\n\ndef dispatch(func_name: str, func_args: dict):\n    if func_name == \"search_documents\":\n        return search_documents(**func_args)\n    elif func_name == \"search_by_category\":\n        return search_by_category(**func_args)\n    elif func_name == \"list_categories\":\n        return list_categories()\n    return {\"error\": f\"unknown: {func_name}\"}\n\ndef run_search_worker(question: str) -> dict:\n    \"\"\"\n    Main function for the search worker.\n\n    Returns:\n        {\n            \"docs\": [list of retrieved documents],\n            \"steps\": [number of steps executed],\n            \"worker\": \"search\"\n        }\n    \"\"\"\n    print(f\"  [Search Worker] Question: {question[:40]}...\")\n\n    contents = [\n        types.Content(role=\"user\", parts=[types.Part(text=question)])\n    ]\n\n    docs = []\n    steps = 0\n\n    for _ in range(5):\n        for attempt in range(3):\n            try:\n                response = client.models.generate_content(\n                    model=\"gemini-2.5-flash\",\n                    contents=contents,\n                    config=types.GenerateContentConfig(\n                        system_instruction=SEARCH_WORKER_SYSTEM,\n                        tools=[SEARCH_TOOLS],\n                    ),\n                )\n                break\n            except Exception as e:\n                if (\"503\" in str(e) or \"429\" in str(e)) and attempt < 2:\n                    time.sleep((attempt + 1) * 10)\n                else:\n                    raise\n\n        candidates = response.candidates\n        if not candidates or not candidates[0].content.parts:\n            break\n\n        part = candidates[0].content.parts[0]\n\n        if part.function_call:\n            func_name = part.function_call.name\n            func_args = dict(part.function_call.args)\n            result = dispatch(func_name, func_args)\n            steps += 1\n            print(f\"  [Search Worker] {func_name} → {len(result) if isinstance(result, list) else result} results\")\n\n            if func_name in (\"search_documents\", \"search_by_category\") and isinstance(result, list):\n                docs.extend(result)\n\n            contents.append(\n                types.Content(role=\"model\", parts=[types.Part(function_call=part.function_call)])\n            )\n            contents.append(\n                types.Content(\n                    role=\"user\",\n                    parts=[types.Part(\n                        function_response=types.FunctionResponse(\n                            name=func_name,\n                            response={\"result\": result},\n                        )\n                    )]\n                )\n            )\n        else:\n            break\n\n    # Deduplicate by title\n    seen = set()\n    unique_docs = []\n    for doc in docs:\n        if doc[\"title\"] not in seen:\n            seen.add(doc[\"title\"])\n            unique_docs.append(doc)\n\n    print(f\"  [Search Worker] Done: retrieved {len(unique_docs)} documents\")\n    return {\"docs\": unique_docs, \"steps\": steps, \"worker\": \"search\"}\n```\n\n`multiagent/quality_worker.py`\n\nA worker focused exclusively on evaluating answer quality.\n\n```\n# multiagent/quality_worker.py\n\"\"\"\nQuality check specialist worker\n\nRole: Evaluate the quality of generated answers\nResponsibility: Quality evaluation only (no search or answer generation)\n\"\"\"\nimport sys\nimport os\nsys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))\n\nfrom google import genai\nfrom google.genai import types\nfrom dotenv import load_dotenv\nimport time\nimport json\nimport re\n\nload_dotenv()\n\nclient = genai.Client(api_key=os.getenv(\"GEMINI_API_KEY\"))\n\nQUALITY_WORKER_SYSTEM = \"\"\"You are a quality evaluation specialist for AI answers.\nEvaluate the given \"question, reference documents, and answer\" set.\n\nEvaluation criteria:\n1. Faithfulness (0.0–1.0): Is the answer based on the documents?\n2. Relevancy (0.0–1.0): Does it correctly answer the question?\n3. Completeness (0.0–1.0): Does it include all necessary information?\n\nReturn ONLY the following JSON format:\n{\n  \"faithfulness\": 0.0–1.0,\n  \"relevancy\": 0.0–1.0,\n  \"completeness\": 0.0–1.0,\n  \"overall\": 0.0–1.0,\n  \"feedback\": \"Note any areas for improvement\"\n}\n\"\"\"\n\ndef run_quality_worker(question: str, docs: list[dict], answer: str) -> dict:\n    \"\"\"\n    Main function for the quality check worker.\n\n    Returns:\n        {\n            \"faithfulness\": float,\n            \"relevancy\": float,\n            \"completeness\": float,\n            \"overall\": float,\n            \"feedback\": str,\n            \"worker\": \"quality\"\n        }\n    \"\"\"\n    print(f\"  [Quality Worker] Evaluating answer quality...\")\n\n    context = \"\\n\\n\".join([f\"[{d['title']}]\\n{d['body']}\" for d in docs])\n\n    prompt = f\"\"\"Please evaluate the following.\n\n# Question\n{question}\n\n# Reference Documents\n{context}\n\n# Answer to Evaluate\n{answer}\n\nReturn the evaluation result in JSON format.\"\"\"\n\n    for attempt in range(3):\n        try:\n            response = client.models.generate_content(\n                model=\"gemini-2.5-flash\",\n                contents=prompt,\n                config=types.GenerateContentConfig(\n                    system_instruction=QUALITY_WORKER_SYSTEM,\n                ),\n            )\n            break\n        except Exception as e:\n            if (\"503\" in str(e) or \"429\" in str(e)) and attempt < 2:\n                time.sleep((attempt + 1) * 10)\n            else:\n                raise\n\n    raw = response.text.strip()\n    match = re.search(r'\\{.*\\}', raw, re.DOTALL)\n    if match:\n        try:\n            result = json.loads(match.group())\n            result[\"worker\"] = \"quality\"\n            print(f\"  [Quality Worker] Overall: {result.get('overall', 'N/A')}\")\n            return result\n        except json.JSONDecodeError:\n            pass\n\n    print(f\"  [Quality Worker] Parse failed, using default values\")\n    return {\n        \"faithfulness\": 0.5,\n        \"relevancy\": 0.5,\n        \"completeness\": 0.5,\n        \"overall\": 0.5,\n        \"feedback\": \"Could not retrieve evaluation\",\n        \"worker\": \"quality\",\n    }\n```\n\n`multiagent/orchestrator.py`\n\nThe command center that coordinates the two workers.\n\n```\n# multiagent/orchestrator.py\n\"\"\"\nOrchestrator\n\nRole: Receive user questions, direct workers, and integrate results\nResponsibility: Task decomposition, worker invocation, result integration\n\"\"\"\nimport sys\nimport os\nsys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))\n\nfrom google import genai\nfrom google.genai import types\nfrom dotenv import load_dotenv\nfrom multiagent.search_worker import run_search_worker\nfrom multiagent.quality_worker import run_quality_worker\nimport time\n\nload_dotenv()\n\nclient = genai.Client(api_key=os.getenv(\"GEMINI_API_KEY\"))\n\nORCHESTRATOR_SYSTEM = \"\"\"You are an orchestrator that decomposes tasks and directs multiple workers.\n\nYour role:\n1. Receive the user's question\n2. Generate an answer based on documents retrieved by the search worker\n3. Review quality check results and improve the answer if needed\n4. Return the final answer to the user\n\nConstraints:\n- Base answers on the reference documents\n- Improve the answer if quality score is below 0.7\n- Aim for concise and accurate answers\n\"\"\"\n\ndef generate_answer(question: str, docs: list[dict]) -> str:\n    time.sleep(15)  # Rate limit safety (5 requests/min free tier)\n    context = \"\\n\\n\".join([f\"[{d['title']}]\\n{d['body']}\" for d in docs])\n\n    prompt = f\"\"\"Answer the question based on the following documents.\n\n# Reference Documents\n{context}\n\n# Question\n{question}\n\n# Answer (concisely, based on the documents)\"\"\"\n\n    for attempt in range(3):\n        try:\n            response = client.models.generate_content(\n                model=\"gemini-2.5-flash\",\n                contents=prompt,\n                config=types.GenerateContentConfig(\n                    system_instruction=ORCHESTRATOR_SYSTEM,\n                ),\n            )\n            return response.text\n        except Exception as e:\n            if (\"503\" in str(e) or \"429\" in str(e)) and attempt < 2:\n                time.sleep((attempt + 1) * 10)\n            else:\n                raise\n    return \"Could not generate an answer.\"\n\ndef improve_answer(question: str, docs: list[dict], answer: str, feedback: str) -> str:\n    context = \"\\n\\n\".join([f\"[{d['title']}]\\n{d['body']}\" for d in docs])\n\n    prompt = f\"\"\"Please improve the following answer.\n\n# Question\n{question}\n\n# Reference Documents\n{context}\n\n# Current Answer\n{answer}\n\n# Improvement Feedback\n{feedback}\n\n# Improved Answer\"\"\"\n\n    for attempt in range(3):\n        try:\n            response = client.models.generate_content(\n                model=\"gemini-2.5-flash\",\n                contents=prompt,\n            )\n            return response.text\n        except Exception as e:\n            if (\"503\" in str(e) or \"429\" in str(e)) and attempt < 2:\n                time.sleep((attempt + 1) * 10)\n            else:\n                raise\n    return answer\n\ndef run_orchestrator(question: str) -> dict:\n    \"\"\"\n    Main orchestrator function.\n\n    Flow:\n    1. Request search from search worker\n    2. Generate answer from search results\n    3. Request evaluation from quality check worker\n    4. Improve answer if score is low\n    5. Return final answer\n    \"\"\"\n    print(f\"\\n{'='*60}\")\n    print(f\"Orchestrator started\")\n    print(f\"Question: {question}\")\n    print(f\"{'='*60}\")\n\n    # ── Step 1: Request search from search worker ─────────────\n    print(\"\\n[Step 1] Delegating to search worker...\")\n    search_result = run_search_worker(question)\n    docs = search_result[\"docs\"]\n\n    if not docs:\n        return {\n            \"answer\": \"No relevant documents found.\",\n            \"quality\": {\"overall\": 0.0},\n            \"docs_count\": 0,\n            \"improved\": False,\n        }\n\n    print(f\"  → Retrieved {len(docs)} documents\")\n\n    # ── Step 2: Generate answer ───────────────────────────────\n    print(\"\\n[Step 2] Generating answer...\")\n    answer = generate_answer(question, docs)\n    print(f\"  → Answer generated ({len(answer)} chars)\")\n\n    # ── Step 3: Request evaluation from quality worker ────────\n    print(\"\\n[Step 3] Delegating to quality check worker...\")\n    quality = run_quality_worker(question, docs, answer)\n\n    improved = False\n\n    # ── Step 4: Improve if quality is low ────────────────────\n    if quality.get(\"overall\", 0) < 0.7:\n        print(f\"\\n[Step 4] Quality score {quality.get('overall')} < 0.7 → Improving answer...\")\n        feedback = quality.get(\"feedback\", \"Please improve the answer\")\n        answer = improve_answer(question, docs, answer, feedback)\n        improved = True\n        print(f\"  → Answer improvement complete\")\n    else:\n        print(f\"\\n[Step 4] Quality score {quality.get('overall')} ≥ 0.7 → No improvement needed\")\n\n    print(f\"\\n{'='*60}\")\n    print(\"Orchestrator complete\")\n    print(f\"{'='*60}\")\n\n    return {\n        \"answer\": answer,\n        \"quality\": quality,\n        \"docs_count\": len(docs),\n        \"improved\": improved,\n    }\n```\n\n`14_multiagent.py`\n\n``` python\n# 14_multiagent.py\nimport time\nimport sys\nimport os\nsys.path.append(os.path.dirname(os.path.abspath(__file__)))\n\nfrom multiagent.orchestrator import run_orchestrator\n\ndef main():\n    questions = [\n        \"Tell me in detail about ML evaluation metrics\",\n        \"What are the methods for reducing AWS costs?\",\n    ]\n\n    for question in questions:\n        result = run_orchestrator(question)\n\n        print(f\"\\nFinal Answer:\")\n        print(result[\"answer\"])\n        print(f\"\\nQuality Scores:\")\n        q = result[\"quality\"]\n        print(f\"  Faithfulness:  {q.get('faithfulness', 'N/A')}\")\n        print(f\"  Relevancy:     {q.get('relevancy', 'N/A')}\")\n        print(f\"  Completeness:  {q.get('completeness', 'N/A')}\")\n        print(f\"  Overall:       {q.get('overall', 'N/A')}\")\n        print(f\"  Improved:      {'Yes' if result['improved'] else 'No'}\")\n        print(f\"  Docs used:     {result['docs_count']}\")\n        print(\"\\n\" + \"=\"*60)\n        time.sleep(30)  # 30s between questions (rate limit safety)\n\nif __name__ == \"__main__\":\n    main()\nmkdir multiagent\ntouch multiagent/__init__.py\npython 14_multiagent.py\n```\n\nSample output:\n\n```\n============================================================\nOrchestrator started\nQuestion: Tell me in detail about ML evaluation metrics\n============================================================\n\n[Step 1] Delegating to search worker...\n  [Search Worker] Question: Tell me in detail about ML ev...\n  [Search Worker] list_categories → 3 results\n  [Search Worker] search_by_category → 2 results\n  [Search Worker] Done: retrieved 2 documents\n\n[Step 2] Generating answer...\n  → Answer generated (324 chars)\n\n[Step 3] Delegating to quality check worker...\n  [Quality Worker] Evaluating answer quality...\n  [Quality Worker] Overall: 0.92\n\n[Step 4] Quality score 0.92 ≥ 0.7 → No improvement needed\n============================================================\n```\n\n| Pattern | Structure | Best For |\n|---|---|---|\nOrchestrator × Worker (this chapter) |\n1 command center + multiple specialists | General task decomposition |\nSequential Pipeline |\nAgent A → Agent B → Agent C | Document processing, contract generation |\nParallel Fan-out |\nMultiple Agents execute simultaneously | High-speed research, comparison analysis |\nHierarchical |\nUpper orchestrator → Lower orchestrator → Workers | Large-scale complex tasks |\n\n**Watch out for cost explosions**\n\nWhen the orchestrator receives full results from all workers, the context window balloons. Always have workers return structured JSON summaries — never pass the full text.\n\n**Set step limits**\n\nAlways configure a `max_steps`\n\nlimit on each Agent to prevent runaway loops.\n\n**Error handling**\n\nUse `try/except`\n\nper worker so that one worker's failure doesn't halt the entire system.\n\n| Error | Cause | Fix |\n|---|---|---|\n`ModuleNotFoundError: multiagent` |\nMissing `__init__.py`\n|\nRun `touch multiagent/__init__.py`\n|\n`NameError: name 'time' is not defined` |\nMissing `import time`\n|\nAdd `import time` at top of `14_multiagent.py`\n|\n`429 RESOURCE_EXHAUSTED` (per-minute) |\nGemini free tier: 5 req/min | Add `time.sleep(15)` at start of `generate_answer()`\n|\n`429 RESOURCE_EXHAUSTED` (daily) |\nGemini free tier: 20 req/day | Re-run the next day |\n| Quality worker JSON parse failure | LLM returns malformed JSON | Extract JSON block with `re.search`\n|\n| Cost overrun | Sending full text to workers | Pass summaries / structured JSON |\n\n`asyncio.gather()`\n\nto parallelize the search and quality workers for speed", "url": "https://wpnews.pro/news/multi-agent-implementing-the-orchestrator-worker-pattern", "canonical_source": "https://dev.to/hiroki-kameyama/multi-agent-implementing-the-orchestrator-x-worker-pattern-38gd", "published_at": "2026-07-04 12:01:58+00:00", "updated_at": "2026-07-04 12:18:58.575540+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research", "developer-tools"], "entities": ["Anthropic", "pgvector", "Gemini", "Google", "LoRA"], "alternates": {"html": "https://wpnews.pro/news/multi-agent-implementing-the-orchestrator-worker-pattern", "markdown": "https://wpnews.pro/news/multi-agent-implementing-the-orchestrator-worker-pattern.md", "text": "https://wpnews.pro/news/multi-agent-implementing-the-orchestrator-worker-pattern.txt", "jsonld": "https://wpnews.pro/news/multi-agent-implementing-the-orchestrator-worker-pattern.jsonld"}}