{"slug": "agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed", "title": "Agentic RAG vs. Plain Hybrid RAG: What My Hydropower Benchmark Revealed", "summary": "A locally run Agentic RAG system built for Small Hydropower Plant (SHP) engineering improved retrieval precision over plain hybrid RAG but carried a substantial latency cost and did not automatically outperform a well-designed plain RAG pipeline on faithfulness, according to the developer's published evaluation. The implementation, posted to GitHub by NaimishKB, runs its core pipeline on Ollama with open-weight models so document processing, embeddings, retrieval, answer generation and runtime quality checks require no paid cloud inference APIs; Gemini and Groq free tiers were used only for offline benchmarking against a golden dataset. Document ingestion routes files through a multi-tier parser that converts PDFs to Markdown with pymupdf4llm and accepts output of at least 80 characters before falling back to Docling with OCR and table-structure recovery for scanned or failed PDFs.", "body_md": "Most RAG applications follow a simple path:\n\nRetrieve a few documents → send them to an LLM → generate an answer.\n\nThat works for straightforward lookup questions. But it becomes fragile when the question spans multiple manuals, needs numerical reasoning, contains tables, or asks for an engineering recommendation grounded in plant documentation.\n\nI built an Agentic RAG system for Small Hydropower Plant (SHP) engineering to explore a harder question:\n\nCan a multi-agent retrieval workflow improve the quality of answers from private technical documents without depending on paid inference APIs?\n\nThe answer, based on my evaluation so far, is nuanced: it improves retrieval precision, but comes with a substantial latency cost and does not automatically outperform a well-designed plain RAG pipeline on faithfulness.\n\n**The complete implementation is available here:** [**Agentic RAG for Small Hydropower**](https://github.com/NaimishKB/Agentic_RAG)**.**\n\nOne of my goals was to make this project accessible to students and independent learners who cannot afford recurring API bills.\n\nThe core RAG pipeline runs locally using Ollama and open-weight models. Document processing, embeddings, retrieval, answer generation, and runtime quality checks do not require paid cloud inference APIs. Once the required models and dependencies are downloaded, you can experiment on compatible hardware without paying per question or per token.\n\nFor offline benchmarking, I used Gemini and Groq through their available free tiers to evaluate answers against the golden dataset. Those services are optional: evaluation can also use a local Ollama judge. Free-tier quotas and availability may change, so they should not be treated as an unlimited resource.\n\nThis does not mean running the system has zero cost — hardware, electricity, and inference time still matter. But if you already have a suitable computer, it removes a major barrier to learning: a recurring inference bill.\n\nFor me, that makes the project useful beyond hydropower. It provides a practical starting point for learning document ingestion, hybrid retrieval, agent orchestration, tool use, and evaluation within a limited budget.\n\nSmall hydropower projects involve a mix of technical manuals, regulatory guidelines, turbine-selection references, hydrology documents, financial policies, scanned reports, tables, and drawings.\n\nA typical question may require several kinds of evidence at once:\n\nA traditional RAG pipeline may retrieve relevant passages, but it usually has no mechanism to decide whether the retrieved context is sufficient, whether a calculation should use a tool, or whether the generated answer contains unsupported claims.\n\nThat is where the agentic approach becomes useful.\n\nThe first challenge was preparing the documents. Hydropower knowledge arrives in different formats: digital PDFs, scanned reports, Word documents, presentations, spreadsheets, and images. Each format needs a different extraction approach.\n\nI built a multi-tier parsing pipeline that converts these files into Markdown before chunking and embedding.\n\nFor formats with accessible document structure, the pipeline uses dedicated libraries:\n\nThis avoids sending every file through a more expensive OCR process.\n\nFor PDFs, the pipeline first uses pymupdf4llm to extract text into Markdown, aiming to retain headings and table structure.\n\nIf the extracted output contains at least 80 characters, the current implementation accepts it. If the output is shorter — or extraction fails — the document moves to the Docling fallback.\n\nThis is a practical routing heuristic, but it has a limitation: a PDF containing both digital text and scanned pages may pass the threshold even when some pages need OCR. Page-level quality checks would improve this approach.\n\nScanned PDFs and failed PDF extractions are processed with Docling, with OCR and table-structure recovery enabled when the required backend is available. Images also enter the Docling conversion path.\n\nThe PDF OCR configuration prefers EasyOCR, with RapidOCR as an alternative. It uses CUDA acceleration when available and otherwise falls back to CPU execution.\n\nThe aim is to recover readable text and useful table structure. Complex drawings, dense tables, and formulas remain difficult, so extracted Markdown should not be assumed to reproduce every detail correctly.\n\nAlongside the Markdown, the parser records source information such as the filename, file type, content hash, parsing duration, and page count where available. Parsing failures are recorded explicitly.\n\nThe content hash also supports caching: an unchanged document can reuse its existing Markdown output, avoiding unnecessary processing.\n\nThe resulting flow is:\n\n**Raw documents → format-specific parsing or OCR → Markdown and metadata → chunking → embeddings and indexing**\n\nThis stage directly affects everything that follows. If a table loses its units or OCR misreads a formula, the retrieval and generation stages may work with incorrect evidence. Improving parsing is therefore part of improving answer quality.\n\nBinary SHA-256 deduplication is an efficient first line of defense to prevent wasting heavy parsing and OCR compute on identical files, but for the same converted file, Binary deduplication **will NOT be applicable** — the system will treat them as two completely separate files\n\n```\nAn actual record saved in data/parsed/metadata.json{  \"source_path\": \"D:\\\\GenAi_Code\\\\Agentic_RAG\\\\data\\\\raw\\\\AK - Hydrology of Ungauged Catchment.pdf\",  \"file_name\": \"AK - Hydrology of Ungauged Catchment.pdf\",  \"file_type\": \"PDF\",  \"file_hash\": \"127653ce9e3151fd6732fd83bfc66e570ab762bdafb22d568fcef524ce0d19bc\",  \"page_count\": 0,  \"parsed_at\": 1793168584.5575593,  \"parse_duration_s\": 0.0,  \"parsed_md_path\": \"D:\\\\GenAi_Code\\\\Agentic_RAG\\\\data\\\\parsed\\\\AK - Hydrology of Ungauged Catchment__127653ce.md\",  \"error\": null}\n```\n\nAfter parsing, the documents are available as Markdown. The next challenge is dividing them into passages that are useful for retrieval without losing their context. I implemented a Markdown-aware hierarchical chunker. It uses document structure and text length for boundaries, rather than an LLM or embedding-based semantic segmentation model.\n\nThe chunker uses a **4-pass rule-based pipeline** rather than an LLM or arbitrary character slicing:\n\n```\n[Raw Markdown Document]       │       ▼   ┌─────────┐   │ Pass 1  │  Scan whole document → Divide into Structural Sections   └─────────┘       │  Output: `sections` list       ▼   ┌─────────┐   │ Pass 2  │  Process all sections → Split into Target-Sized Chunks (1,800 chars)   └─────────┘       │  Output: `raw_chunks` list       ▼   ┌─────────┐   │ Pass 3  │  Scan all raw chunks → Merge Tiny Leftover Fragments (< 150 chars)   └─────────┘       │  Output: `merged_chunks` list       ▼   ┌─────────┐   │ Pass 4  │  Enrich all chunks → Inject Citation Banners & Compute Token Counts   └─────────┘       │  Output: `final_chunks` list (Written to `chunks.jsonl`)       ▼[Final Output]\n```\n\nInstead of chopping text blindly every NNN characters, the chunker walks through the document line-by-line, tracking markdown headings (#, ##, ###, ####) like a tree or outline. Whenever a heading or a horizontal divider (---) appears, it finishes the current section. It also checks if the heading mentions a slide or page number.\n\nWhen a heading includes a label such as “Slide 12” or “Page 8,” that location is captured as metadata. Page references therefore depend on what the parsing stage preserves; they are not automatically available for every chunk.\n\nWithin a section, long text is divided at paragraph boundaries. Oversized paragraphs are further split at sentence boundaries.\n\n**In simple words:** If a text section is too long (>1,800 characters):\n\nThe default settings are:\n\nThe overlap is conditional, rather than a fixed sliding window: the previous paragraph is repeated only when it fits within the overlap allowance.\n\nThe 1,800-character setting is a target, not a strict maximum. Long individual sentences, table rows, fragment merging, and added source metadata can produce larger chunks.\n\n```\nmax_chars = 1800overlap_chars = 150Imagine a document section with four paragraphs:Paragraph 1 (900 chars): Details about Francis turbine runner design.Paragraph 2 (800 chars): Details about guide vanes and water flow.Paragraph 3 (85 chars): \"Cavitation occurs when local water pressure drops below vapor pressure.\"Paragraph 4 (600 chars): Explaining preventative coatings and blade maintenance.What happens:1. Building Chunk 1:  Adds Para 1 (900 chars) + Para 2 (800 chars) + Para 3 (85 chars) = 1,785 chars.  Current chunk is full (≈1800).2. Along comes Paragraph 4 (600 chars):  1,785+600=2,385 chars (exceeds 1,800!).  So Chunk 1 is saved containing: [Para 1 + Para 2 + Para 3].3. Now, how should Chunk 2 start?  The code looks at the last paragraph of Chunk 1 (Paragraph 3).  Paragraph 3 is 85 chars long.  Is 85≤150 (overlap_chars)? YES!  Because Paragraph 3 is a short, complete sentence, the code carries it over as a natural bridge into Chunk 2:Chunk 2 begins with:\"Cavitation occurs when local water pressure drops below vapor pressure.[Paragraph 4 text about coatings and blade maintenance...]\"  Result: Chunk 2 now has immediate context on why coatings are needed, without cutting any words or sentences in half!What if Paragraph 3 had been huge (e.g. 600 chars)?  If Paragraph 3 had been 600 chars, len(current_chunk[-1]) <= 150 would be False.  Repeating a massive 600-character block would waste one-third of Chunk 2's capacity.  The condition fails, so Chunk 2 starts cleanly with Paragraph 4 alone.\n```\n\nTables need different handling from narrative text. In standard chunkers, a long table gets sliced across arbitrary character limits, leaving half the data rows orphaned without column names. This chunker: If a table has 40 rows and exceeds 1,800 characters, it slices it into smaller tables, repeating the top column headers on every slice:\n\nWhen an embedding model or LLM sees an isolated chunk like *“The operating head is 45 meters”*, it doesn’t know which plant or turbine that applies to. To fix this, the chunker injects a **context banner** directly at the top of the chunk’s text:\n\n[Source: Turbine_Manual.pdf | Location: Slide/Page 12 | Section: Turbine Manual > Selection Guidelines > Francis Turbines]\n\nThe operating head is 45 meters...\n\n```\nIn Pass 2:The chunker fills Chunk 1 up to the target limit (∼1,800 characters).There are 50 characters left over at the end of the section: example:\"Refer to Figure 4 for full assembly diagrams.\" (49 chars)Pass 2 splits this leftover into Chunk 2.The Problem:If left alone, Chunk 2 would be indexed into Qdrant as its own chunk with only 49 characters! If a user searches for something, retrieving that tiny chunk alone is useless.Pass 3 to the rescue:Pass 3 looks at Chunk 2:  Size = 49 characters.  Is 49<150 (min_chars)? Yes!  It appends that 49-character sentence back into Chunk 1.Result: Chunk 1 becomes 1,849 characters (which is completely fine because 1,800 is a soft target), and the tiny orphan chunk is eliminated!\n```\n\nBoth the raw content and the banner-enriched content are saved.\n\nThese counts help inspect chunk sizes, but they are not exact token counts for the Qwen embedding model because the tokenizers differ.\n\nThe goal is to create passages that remain understandable after retrieval: a paragraph carries its section context, a table slice retains its headers, and each result can be traced back to its source.\n\n```\n# 1. Build the citation context bannerbanner_parts = [    f\"Source: {source_file}\",                    # \"HDP_AHN511.ppt\"    f\"Location: Slide/Page {page_or_slide}\",     # \"Location: Slide/Page 1\"    f\"Section: {breadcrumb}\"                     # \"Section: HDP_AHN511 > Slide 1: Dynamic Pricing...\"]banner = f\"[{' | '.join(banner_parts)}]\\n\\n\"# 2. Prepend banner to the raw textenriched_content = banner + raw_text# 3. Generate structured chunk IDchunk_id = f\"{file_stem}__{file_hash[:8]}__c{idx:04d}\"# Output -> \"HDP_AHN511__9fe0f668__c0002\"The chunk is written as a single-line JSON record:{  \"chunk_id\": \"HDP_AHN511__9fe0f668__c0002\",  \"source_file\": \"HDP_AHN511.ppt\",  \"file_type\": \"PowerPoint (legacy)\",  \"document_title\": \"HDP_AHN511\",  \"breadcrumb\": \"HDP_AHN511 > Slide 1: Dynamic Pricing of electrical power\",  \"section_hierarchy\": [    \"HDP_AHN511\",    \"Slide 1: Dynamic Pricing of electrical power\"  ],  \"page_or_slide\": 1,  \"chunk_index\": 2,  \"total_chunks\": 37,  \"is_table\": false,  \"char_count\": 531,  \"token_count\": 121,  \"content\": \"[Source: HDP_AHN511.ppt | Location: Slide/Page 1 | Section: HDP_AHN511 > Slide 1: Dynamic Pricing of electrical power]\\n\\n## Slide 1: Dynamic Pricing of electrical power\\nBased On UI Mechanism \\n(Availability Base Tariff)\\n\\n- Submitted to :-                                   Prepared By :-\\n- Prof.Arunkumar                              Harshad D.Patel\\n- AHEC                                                Research scholar\\n- IIT Roorkee\\t                                       Enroll.No.16901007\\n\\n> **[Slide 1 Image/Diagram OCR]**: HETT\",  \"raw_content\": \"## Slide 1: Dynamic Pricing of electrical power\\nBased On UI Mechanism...\"}\n```\n\nAfter chunking, each passage is embedded locally using qwen3-embedding:0.6b. The indexer expects a 1,024-dimensional vector and stores it in a Qdrant collection named shp_knowledge. Every chunk is represented by a list of 1,024 numbers.\n\nEach stored point has three parts:\n\n```\npoint_uuid = manager.chunk_id_to_uuid(\"HDP_AHN511__9fe0f668__c0002\")# Returns deterministic UUID5: \"877250c9-fe3a-55c0-96a9-89d7d65a689a\"\n{  \"chunk_id\": \"HDP_AHN511__9fe0f668__c0002\",  \"source_file\": \"HDP_AHN511.ppt\",  ...  \"embedding\": [    0.002478, 0.005229, -0.005374, -0.025804, 0.025764,     ... 1024 float numbers total ...  ]}\npoint = models.PointStruct(    # 1. Unique deterministic ID    id=\"877250c9-fe3a-55c0-96a9-89d7d65a689a\",    # 2. The 1024-dim vector for Cosine similarity search    vector=[0.002478, 0.005229, -0.005374, ...],    # 3. Payload: Metadata + Context text for LLM retrieval and filtering    payload={        \"chunk_id\": \"HDP_AHN511__9fe0f668__c0002\",        \"source_file\": \"HDP_AHN511.ppt\",        \"file_type\": \"PowerPoint (legacy)\",        \"is_table\": \"false\",        \"page_or_slide\": 1,        \"breadcrumb\": \"HDP_AHN511 > Slide 1: Dynamic Pricing of electrical power\",        \"section_hierarchy\": [\"HDP_AHN511\", \"Slide 1: Dynamic Pricing of electrical power\"],        \"token_count\": 121,        \"char_count\": 531,        \"content\": \"[Source: HDP_AHN511.ppt | Location: Slide/Page 1 | Section: ...]\\n\\n## Slide 1: ...\",        \"text\": \"[Source: HDP_AHN511.ppt | Location: Slide/Page 1 | Section: ...]\\n\\n## Slide 1: ...\"    })\n```\n\nThe embedding helps locate passages with similar meaning. The payload lets the application read the evidence, apply filters, and identify where it came from.\n\nMetadata is assembled during parsing and chunking, then carried into the index. It is derived from filenames, document structure, extracted location labels, and chunk statistics rather than generated by an LLM.\n\nThis metadata helps the agents retrieve more selectively. A request for a discharge table can use a table filter, while a document-specific question can constrain retrieval to a source file.\n\nThe indexer reads embedded chunks from JSONL, checks that each vector has the expected dimensions, and uploads points in batches of 128 by default.\n\nIt converts each chunk ID into a deterministic UUID5 identifier. Re-indexing the same chunk updates the corresponding point instead of creating another copy.\n\nThat handles repeated indexing of unchanged chunks. Updating a document is a separate concern: changed content can produce new chunk IDs, so removing obsolete document versions requires an explicit cleanup strategy.\n\nThe collection uses cosine similarity to compare query vectors with stored chunk vectors. Its configuration requests on-disk vectors and payloads, and the indexer creates payload indexes for source_file, file_type, is_table, and page_or_slide.\n\nQdrant fits three requirements of this project.\n\n**Local deployment.** The application can connect to a Qdrant server inside the organization’s infrastructure. The index manager also supports embedded local disk storage for development, reducing setup requirements for learners.\n\n**Similarity search with metadata filters.** Engineering questions often need both semantic matching and precise constraints. Keeping vectors and source metadata together allows the retriever to combine those requirements.\n\n**A straightforward indexing workflow.** Collections, payloads, deterministic point IDs, and batch upserts map naturally to the project’s document-and-chunk model.\n\nThese were practical reasons for choosing Qdrant; ChromaDB was also a viable option for local development and metadata-based retrieval. I chose Qdrant for its dedicated vector-search service and configurable payload indexes, which fit my document-filtering requirements. This was an architectural choice, rather than a claim that Qdrant universally outperforms ChromaDB. The project does not benchmark it against other vector databases.\n\nQdrant provides the dense retrieval component. A separate local BM25 retriever handles keyword matching, and the application combines their rankings using Reciprocal Rank Fusion.\n\nDense retrieval helps with conceptual similarity. BM25 helps with exact terminology, project names, and document identifiers. Combining the two gives the system complementary ways to find evidence before synthesis and answer generation.\n\nThe system is designed to run locally with Ollama and Qdrant, making it suitable for document sets that should remain inside an organization.\n\nThe high-level workflow is:\n\n```\nHere is Deatailed archietcture:                                      User Query                                  │                                  ▼                         ┌─────────────────┐                         │  Planner Agent  │◄──────────────────────────────────┐                         └────────┬────────┘                                   │                ┌─────────────────┼──────────────────┐                         │   (direct_answer)          (out_of_scope)     (rag_retrieval)                 │        │                         │                  │                         │       END                        │                  ▼                         │                                  │        ┌───────────────────────┐           │                                  │        │  Orchestrator Agent   │           │                                  │        └───────────┬───────────┘           │                                  │                    │ Send() API            │                                  │   ┌────────────────┼────────────────┐      │                                  │   ▼                ▼                ▼      │                                  │ [Narrative]    [Tabular]         [MCP]     │                                  │ (Manuals)       (Tables)       (Formulas)  │                                  │   │                │                │      │                                  │   └────────────────┼────────────────┘      │                                  │                    ▼                       │                                  │        ┌───────────────────────┐           │                                  │        │   Aggregate Results   │           │                                  │        └───────────┬───────────┘           │                                  │                    ▼                       │                                  │        ┌───────────────────────┐           │                                  │        │   Synthesizer Agent   │           │                                  │        └───────────┬───────────┘           │                                  │                    ▼                       │                                  │        ┌───────────────────────┐           │                                  │        │   Retrieval Grader    │           │                                  │        │  (Gate 1: Pre-Gen)    │           │                                  │        └───────────┬───────────┘           │                                  │                    │                       │                                  │   (Insufficient    │                       │                                  │    & retry < max)  │    (Sufficient or     |                                  |                    |Insufficient&retry>=max|                                         │          ┌─────────┴─────────┐             │                                  │          │                   ▼             │                                  │          │         ┌───────────────────┐   │                                  │          │         │  Generator Agent  │   │                                  │          │         └─────────┬─────────┘   │                                  │          │                   ▼             │                                  │          │         ┌───────────────────┐   │                                  │          │         │Generation Guardail│   │                                  │          │         │(Gate 2: Post-Gen) │   │                                  │          │         └─────────┬─────────┘   │                                  │          │                   │             │                                  │          │   (Grounded)      │             │                                  │          │       ┌───────────┴─────────┐   │                                  │          │       ▼                     │   │                                  │          │      END                    │   │                                  │          │                (Ungrounded  |   |                                       │          │               & retry < max)|   |                                       │          ▼                             ▼   │                                  │    ┌───────────────────────────────────────┴┐                                  │    │             Feedback Agent             │                                  │    │  (Diagnoses query gaps & missing terms)│                                  │    └────────────────────────────────────────┘                                  │                                  │ (Ungrounded & retry >= max)                                  ▼                    ┌────────────────────────────┐                    │       Fallback Agent       │                    │      (Circuit Breaker)     │                    │ - Out-of-scope refusal     │                    │ - Partial answer + warning │                    └─────────────┬──────────────┘                                  ▼                                 END\n```\n\nEach stage has a focused responsibility.\n\nThe Planner Agent first classifies the request into one of three routes:\n\nFor engineering questions, it elaborates domain terms and decomposes broad queries into focused sub-queries with targeted metadata filters (e.g., tagging a discharge inquiry with is_table: true).\n\nThe Orchestrator dispatches sub-queries to specialized workers using LangGraph’s dynamic Send() API:\n\nYou can add multiple workers as you want. For document retrieval, the system combines **dense semantic search (Qdrant)** and **sparse keyword search (BM25)** using **Reciprocal Rank Fusion (RRF)**. Exact equipment designations, Indian Standards (IS codes), and plant names receive exact-match boosting.\n\nThe Synthesizer performs deterministic evidence assembly without an additional LLM call. It gathers worker outputs, removes duplicate chunks, balances evidence across sub-queries, and constructs a context dossier with source references for the generator. This stage still takes processing time, but avoids the inference overhead of another model call.\n\nTo improve answer reliability, the system implements two independent verification gates:\n\nTo make the system practical for plant engineers and operators, the user-facing interface is completely decoupled from the underlying agent execution engine.\n\n```\n  ┌────────────────────────────────────────────────────────┐  │              Streamlit Web Frontend (Port 8601)        │  │  [Engineering Chat]  [Hydro Calculator]  [Citations]   │  └───────────────────────────┬────────────────────────────┘                              │                    JSON REST HTTP Requests           (session_id, query, health probes, uploads)                              │                              ▼  ┌────────────────────────────────────────────────────────┐  │                 FastAPI Server (Port 8600)             │  │   - Pre-compiles LangGraph Singleton at startup        │  │   - Manages session threads & MemorySaver checkpointing│  │   - Probes Ollama, Qdrant, and LangSmith health        │  └───────────────────────────┬────────────────────────────┘                              │                              ▼                [Compiled LangGraph Multi-Agent Engine]\n```\n\nThe system decouples the presentation layer from the agent execution engine and calculation services across three dedicated, non-conflicting service ports (8600–8602):\n\nThe frontend interacts with the agent engine across three primary endpoints:\n\nThe Streamlit interface organizes engineering workflows into focused tabs:\n\nA core design goal of this project is that the RAG system can run fully on-premises without depending on external inference APIs.\n\nAt runtime, the architecture uses local components:\n\nThis means that plant manuals, DPRs, maintenance reports, drawings, and operational documents can remain within the organization’s own infrastructure.\n\nExternal APIs such as Gemini and Groq were used only in the offline evaluation workflow as LLM-as-a-judge providers for scoring the golden dataset. They are not required for the runtime RAG pipeline.\n\nFor environments with stricter data policies, the evaluation layer can also be replaced with a local Ollama-based judge. The trade-off is that external judge models can provide an additional independent perspective during benchmarking, while a fully local evaluation setup preserves complete data isolation.\n\nI used two evaluation layers: checks inside the running agent workflow, and an offline benchmark against a golden dataset. They answer different questions.\n\n```\n┌────────────────────────────────────────────────────────────────────────┐│ 1. Runtime In-Flight Evaluation (Zero External APIs)                   ││    - Gate 1: Retrieval Sufficiency Grader (qwen3.5:9b)                 ││    - Gate 2: Faithfulness & Numerical Guardrail (qwen3.5:9b, ≥ 0.90)   ││    - Feedback Loop & Circuit Breaker (Bounded retry budget)            │└────────────────────────────────────────────────────────────────────────┘                                    │                                    ▼┌────────────────────────────────────────────────────────────────────────┐│ 2. Offline Benchmark: Agentic RAG vs. Plain Hybrid-RAG                 ││    - 10 Head-to-Head Policy & Licensing Benchmark Cases                ││    - Multi-provider LLM-as-a-Judge (Google Gemini 2.5 Flash & Groq)    ││    - Evaluates: Faithfulness, Precision, Recall, Relevancy, Latency    │└────────────────────────────────────────────────────────────────────────┘\n```\n\nDuring execution, the system uses the local qwen3.5:9b model as a judge at two stages.\n\nBoth checks share a bounded retry budget (cycle_count < MAX_CYCLES). When the budget is exhausted, insufficient retrieval proceeds to best-effort generation; a failed faithfulness check then routes to a fallback response with a warning.\n\nThese checks do not require a golden answer or an external inference API. However, an LLM judge is still fallible: passing the check is a probabilistic model assessment, not a mathematical proof of factual truth\n\nFor empirical benchmarking, I evaluated the updated Agentic RAG workflow against a standard **Plain Hybrid-RAG baseline** (single dense vector search + BM25 keyword search without agents or decomposition) across 10 complex Small Hydropower policy and licensing questions from the golden dataset.\n\nHere, “offline” means evaluation outside the live request workflow — not disconnected from the internet. API-based judging sends the evaluation content to those providers. A fully local evaluation setup can instead use an Ollama-based judge. Evaluations were performed using automated **LLM-as-a-Judge** scoring via Google Gemini 2.5 Flash with automated failover to Groq (openai/gpt-oss-120b).\n\nThe first four metrics are LLM-assigned scores on a 0–1 scale in this benchmark. They should be interpreted as judge estimates, rather than exact measurements of factual accuracy.\n\nHallucination rate is reported separately from average faithfulness; it is not calculated as 1 − faithfulness. Latency is measured from execution, with mean and median used to compare the two pipelines.\n\nTogether, these measures help distinguish a grounded but incomplete answer from a comprehensive answer containing unsupported claims — and show how much processing time each architecture requires.\n\nThe updated head-to-head comparison covers the same ten policy/licensing questions for both systems. It does not establish performance across all engineering tasks.\n\nThe benchmark also uses multiple judge models through provider fallback, which can introduce scoring variation. A stronger follow-up would use a fixed judge configuration, repeated runs, and human review of difficult cases, especially calculations, tables, and conflicting evidence.\n\nI compared the updated Agentic RAG workflow with a plain hybrid-RAG baseline across 10 policy and licensing questions.\n\nThe result surprised me in a useful way.\n\nThat is the central lesson from this project:\n\nMore agents do not automatically mean more reliable answers.\n\nThey add value when they improve retrieval quality, tool selection, and safe failure behavior. But every extra reasoning step adds latency, complexity, and more opportunities for error.\n\nThis project also taught me where Agentic RAG remains difficult.\n\nAn earlier version of the numerical guardrail incorrectly flagged heading labels such as 3.1 and 4.3 as unsupported engineering claims. That created false failures and unnecessary fallback responses. The fix was not simply “use another LLM.” It required better structure-aware preprocessing before evaluating numbers.\n\nMultiple LLM calls, retrieval operations, validation checks, and correction loops add processing overhead. In my local benchmark, this made the agentic workflow substantially slower than a single retrieval-and-generation pipeline.\n\nThis latency is not necessarily a fixed limitation. More capable GPUs — either on-premises or cloud-hosted — and an optimized inference engine such as **vLLM** could reduce model-serving overhead. Prefix caching can avoid reprocessing shared prompt prefixes, while efficient batching can improve throughput when serving concurrent requests. However, higher throughput does not automatically mean a faster answer for an individual user. [vLLM optimization documentation](https://docs.vllm.ai/en/stable/configuration/optimization/)\n\nIn one benchmark question, the generated answer was grounded in the retrieved content, but the context lacked the expected preference criteria. The system was faithful, but incomplete. This distinction is important:\n\nA trustworthy system needs both.\n\nFuture improvements should focus on **practical reliability, speed, and testing**, rather than merely adding more agents.\n\nThe 3.1× latency penalty observed during local testing is a solvable engineering challenge:\n\nNot every question wants a multi-agent swarm:\n\nBuilding Agentic RAG is not about making a chatbot appear more intelligent.\n\nIt is about deciding **when to retrieve, when to calculate, when to retry, when to abstain, and how to show the user what evidence supports an answer.**\n\nMy current system is not a finished production engineering advisor. It is an evaluated local-first prototype that demonstrates both the promise and the trade-offs of Agentic RAG for domain-specific technical knowledge.\n\nThe biggest result was not that Agentic RAG “beat” plain RAG.\n\nIt was learning exactly where it helped: retrieval precision and structured workflows — and where it still needs work: latency, recall, testing, and reliable engineering-grade verification.\n\n[Agentic RAG vs. Plain Hybrid RAG: What My Hydropower Benchmark Revealed](https://pub.towardsai.net/agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed-40532c1dec55) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed", "canonical_source": "https://pub.towardsai.net/agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed-40532c1dec55?source=rss----98111c9905da---4", "published_at": "2026-11-14 13:01:02+00:00", "updated_at": "2026-11-14 13:22:46.674847+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "large-language-models", "ai-research", "ai-tools"], "entities": ["NaimishKB", "Ollama", "Gemini", "Groq", "Docling", "pymupdf4llm", "GitHub", "Small Hydropower Plant (SHP)"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed", "markdown": "https://wpnews.pro/news/agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed.md", "text": "https://wpnews.pro/news/agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed.txt", "jsonld": "https://wpnews.pro/news/agentic-rag-vs-plain-hybrid-rag-what-my-hydropower-benchmark-revealed.jsonld"}}