{"slug": "rag-async-pipelines-mcp", "title": "RAG - Async Pipelines, MCP", "summary": "A developer explains how asynchronous pipelines can reduce latency in retrieval-augmented generation (RAG) systems, cutting context retrieval time from 17 seconds to 10 seconds by running vector search, text search, and semantic caching concurrently. The post also introduces Model Context Protocol (MCP) as a standardized interface for exposing RAG functionality to other applications and LLMs, avoiding duplicate tool code and reducing HTTP latency.", "body_md": "Everything goes sequentially.\n\nP1, P2, and P3 are the three processes.\n\nSynchronous means P2 will start when P1 starts and finishes its job. P3 will start when P2 starts and finishes its job.\n\nIt is not required all the time.\n\nDuring retrieval, in the case of **hybrid search**, first we will do vector search, then text search, and then any work related to semantic caching / context search.\n\nThe goal here is to give more relevant context to the LLM.\n\nSuppose:\n\nIn the case of a sequential process, we need to wait **17 seconds** to get the context.\n\nSo, is there any way we can reduce the waiting time?\n\nThe answer is **Async Pipelines**.\n\nHere, we will start 3 threads at the same time.\n\nTheir finishing times may be different.\n\nAt most, we have to wait only **10 seconds** to get the context.\n\nThrough this async pipeline, we are reducing latency.\n\nA single thread switches between multiple tasks when needed. It is also called **parallel processing**.\n\nEach task will be allotted a separate thread. So, it can finish on its own time.\n\nThese threads are limited to the available CPU cores.\n\nWe can see the real difference in latency when we do this at the production level.\n\nIf we ask the question **\"What happened today?\"**, an LLM cannot answer.\n\nIf we ask the same question to ChatGPT or Gemini, it gives a response. How?\n\nBecause of **tool calling**, which means attaching some tools or functionality to the LLM. The result of the tool will be fed into the LLM.\n\nThe application developer will be responsible for writing the code for the tool/functionality.\n\nSuppose the idea of the application is to fetch weather details.\n\nEvery developer may write their own code to fetch the weather details. At the end, all users/developers will get more or less the same result.\n\nThis is not the right approach.\n\nSo, here comes the concept of **MCP**.\n\nThe responsible body/owner that provides the weather details will write the common tool or protocol (a set of rules).\n\nThe consumer/developer will use the common tool/protocol to fetch the result.\n\nHere, the developer does not need to write their own code.\n\nThe common tool/protocol is called **MCP**.\n\nMCP will have functions and their descriptions. The LLM will decide which function to call.\n\nWe can call the tool either by:\n\nMCP is within the same machine.\n\nWe can also use the HTTP method to call the MCP, such as calling an API.\n\nHowever, we are introducing latency when using HTTP.\n\nCreate an MCP for a RAG system.\n\nFor example, suppose we have built a RAG system using documents that we gathered.\n\nSince you have the database and RAG system, you are the only person who can invoke the RAG system directly.\n\nInstead of keeping the RAG functionality restricted to your application, we can expose the RAG functionality through **MCP**.\n\nWe can segregate the functionality according to different responsibilities or divisions.\n\nFor example:\n\nNow, anyone who has access to the MCP can use the RAG functionality without directly accessing the underlying database or writing their own retrieval code.\n\nIn this way, **MCP acts as a common interface between the RAG system and different applications or LLMs**.\n\nThe RAG system continues to handle document retrieval, while MCP provides a standardized way for other applications or agents to access that functionality.", "url": "https://wpnews.pro/news/rag-async-pipelines-mcp", "canonical_source": "https://dev.to/ramya_perumal/rag-async-pipelines-mcp-4j8j", "published_at": "2026-08-18 19:47:08+00:00", "updated_at": "2026-08-18 20:13:52.238199+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["RAG", "MCP", "ChatGPT", "Gemini"], "alternates": {"html": "https://wpnews.pro/news/rag-async-pipelines-mcp", "markdown": "https://wpnews.pro/news/rag-async-pipelines-mcp.md", "text": "https://wpnews.pro/news/rag-async-pipelines-mcp.txt", "jsonld": "https://wpnews.pro/news/rag-async-pipelines-mcp.jsonld"}}