How to Build a High-Performance RAG Pipeline with Ollama, Python and TypeScript

A developer built a high-performance RAG pipeline using Ollama, Python, and TypeScript that runs entirely locally, eliminating cloud API latency and data compliance issues. The architecture uses Ollama for embedding generation and model inference, with cosine similarity for document retrieval. The guide provides code examples for both TypeScript and Python implementations.

If you need to spin up a local, privacy-first AI agent that can query your own internal documents without sending data to third-party APIs, this guide covers the exact architecture using TypeScript, Python, and Ollama. Time to complete:~15 minutes. Prerequisites:Python 3.10+ or Node.js installed, basic familiarity with embeddings. When building production-ready LLM features, relying solely on cloud providers introduces two major friction points: variable API latency and data compliance bottlenecks. By shifting the embedding generation and model inference locally, we completely bypass network overhead and keep sensitive data securely inside our infrastructure. Here is how the data flows through our system: First, ensure you have Ollama running locally and pull the required models. Open your terminal and run: Pull the LLM ollama pull llama3 Pull the embedding model explicitly ollama pull nomic-embed-text Choose your preferred language environment to house the orchestration logic. js // index.ts import { Ollama } from 'ollama'; const ollama = new Ollama { host: 'http://127.0.0.1:11434' } ; async function generateLocalEmbedding text: string : Promise<number { const response = await ollama.embeddings { model: 'nomic-embed-text', prompt: text, } ; return response.embedding; } First, install the official client: pip install ollama python orchestrator.py import asyncio from ollama import AsyncClient Initialize the asynchronous local client client = AsyncClient host='http://127.0.0.1:11434' async def generate local embedding text: str - list float : response = await client.embed model='nomic-embed-text', input=text The client returns a list of embedding arrays inside 'embeddings' return response 'embeddings' 0 When querying the local vector array, we calculate the similarity score to find the most relevant document chunks. js function cosineSimilarity vecA: number , vecB: number : number { const dotProduct = vecA.reduce sum, a, i = sum + a vecB i , 0 ; const normA = Math.sqrt vecA.reduce sum, a = sum + a a, 0 ; const normB = Math.sqrt vecB.reduce sum, b = sum + b b, 0 ; return dotProduct / normA normB ; } php import math def cosine similarity vec a: list float , vec b: list float - float: dot product = sum a b for a, b in zip vec a, vec b norm a = math.sqrt sum a a for a in vec a norm b = math.sqrt sum b b for b in vec b if not norm a or not norm b: return 0.0 Prevent division by zero return dot product / norm a norm b Building local agentic workflows gives you complete control over your data lifecycle and cuts API bills down to zero. Let me know in the comments below: Are you running your LLMs locally or sticking to cloud APIs for production? This tutorial on Building Local RAG with Ollama https://www.youtube.com/watch?v=IM6bq3BUQAI provides an excellent visual look at parsing document chunks and handling embedding shapes using the official python libraries we integrated into the text.