How to Build a High-Performance RAG Pipeline with Ollama, Python and TypeScript

If you need to spin up a local, privacy-first AI agent that can query your own internal documents without sending data to third-party APIs, this guide covers the exact architecture using TypeScript, Python, and Ollama.

Time to complete:~15 minutes.

Prerequisites:Python 3.10+ or Node.js installed, basic familiarity with embeddings.

When building production-ready LLM features, relying solely on cloud providers introduces two major friction points: variable API latency and data compliance bottlenecks.

By shifting the embedding generation and model inference locally, we completely bypass network overhead and keep sensitive data securely inside our infrastructure.

Here is how the data flows through our system:

First, ensure you have Ollama running locally and pull the required models. Open your terminal and run:

ollama pull llama3

ollama pull nomic-embed-text

Choose your preferred language environment to house the orchestration logic.

// index.ts
import { Ollama } from 'ollama';

const ollama = new Ollama({ host: 'http://127.0.0.1:11434' });

async function generateLocalEmbedding(text: string): Promise<number[]> {
  const response = await ollama.embeddings({
    model: 'nomic-embed-text',
    prompt: text,
  });
  return response.embedding;
}

First, install the official client: pip install ollama

import asyncio
from ollama import AsyncClient

client = AsyncClient(host='http://127.0.0.1:11434')

async def generate_local_embedding(text: str) -> list[float]:
    response = await client.embed(
        model='nomic-embed-text',
        input=text
    )
    return response['embeddings'][0]

When querying the local vector array, we calculate the similarity score to find the most relevant document chunks.

function cosineSimilarity(vecA: number[], vecB: number[]): number {
  const dotProduct = vecA.reduce((sum, a, i) => sum + a * vecB[i], 0);
  const normA = Math.sqrt(vecA.reduce((sum, a) => sum + a * a, 0));
  const normB = Math.sqrt(vecB.reduce((sum, b) => sum + b * b, 0));
  return dotProduct / (normA * normB);
}
php
import math

def cosine_similarity(vec_a: list[float], vec_b: list[float]) -> float:
    dot_product = sum(a * b for a, b in zip(vec_a, vec_b))
    norm_a = math.sqrt(sum(a * a for a in vec_a))
    norm_b = math.sqrt(sum(b * b for b in vec_b))

    if not norm_a or not norm_b:
        return 0.0  # Prevent division by zero

    return dot_product / (norm_a * norm_b)

Building local agentic workflows gives you complete control over your data lifecycle and cuts API bills down to zero.

Let me know in the comments below: Are you running your LLMs locally or sticking to cloud APIs for production?

This tutorial on Building Local RAG with Ollama provides an excellent visual look at parsing document chunks and handling embedding shapes using the official python libraries we integrated into the text.