I Built a RAG Pipeline in TypeScript Without LangChain — The Whole Thing in 200 Lines

A developer built a complete retrieval-augmented generation (RAG) pipeline in TypeScript without using frameworks like LangChain or LlamaIndex, totaling about 200 lines across six files. The pipeline runs entirely on a local laptop, using a local LLM and an in-memory vector store, and the developer explains each stage from scratch, including the data structures and common bugs encountered.

Every RAG tutorial I found looked like this: js const chain = RetrievalQAChain.fromLLM model, vectorStore.asRetriever ; const res = await chain.call { query: "what is this document about?" } ; Twelve lines, a Pinecone key, a screenshot of it answering one question about one PDF, and a confident closing paragraph about "production readiness." I read four of them and still couldn't have told you what an embedding actually was, why cosine similarity was the metric everyone used, or what would happen if my documents were 800 pages instead of 8. I could copy the code. I couldn't debug it. So I deleted the frameworks and wrote the whole thing by hand. No LangChain, no LlamaIndex, no hosted vector database, and no cloud LLM — the model runs on my laptop. Six files, a bit over 200 lines of TypeScript, and nothing imported that I can't explain. This post is the whole pipeline, the data structures behind each stage and why they were chosen, the four bugs that cost me the most time, and a debugging method that will save you an afternoon. I'm assuming you write JavaScript or TypeScript, you're comfortable with async / await , arrays, and classes, and you've installed an npm package before. That's it. I am not assuming you know anything about machine learning, vectors, embeddings, or information retrieval. Every one of those is explained from zero as it comes up, and if a line of code does something non-obvious, I explain the line. If you already know what a vector store is, skip to the bug list at the bottom. Strip the acronym away and RAG is one idea: Language models can't read your files. So find the relevant paragraphs yourself, paste them into the prompt, and ask the question. The rest of the pipeline exists to make that sentence practical. Finding the right paragraphs is the hard part. You can't keyword-search your way there, because a user asking "how do I stop duplicate rows" won't use the word "DISTINCT" that appears in your document. Keyword search matches letters, and you need something that matches meaning. That's what the pipeline below does, in five steps: I split those five steps across six files. | File | Job | |---|---| pdf.ts | Pull raw text out of a PDF | chunker.ts | Sliding-window split with overlap | embedder.ts | Call the embedding API, get number arrays back | vectorStore.ts | Hold chunks in memory, cosine similarity search | rag.ts | Orchestrate the four above, call the model | index.ts | CLI entry point | npm init -y npm i pdf-parse@1.1.1 voyageai openai dotenv npm i -D typescript tsx @types/node Two things you need before writing a line: .env as VOYAGE API KEY — the Voyage client picks it up from the environment on its own, so you never pass it explicitly. localhost:1234 and speaks the OpenAI API format, which means the official openai SDK talks to it without modification. No API key, no per-token cost, no internet required.That second choice matters more than it looks. Running the model locally means you can hammer this thing for a whole weekend without watching a billing dashboard, and it forces you to deal with a small context window early, which is where one of the more instructive bugs came from. Note the pinned pdf-parse@1.1.1 . That is not an accident. More on it later. python // src/pdf.ts import pdfParse from "pdf-parse"; import fs from "fs"; export async function readPdf filePath: string : Promise<string { const buffer = fs.readFileSync filePath ; const parsedData = await pdfParse buffer ; return parsedData.text; } Reading the code: fs.readFileSync without an encoding argument returns a Buffer , Node's raw byte container, rather than a string. That's deliberate. A PDF is a binary format, and if you passed "utf-8" here Node would try to decode those bytes as text and hand you mangled garbage. pdf-parse wants the raw bytes so it can walk the file's internal structure itself. Yes, that's a synchronous read inside an async function, which blocks the event loop while the file loads. For a CLI that reads one file at startup and then sits waiting for you to type, it makes no observable difference. In a server handling concurrent uploads it would, and you'd swap in fs/promises . There's no interesting data structure at this stage. Extraction is a solved problem and pdf-parse solves it. What comes back is one long string with the layout thrown away: collapsed tables, lost column boundaries, page headers repeated every forty lines. We'll live with that for now. You can't embed the whole document as one unit. A fifty-page PDF becomes one blurry vector that's vaguely about everything and precisely about nothing, because all its meaning gets averaged into a single point. So you cut it into pieces. The algorithm here is a sliding window : a fixed-size window that advances across the input in fixed steps, where the step is smaller than the window, so consecutive windows overlap. // src/chunker.ts export function chunkText text: string, options: { chunkSize: number; overlap: number; }, : string { const words = text.split " " ; let step = options.chunkSize - options.overlap; const chunks: string = ; for let i = 0; i < words.length; i += step { let currentChunk = ; for let j = i; j < Math.min i + options.chunkSize, words.length ; j += 1 { currentChunk.push words j ; } chunks.push currentChunk.join " " ; } return chunks; } I call it with { chunkSize: 500, overlap: 100 } . Reading the code, line by line: const step = options.chunkSize - options.overlap is where the overlap actually gets implemented. With a 500-word window and 100 words of overlap, the window advances 400 words at a time. So chunk 0 covers words 0–499, chunk 1 covers 400–899, chunk 2 covers 800–1299. Each pair of neighbours shares 100 words. The inner loop is a manual copy of the window into currentChunk . Math.min i + options.chunkSize, words.length is what stops it walking past the end of the array on the final chunk. You could write the same thing as words.slice i, i + chunkSize and slice would handle the boundary for you, but I wrote the loop out because I wanted the window arithmetic visible rather than hidden behind a method call. Why an array of words rather than characters. A character-based window would slice words in half, producing "norm" and "alization" , and the embedding model would see two fragments that mean nothing. Splitting on words keeps every boundary between real tokens. Why not sentences? Because splitting English on . breaks on abbreviations, decimals, and version numbers. Proper sentence-aware chunking is a real improvement, and it's the subject of the next post. Two things wrong with this function , both of which I left in because they're instructive: text.split " " splits on a single space character, not on whitespace generally. PDF text is full of newlines, so a line ending mid-sentence produces a token like "end.\nBeginning" — two words glued together with a newline in the middle, which the embedding model has never seen as a unit. split /\s+/ fixes it. I didn't notice for two days. There's also no early exit. The loop keeps stepping while i < words.length , so once one window has already reached the end of the text, the next iterations emit progressively shorter chunks covering ground the previous chunk already covered. With 850 words you get chunk 0 at 0–499, chunk 1 at 400–849, and then chunk 2 at 800–849 — fifty words that are a strict subset of chunk 1. It costs you an embedding call and adds a near-duplicate to your search results. Complexity. The outer loop advances by step and the inner loop copies chunkSize words, so the total work is O n × chunkSize / step — with 500/400, about 1.25 copies per word. Raise the overlap and that multiplier climbs: at 500/250 every word is copied twice. So overlap costs you memory and embedding calls, not just storage. Why the overlap exists at all. Without it, a sentence that straddles a boundary gets split across two chunks and neither one carries the complete idea. With 100 words of overlap, every boundary appears intact inside at least one chunk. I ran the same PDF through at 150 words and at 500 and got materially different answers to the same question. That surprised me at the time. It shouldn't have: different boundaries produce different vectors, which produce different similarity scores, which retrieve different chunks, which means the model is reading different text. Chunking isn't preprocessing. It's the retrieval algorithm. An embedding is a list of numbers that represents meaning. Text goes in, an array of floats comes out, typically several hundred to a couple of thousand of them. Two pieces of text that mean similar things produce arrays that point in similar directions. js // src/embedder.ts import { VoyageAIClient } from "voyageai"; const voyage = new VoyageAIClient ; // reads VOYAGE API KEY from process.env automatically export async function embedText text: string : Promise<number { const response = await voyage.embed { input: text, model: "voyage-3-lite", } ; return response.data?. 0 .embedding ?? ; } export async function embedBatch text: string : Promise<number { const response = await voyage.embed { input: text, model: "voyage-3-lite", } ; const responses = response.data ?? .map resp: any = resp.embedding ; return responses; } The data structure is number , an array of arrays. Each inner array is one chunk's vector, and they're all the same length. That length is the model's Why two functions. embedBatch runs once at ingest over every chunk in the document. embedText runs once per question. Same API call underneath — Voyage accepts either a string or an array of strings — but keeping them separate means the call sites read honestly, and it leaves an obvious place to add query-specific handling later. Reading the code: new VoyageAIClient takes no arguments because the client reads VOYAGE API KEY from the environment itself. That works only if dotenv.config has already run, which is why it's called at the top of rag.ts before this module gets used. response.data ?? is the nullish coalescing operator: if response.data is null or undefined , substitute an empty array so .map has something to iterate. That fallback exists because of a bug I'll describe below; it wasn't there in the first version. resp: any is a cast I'm not proud of. The generated SDK types wouldn't narrow the way I expected and I stopped fighting them. It works, and it's the kind of thing that quietly removes the type safety you installed TypeScript for. response.data?. 0 .embedding in embedText guards data but not 0 , so an empty results array would still throw. The ?? at the end catches a missing embedding , not a missing element. It hasn't bitten me yet, which is not the same as being correct. One thing this doesn't do. Voyage supports an inputType parameter that distinguishes documents you're storing from queries you're searching with, and using it improves retrieval measurably. I'm not passing it in either function. That's a known gap rather than a decision. The fragile invariant. embedBatch returns a number that lines up positionally with the array of chunks you passed in — vectors i belongs to chunks i , and nothing in the type system enforces it. Filter, sort, or reorder one array without the other and every chunk gets silently paired with the wrong vector, with no error thrown. You'll see below where those two arrays get zipped back together. Why the whole array goes in one call. Batching means one HTTP round trip for 200 chunks instead of 200. Beyond the latency, you will hit a rate limit long before you finish the naive one-call-per-chunk version. Here's the part everyone assumes needs a database. It doesn't, not yet. Store the chunks and their vectors in an array. To search, compute cosine similarity against every single one and take the top few. // src/vectorStore.ts export interface VectorEntry { text: string; embedding: number ; } export class VectorStore { private entries: VectorEntry = ; private cosineSimilarity a: number , b: number : number { // Step 1 : Dot product | Measures agreement let dot = 0; for let i = 0; i < a.length; i += 1 { dot += a i b i ; } // Step 2 : Magnitude | Measures length, loudness let magA = 0; for let i = 0; i < a.length; i += 1 { magA += a i a i ; } magA = Math.sqrt magA ; let magB = 0; for let i = 0; i < b.length; i += 1 { magB += b i b i ; } magB = Math.sqrt magB ; // Step 3 : Divide | Cancels out the length, leaving only direction return dot / magA magB ; } add text: string, embedding: number { let newEntry: VectorEntry = { text, embedding }; this.entries.push newEntry ; } query queryEmbedding: number , topK: number { const cosineMapResults = this.entries.map entry = { let score = this.cosineSimilarity queryEmbedding, entry.embedding ; return { entry, score }; } ; cosineMapResults.sort a, b = b.score - a.score ; let cosineSortedEntries = cosineMapResults.map result = result.entry ; return cosineSortedEntries.slice 0, topK ; } size { return this.entries.length; } } The data structure is an array of records. add takes one chunk and one vector and stores them as a single VectorEntry , so from that point on a chunk and its vector travel together and no amount of sorting can separate them. That's the answer to the parallel-array problem from the previous step — the pairing happens once, at insert time, and then stops being something you can get wrong. query returns whole entries rather than bare strings, which matters more than it looks. Once you start attaching page numbers or source filenames to each entry, the caller already has them. Picture each embedding as an arrow starting at the origin. An arrow has two properties: which way it points, and how long it is. Length encodes things you mostly don't care about — roughly, how much text there was and how emphatic it was. Direction is what encodes meaning. So you want a comparison that looks only at direction and ignores length. That's the cosine of the angle between the two arrows: Why the formula works. The dot product step 1 in the code: multiply matching positions, add them up measures how much two arrows agree in direction, but it grows when either arrow gets longer. Dividing by both magnitudes cancels the length out, leaving direction. magA and magB accumulate the sum of squares, and Math.sqrt turns each into a length — Pythagoras, extended to a thousand dimensions instead of two. Why three separate loops. All three could run in a single pass over the arrays, and that would be faster because it walks memory once instead of three times. I split them because each loop is one line of the formula and I wanted to read it that way while I was still learning it. At a thousand chunks the difference is unmeasurable. At a million it wouldn't be. Why not Euclidean distance? Straight-line distance between two points is sensitive to magnitude, so a long chunk and a short chunk about the same topic land far apart even though they mean the same thing. Cosine sidesteps that. If your vectors are already normalised to length 1, the two rank identically, but don't assume yours are. Complexity: O n × d , n chunks by d dimensions, times three for the separate passes. A thousand chunks at 512 dimensions is about 1.5 million multiply-adds per query, which modern JS engines do in single-digit milliseconds. The sort is the sloppier part. sort orders all n results at O n log n when you only need the top 3; a fixed-size min-heap would do it in O n log k . At n = 1000 the difference is invisible, and I'd rather ship the version a reader can verify at a glance. This is a linear scan against every vector in the corpus, and for a few thousand chunks it's fast enough that you won't notice. Unlike an approximate index, it also never returns a wrong neighbour. I'd rather understand the forty lines above than configure a managed index that does the same thing behind an API I've never read. python // src/rag.ts import dotenv from "dotenv"; dotenv.config ; import { chunkText } from "./chunker"; import { embedBatch, embedText } from "./embedder"; import { readPdf } from "./pdf"; import OpenAI from "openai"; import { VectorStore } from "./vectorStore"; const lmstudio = new OpenAI { baseURL: process.env.LM STUDIO BASE URL ?? "http://localhost:1234/v1", apiKey: "lm-studio", } ; const DEBUG = false; // flip to true when debugging export class RAGPipeline { private store = new VectorStore ; // your in-memory "database" async indexPDF filePath: string : Promise<void { const rawText = await readPdf filePath ; const chunkedText = chunkText rawText, { chunkSize: 500, overlap: 100 } ; const embeddedBatch = await embedBatch chunkedText ; chunkedText.forEach chunk, i = { this.store.add chunk, embeddedBatch i ; } ; } async ask question: string : Promise<string { const questionEmbedding = await embedText question ; const relevantChunks = this.store.query questionEmbedding, 3 ; if DEBUG { console.log "\n--- Retrieved Chunks ---" ; relevantChunks.forEach c, i = console.log \nChunk ${i + 1}:\n${c.text} , ; console.log "\n--- End Chunks ---\n" ; } const context = relevantChunks.map c = c.text .join "\n\n" ; const prompt = Answer the question using ONLY the context below. If the answer is not in the context, say "I don't have that information." Context: ${context} Question: ${question} Answer: ; const response = await lmstudio.chat.completions.create { model: "local-model", messages: { role: "user", content: prompt } , max tokens: 1024, temperature: 0.2, } ; if DEBUG console.log response ; return response.choices 0 .message.content ?? "No response from model."; } } Reading the code: overriding baseURL is what makes the local setup work. The official OpenAI SDK will talk to anything that implements the same HTTP interface, and LM Studio does. apiKey is required by the SDK's constructor and thrown away by the server, so any non-empty string works. The LM STUDIO BASE URL override exists so you can point at a different port without editing code. dotenv.config sits at the very top, above the other imports, and that placement is load-bearing. The Voyage client in embedder.ts reads its key at module-initialisation time, so the environment has to be populated before that import is evaluated. indexPDF is where the two parallel arrays get zipped: chunkedText.forEach chunk, i = this.store.add chunk, embeddedBatch i walks the chunks and pairs each with the vector at the same index. This one line is the only place the invariant from Step 3 can be violated, which is exactly where you want it — one line, easy to stare at. The DEBUG flag prints every retrieved chunk before the model sees it. I added it after the incident described below and I've never turned it off for long. Two lines in that prompt are doing real work: "using ONLY the context below" — without it, the model happily answers from its own training data and you have no idea whether retrieval worked at all. "say I don't have that information" — you have to tell the model that refusing is allowed. If you don't offer that option, it will fill the gap with something plausible. temperature: 0.2 rather than 0. This is a lookup task, so you want near-deterministic answers, but I found a small amount of slack produced better-phrased responses without changing the facts. Set it to 0 if you want reproducibility for testing. max tokens: 1024 reserves room for the answer, and that reservation comes out of the same context budget as your retrieved chunks. Remember that number — it's about to matter. One thing to change. The chunks are joined with \n\n and nothing else. They usually aren't adjacent in the original document, so the model reads them as continuous prose and occasionally invents a connection between unrelated passages. A visible separator like \n\n---\n\n costs three characters and stops it. // src/index.ts import as readline from "readline"; import as path from "path"; import dotenv from "dotenv"; import { RAGPipeline } from "./rag"; dotenv.config ; async function main : Promise<void { const pdfPath = process.argv 2 ; if pdfPath { console.error "Usage: npm start <path-to-pdf " ; console.error "Example: npm start ./my-document.pdf" ; process.exit 1 ; } const rag = new RAGPipeline ; await rag.indexPDF path.resolve pdfPath ; const rl = readline.createInterface { input: process.stdin, output: process.stdout, } ; console.log "\nAsk questions about your PDF. Type 'exit' to quit.\n" ; // Wrap rl.question in a Promise so we can use it in an async loop const ask = : Promise<string = new Promise resolve = rl.question "You: ", resolve ; while true { const question = await ask .trim ; if question continue; if question.toLowerCase === "exit" { console.log "------------------- Goodbye -------------------" ; rl.close ; break; } try { console.log "\nThinking..." ; const answer = await rag.ask question ; console.log \nLLM: ${answer}\n ; } catch err: unknown { if err instanceof Error && err.message.includes "ECONNREFUSED" { console.error "Cannot connect to LM Studio. Is the server running?" ; } else { console.error "Error:", err ; } } } } main .catch console.error ; Everything lives inside main so there's no top-level await to configure. readline 's question is callback-based, so the ask helper wraps it in a promise and the loop can await a line of input like any other async operation. The ECONNREFUSED branch exists because I hit it about nine times. The raw error is a wall of stack trace that says nothing about LM Studio, and forgetting to start the server is the single most likely reason this program fails on a fresh machine. npx tsx src/index.ts ./docs/sql-reference-guide.pdf I pointed it at a SQL reference guide, asked it a question about joins, and got a correct answer assembled from three chunks of a document the model had never seen. It took most of a Saturday to get there. Full code, tagged at exactly the state described in this article: https://github.com/APURWA98/rag-from-scratch/tree/part-1 https://github.com/APURWA98/rag-from-scratch/tree/part-1 You: How many types of joins are there ? Thinking... LLM: There are 6 types of joins mentioned in the context: 1. INNER JOIN 2. LEFT JOIN 3. RIGHT JOIN 4. FULL OUTER JOIN 5. CROSS JOIN 6. Implicit INNER JOIN used when only one table is specified, e.g., SELECT FROM users You: what is implicit join ? Thinking... LLM: I don't have that information. Item 6 is wrong. SELECT FROM users has one table in it. There's nothing to join. The word "implicit" doesn't appear anywhere in the PDF — I searched it. So the model found five joins in the retrieved chunks and added a sixth, with an example that doesn't match the term it had just invented. temperature was 0.2, and the first line of the prompt says to use only the context. Then it refused to define the term. That part is correct — there was nothing to retrieve. The instruction works. It just didn't stop the first answer. Worth knowing that the follow-up went in cold. Each ask embeds only the question you typed; there's no history anywhere in this code. "what is implicit join ?" reached the retriever as four words with no memory of the previous turn. It looks like a conversation in the terminal, but nothing carries between turns. I haven't fixed any of it. It's in the repo the way it happened. The code above is the version that works. Getting there cost me four bugs, and three of them had nothing to do with RAG. pdf-parse v2 changed its API The install pulled v2. Every tutorial and every Stack Overflow answer was written against v1, and the import shape had changed underneath them. The error message was unhelpful. Fix: pin pdf-parse@1.1.1 . Lesson: when a library's examples don't match its behaviour, check the installed major version before you check anything else. This accounts for a lot of broken tutorials. I wrote import { VoyageAI } from "voyageai" because that's the obvious name. Undefined. The actual export is VoyageAIClient . I found it in one command: js node -e "console.log Object.keys require 'voyageai' " That prints every top-level export of a package. It's now the first thing I run when an import comes back undefined. data came back undefined response.data.map blew up mid-ingest, after several successful API calls, which made it look intermittent rather than structural. Fix: the ?? fallback you can see in embedBatch . Lesson: an SDK's TypeScript types describe the happy path, not the guaranteed path. In a generated client, fields marked optional usually are. Note that embedText still has the weaker version of this guard — I fixed the function that crashed and not the one that hadn't yet. n keep = n ctx This one took the longest to find. A 400 from LM Studio, mid-conversation, no obvious trigger. The local model had a 4096-token context window. I was retrieving topK = 5 chunks of 500 words each. English runs roughly 1.3 tokens per word, so five chunks is about 3,250 tokens, plus the instructions and the question, plus max tokens: 1024 reserved for the answer. That's over 4,400 against a 4,096 ceiling before the model generates a single word. Fix: drop topK from 5 to 3. Three chunks is about 1,950 tokens, and the whole request lands near 3,100 — comfortably inside. Why it matters: retrieving more chunks is not free and it is not obviously better. Every extra chunk spends context window, and max tokens spends it too, before you've retrieved anything. There's a ceiling on how much context actually helps. At one point the pipeline answered "I don't have that information" to a question I could see answered on page 3. Three places can fail, and it's worth checking them in this order: Layer 1 — Extraction. Log rawText.slice 0, 200 and rawText.length right after readPdf . Is the text there at all? Is it readable, or is it ligature soup? A scanned PDF with no text layer produces an empty string, and every downstream stage runs fine on an empty string. Layer 2 — Retrieval. Log the chunks that come back from store.query . Is the right chunk in there? If extraction is fine but the correct chunk never surfaces, your problem is chunking or embedding, not the model. This is what the DEBUG block in ask is for. Layer 3 — Prompt. Log the complete prompt string immediately before the API call. Is the context actually in there, or did you interpolate an empty array? I skipped straight to blaming the model. It was Layer 2. It usually is. Build the flag in from day one. You will use it constantly. The defaults above are tuned for one thing: a moderately sized text-heavy PDF, queried by one person, on a laptop. Change any of those and something needs to move. | If your situation is... | Change this | |---|---| | Legal contracts, dense definitions | Smaller chunks, larger overlap, prepend the section heading to every chunk | | Scanned documents | The extraction stage entirely — you need OCR before any of this runs | | Users search error codes, IDs, product names | Pure vector search will fail you. You need keyword search alongside it | | Tens of thousands of chunks | The linear scan starts to hurt. Move to a real index | | Data can't leave your machine | You already have this — local model, and swap the embedding API for a local model too | | Answers must be auditable | Carry page numbers on each VectorEntry and force citations in the prompt | If you build this, change one parameter at a time and watch what happens. Chunk size. Overlap. topK . The strictness of that grounding instruction. Each one changes the answers in ways worth seeing for yourself. Chunking.