I Built a Hybrid Search Engine From Scratch — Here's What I Learned (LLM Zoomcamp 2026, Module 2)

A developer completed Module 2 of the LLM Zoomcamp 2026 and built a hybrid search engine from scratch, combining keyword and vector search. The project implemented vector search using ONNX runtime embeddings and cosine similarity, then integrated it with keyword search using Reciprocal Rank Fusion (RRF) for improved retrieval accuracy. The developer found that vector search captures semantic meaning while keyword search matches exact words, and hybrid search outperforms either alone.

I just completed Module 2 of the LLM Zoomcamp 2026 by @DataTalksClub https://github.com/DataTalksClub/llm-zoomcamp/ — and this module completely changed how I think about search. Module 1 taught me RAG and agentic pipelines. Module 2 taught me that the search step inside RAG matters far more than I realized — and that keyword search is only half the story. Here's everything I built and learned. Traditional keyword search matches words. If you search for "enroll", it finds documents containing "enroll" — but misses documents about "joining", "signing up", or "registration" even if they mean exactly the same thing. Vector search matches meaning, not words. Every piece of text gets converted into a vector — a list of hundreds of numbers that captures its semantic meaning. Similar meanings produce similar vectors, so you can find relevant documents even when they use completely different words. This is the foundation of modern AI-powered search, and it's what makes RAG systems actually work at scale. Instead of downloading the full PyTorch + CUDA stack ~2GB , I used a lightweight ONNX runtime embedder — same vectors, 30x smaller installation, runs on any CPU: python from embedder import Embedder embedder = Embedder loads Xenova/all-MiniLM-L6-v2 via ONNX v = embedder.encode "How does approximate nearest neighbor search work?" print len v 384 dimensions The model produces 384-dimensional vectors — each number represents a dimension of meaning in the text. Before using any library, I implemented vector search by hand to understand what's happening under the hood: python import numpy as np cosine similarity — vectors are normalized, so dot product works directly def cosine similarity a, b : return np.dot a, b score all chunks against a query scores = X.dot v X is the matrix of all chunk embeddings best idx = np.argmax scores This is exactly what vector databases like Qdrant and pgvector do internally — just much faster at scale using HNSW indexing. Full pages are too long and dilute the embedding — a match buried deep in a 10,000-character page still pulls in the whole page. The fix is chunking: python from gitsource import chunk documents chunks = chunk documents documents, size=2000, step=1000 72 pages → 295 overlapping chunks Overlapping chunks step < size ensure sentences at boundaries don't get cut off. After chunking, retrieval becomes far more precise. minsearch now has a VectorSearch class that wraps the numpy math into a clean interface: python from minsearch import VectorSearch vector index = VectorSearch keyword fields= "filename" vector index.fit X, chunks results = vector index.search query vector, num results=5 For the query "How do I store vectors in PostgreSQL?" : 08-pgvector.md entirely because "pgvector" wasn't in the query 08-pgvector.md first because it understood the semantic connection between "store vectors" and "pgvector"This is the key insight: vector search finds meaning, keyword search finds words. Neither approach is perfect on its own: The solution is hybrid search — run both and merge the results using RRF: python def rrf result lists, k=60, num results=5 : scores = {} docs = {} for results in result lists: for rank, doc in enumerate results : key = doc "filename" , doc "start" scores key = scores.get key, 0 + 1 / k + rank docs key = doc ranked = sorted scores, key=scores.get, reverse=True return docs key for key in ranked :num results results = rrf vector results, text results RRF ignores raw scores which live on different scales and only looks at rank position. A document that ranks well in both lists beats one that's only strong in a single list — even if it wasn't first in either. 1. Embeddings capture meaning, not words. "Enroll" and "join" produce similar vectors. "Pizza" and "enrollment" don't. This is what makes semantic search powerful. 2. Chunking is not optional. Full pages dilute embeddings. 2,000-character overlapping chunks dramatically improve retrieval precision and cut LLM input tokens by 3x. 3. Neither keyword nor vector search is best. Use hybrid search RRF in production. It consistently outperforms either approach alone. 4. ONNX makes embeddings practical anywhere. No GPU, no PyTorch, no CUDA. 67MB download, runs on a basic laptop. There's no reason not to use vector search even in constrained environments. 5. The right search approach depends on your data. Vector search wins for semantic queries. Keyword search wins for exact terms names, codes, IDs . Hybrid wins most of the time — but measure to be sure. All my code for Module 2 is open source: github.com/Derrick-Ryan-Giggs/llm-zoomcamp-2026 https://github.com/Derrick-Ryan-Giggs/llm-zoomcamp-2026 It includes: vector-search.ipynb — embeddings, Qdrant, and vector RAG pipeline Vector Search Homework.ipynb LLM Zoomcamp is completely free — no paywall, no certificate fees. Sign up: github.com/DataTalksClub/llm-zoomcamp https://github.com/DataTalksClub/llm-zoomcamp/ Are you working through LLM Zoomcamp 2026? Drop a comment — I'd love to compare notes.