{"slug": "semantic-hybrid-search-in-the-browser", "title": "Semantic/Hybrid Search in the Browser", "summary": "A developer built a semantic search engine that runs entirely in the browser using a static embedding model (model2vec) instead of a full transformer, reducing the model size to a 4 MB lookup table. The approach avoids server-side dependencies and large downloads, making it suitable for static sites with limited resources.", "body_md": "Eight years ago I [added client-side search to this blog with Lunr.js](/searching-your-hugo-site-with-lunr/). It creates an inverted index at build time, ships it as JSON, and matches strings in your browser. No server-side engine required. It has worked fine ever since, in the sense that it finds a post if you type a word that is actually in it.\n\nEarlier this year I wrote a [semantic search engine in ±250 lines of Python](/building-a-semantic-search-engine-in-250-lines-of-python/) (the kind that lets you find the “London Beer Flood” when you search for “alcoholic beverage disaster in England,” because it understands that beer is alcoholic and a flood is a disaster). That one needs a machine with `sentence-transformers`\n\ninstalled and a few hundred megabytes of PyTorch; to serve something like that in a production server requires beefy machines with expensive RAM. Not something you run in a browser tab.\n\nSo this blog had keyword search that runs anywhere and understands nothing, but no semantic search that understands things but can’t run anywhere near a static site. This post is about closing that gap: **semantic search that runs entirely in your browser, with no server and no API, where the entire model is essentially a 4 MB lookup table.** You can try all three of the models I benchmarked, [further down](#the-benchmark-your-hardware-my-corpus), running live on your own hardware (I can’t afford GPU clusters).\n\nDoing this surfaced a couple things, chief among which that the keyword search had a bug for a couple of years. TUrns out that doing proper evals is important.\n\n## The obvious approach costs 23 megabytes[#](#the-obvious-approach-costs-23-megabytes)\n\nThe models that power [the Python post](/building-a-semantic-search-engine-in-250-lines-of-python/) (sentence-transformers like `all-MiniLM-L6-v2`\n\n) can, in fact, run in a browser. [Transformers.js](https://github.com/huggingface/transformers.js) will download an ONNX build of one and run it on WebAssembly. I benchmarked it, and on my laptop, the quantized model plus its runtime is **23.45 MB** over the wire, takes about two seconds to load and become available, and then embeds a query in ±18 ms.\n\nTwenty-three megabytes is a bit rich to embed a search box query, for a blog that has 14 posts. That is roughly a dozen high-resolution photos worth of bytes to download so we can have fancy search. It works, and for some applications it is completely worth it, but it is not something I want to inflict on someone who clicked through to read about bloom filters, especially not on a mobile connection.\n\nThe thing is, you do not need a transformer to embed a search query. You need a vector that’s good enough, and there is a much cheaper way to get one.\n\n## A static embedding model is a lookup table[#](#a-static-embedding-model-is-a-lookup-table)\n\nThe trick is a family of models called [model2vec](https://github.com/MinishLab/model2vec) (the specific ones are named “potion”). They are distilled from a real sentence-transformer, but the result is not a neural network. It is a table.\n\nHere is the model’s entire forward pass. Not a simplification — the whole thing, from the library’s source:\n\n```\nids = tokenize(text)          # split into subword token ids\nrows = embedding[ids]         # look up one vector per token\nvector = rows.mean(axis=0)    # average them\nvector = vector / norm(vector)  # normalise to unit length\n```\n\nThat is it. Tokenize, look up a row per token, average, normalize. There is no attention, no layers, no inference. “Running the model” is a handful of array lookups and an average. For `potion-base-8M`\n\n, the table is 29,528 tokens by 256 dimensions of float32 (about 30 MB) and it reaches [81% of MiniLM’s retrieval quality](https://huggingface.co/minishlab/potion-retrieval-32M) while being, definitionally, a dictionary lookup.\n\nThirty megabytes is still too much, but a table is a much friendlier thing to shrink than a transformer.\n\n## Building the index: chunking, and a cache that never earns its keep[#](#building-the-index-chunking-and-a-cache-that-never-earns-its-keep)\n\nEmbeddings are generated at build time. On my laptop, whenever I run `hugo`\n\n, [a Python script](https://github.com/bartdegoede/blog/blob/main/scripts/build_search_index.py) walks through the posts, strips out the front matter and the code blocks, splits each post into overlapping 600-character-ish chunks, embeds every chunk, and writes the vectors to a file the browser downloads when a user clicks the search input box.\n\nThe chunking matters more than I expected, because of that averaging step. A static embedding is the *mean* of its token vectors, and if you average an entire 15,000-character post into one vector, you get something that points at “generic English prose about software” and not much else. The rare, distinctive words (terms like `pydub`\n\nor `mmh3`\n\n) get drowned out by the hundreds of ordinary words around them. Chopping the post into smaller chunks helps keep those signals sharp. I’ll come back to this, because it turns out to be the key to why some models beat others.\n\nI also built an embedding cache, keyed on a hash of each chunk’s text, so that re-embedding only touches chunks that changed. This turned out to be a waste of time and tokens. Embedding every chunk of this blog is a few hundred lookups and an average; it takes about ten milliseconds. I built a cache to speed up an operation that is already pretty instantaneous (turns out 14 blog posts is not a lot of data). It would matter more for the MiniLM model, where embedding is a real neural network forward pass, but I’m not shipping that one. So the cache sits there, correct and pointless, and I’ve left it in as a reminder that overengineering is easy.\n\n## Shrinking the table: the model’s stopword list is hiding in plain sight[#](#shrinking-the-table-the-models-stopword-list-is-hiding-in-plain-sight)\n\nThe table is 30 MB because it is float32. Quantizing it to int8 makes it a quarter of the size. The catch is that you cannot just clip everything to the same scale, and understanding *why* was one of my favorite things I learned building this.\n\nLook at what the row magnitudes actually are. If you sort every token in `potion-base-8M`\n\nby the length of its vector, the shortest vectors (i.e. the ones closest to zero) are:\n\n```\na  .  ,  -  )  the  to  and  of  in\n```\n\nAnd the longest are:\n\n```\nturkmenistan  seychelles  guantanamo  hemingway  vanuatu\n```\n\nThis is not a coincidence and it isn’t noise either. **The model’s stopword list is its row magnitudes.** When you average token vectors together, a word with a tiny vector barely moves the result, and a word with a big vector dominates it. The model has learned, with no stopword list and no special-casing, that a word like “the” should contribute almost nothing and a word like “guantanamo” should contribute a lot. It’s kinda beautiful, and it’s sitting right there in the geometry.\n\nWhich is why quantization needs a *per row* scale: each token gets its own float32 multiplier, so that the relative magnitudes survive being crushed into int8. However, turns out that it didn’t matter much anyway.\n\nI measured it; I quantized the real table with a single global scale and checked how much it actually degraded the query vectors. The answer was: almost nothing. Aggregate cosine similarity against the original stayed at 0.9998. A global scale zeroes out exactly two rows in the entire 29,528-token vocabulary (`.`\n\nand `a`\n\n) which are precisely the two tokens the model had already decided contribute pretty much nothing. The mechanism is real; it just doesn’t matter for this particular model.\n\nI kept the per-row scales anyway, because they cost 118 KB out of 4 MB and they’re correct, but more like “cheap insurance” than “load-bearing”. The whole int8 table, per-row scales and all, reproduces the original float32 model to a cosine of **0.999958**. Good enough to ship a lookup table in the browser rather than a model.\n\n## WordPiece in eighty lines, and its three gotchas[#](#wordpiece-in-eighty-lines-and-its-three-gotchas)\n\nThe browser has the token table, but it still has to turn your typed query into token ids the same way we did in Python, so we can look up the right rows. That means reimplementing the BERT WordPiece tokenizer in JavaScript. It’s about eighty lines, and it has three gotchas that could each silently poison query vector:\n\n**No** BERT tokenizers normally wrap your text in special marker tokens. The`[CLS]`\n\n/`[SEP]`\n\n.`tokenizer.json`\n\nconfig even has a section describing how.`model2vec`\n\ndoesn’t use it, but calls the tokenizer with`add_special_tokens=False`\n\ninstead. Add the markers and you’re averaging in two vectors that shouldn’t be there.**Unknown tokens are** If a word isn’t in the vocabulary,*deleted*, not embedded.`model2vec`\n\ndrops it from the sequence entirely rather than substituting an`[UNK]`\n\nvector. So a query made entirely of gibberish produces an empty token list and a zero vector, and you have to handle that instead of dividing by zero. (In practice this is nearly unreachable; with a 29,528-token vocabulary, every single character is in the vocabulary, so the only way to trigger it is a word longer than 100 characters.)This one is a little nasty. The config says`\"strip_accents\": null`\n\nmeans accents*are*stripped.`strip_accents`\n\nis null, which reads like “off.” But in HuggingFace’s tokenizer library, a null value inherits from the`lowercase`\n\nsetting, which is on. So`café`\n\nbecomes`cafe`\n\n. If we’d copy the config literally, every accented query drifts.\n\nSome frustrations and a bunch of Claude generated test strings ( `pydub`\n\n, `café naïve`\n\n, `C++ vs C#`\n\n, `日本語のみ`\n\n) later, I’m reasonably confident the [JS tokenizer](https://github.com/bartdegoede/blog/blob/main/assets/js/semantic/tokenizer.js) matches the BERT WordPiece tokenizer.\n\n## Search is a few hundred dot products[#](#search-is-a-few-hundred-dot-products)\n\nWith the query embedded, search is almost anticlimactic. There are a few hundred chunk vectors (one per chunk of every post). Computing the cosine similarity against all of them is a few hundred dot products of 128 numbers each, tens of thousands of multiply-adds, which a browser on a reasonably modern machine does in a fraction of a millisecond. Then we group the chunks by post, take each post’s best-scoring chunk, and sort. On my machine (M1 MacBook Air) the entire query (tokenize, embed, score every chunk, rank) takes about **0.4 milliseconds**.\n\nIn a production setting, this is where you reach for approximate-nearest-neighbour indexes (HNSW and friends), and if you have a bazillion documents you should. With a few hundred, an ANN index would be slower than the brute-force loop and much larger on disk. It’s worth saying out loud because “vector search” in this case isn’t a complex database system,but just a `for`\n\nloop.\n\nThere’s one wrinkle worth knowing if you build one of these. The document vectors are stored as int8. Cosine similarity is invariant to a positive scale, and the document matrix uses one global scale, so the browser can dot a float32 query straight against the raw int8 bytes and get the right ranking without ever un-quantizing them. But `int8 × int8`\n\nin JavaScript overflows silently; a dot product whose true value is three million comes back as `-64`\n\n, no error, no warning. So it’s better to accumulate into a regular float. I mention it because this was a fun one to debug 1.\n\n## Two search engines that are each blind in a different way[#](#two-search-engines-that-are-each-blind-in-a-different-way)\n\nBefore adding semantic search, I figured I’d do the responsible thing and measured how bad the existing keyword search actually was, so I’d have a baseline to beat. I had Claude create a set of thirty test queries and split them into three kinds:\n\n**Exact tokens**:`pydub`\n\n,`lunr`\n\n,`mmh3`\n\n,`papermod`\n\n. Words that literally appear in a post.**Paraphrases**: “find documents by what they mean instead of which words they contain.” Concepts, in different words than the post uses.**Navigational**: “how do I add search to a static site.” Broad intent.\n\nKeyword search would surely ace the exact tokens and fail the paraphrases. Then I ran it, and it failed `pydub`\n\n.\n\n`pydub`\n\nis a Python library I’ve used in [a previous post](/use-google-cloud-text-to-speech-to-create-an-audio-version-of-your-blog-posts/). The word is right there in the text-to-speech post. And Fuse.js, the fuzzy-search library this blog uses, returned nothing. It turned out my configuration told Fuse to only score matches near the *start* of a field; a setting called `location`\n\n, with a `distance`\n\nwindow of 1000 characters. `pydub`\n\nfirst appears about 3,700 characters into that post, well outside the window, so as far as my search box was concerned it did not exist. The same was true for `mmh3`\n\n, and for any distinctive word that happened to appear deep in a long post. The words that *did* work only worked because they were in a title.\n\nMy keyword search had been quietly broken for years, and I only found out because I was about to benchmark against it. Derp.\n\nAnd the two engines turn out to be blind in exactly complementary ways:\n\n| keyword finds it | semantic finds it | |\n|---|---|---|\n`pydub` (exact token, out of vocabulary) | ✅ | ❌ |\n| “find documents by what they mean” (paraphrase) | ❌ | ✅ |\n\nKeyword search is perfect on the exact tokens and returns *literally nothing* for all ten paraphrases; not bad results, just zero results. Semantic search is the mirror image: it nails the paraphrases and stumbles on the proper nouns, because `pydub`\n\nisn’t in its vocabulary and gets shattered into meaningless subword fragments. Neither is better. They fail on disjoint sets.\n\n## Reciprocal Rank Fusion, or: how to merge different search result sets[#](#reciprocal-rank-fusion-or-how-to-merge-different-search-result-sets)\n\nIf each engine catches what the other misses, you want both. The problem is combining them. Keyword search returns a fuzzy-match *distance*; vector search returns a *cosine similarity*. These live on different scales that mean different things, and averaging them is meaningless; it’d be like adding a temperature to a weight.\n\nThe answer here is [Reciprocal Rank Fusion](https://medium.com/@devalshah1619/mathematical-intuition-behind-reciprocal-rank-fusion-rrf-explained-in-2-mins-002df0cc5e2a). Run queries against both indices, throw away the scores entirely and keep only the *ranks*. Each document’s “fused score” is the sum, over both engines, of `1 / (k + rank)`\n\n, with `k`\n\nconventionally 60. A document ranked first by one engine and unranked by the other scores `1/61`\n\n. A document ranked third by *both* scores `1/63 + 1/63`\n\n, which is larger. The `k`\n\nflattens the top of the curve so that “both engines quite liked this” beats “one engine loved it”, which is exactly what you want when one engine is blind to paraphrase and the other to proper nouns. It’s about ten lines of code, it has a single parameter with a sensible default, and it never has to compare the two incomparable scores. When you search this blog now, that’s what runs: keyword and semantic, fused by rank. This is easily extensible to many sources of ranked lists.\n\n## The benchmark: it’ll have to run on your hardware[#](#the-benchmark-itll-have-to-run-on-your-hardware)\n\nI benchmarked three query encoders: the model2vec lookup table I’ve been describing, MiniLM q8 through transformers.js, and [ternlight](https://github.com/soycaporal/ternlight), a clever BitNet-style ternary model compiled to WebAssembly that originally sent me down this whole path.\n\nThe widget below runs all three **in your browser, on your hardware, because that’s what matters for this anyway**. Each has its own Run button because one of them downloads 23 MB and you should get to decide whether you want to pay that on your connection. The download sizes are constants I measured (your browser can’t see the size of a cross-origin download that doesn’t send the right header); the timings are live on your device.\n\nOn my laptop the story is pretty clear: the lookup table is **4.2 MB and embeds a query in a third of a millisecond**; MiniLM is **23.5 MB and 18 milliseconds**; ternlight sits in between. And in retrieval quality, measured over those thirty labeled queries, the lookup table scores just as well as MiniLM. *Just as well*. At a fifth of the size and roughly fifty times the speed.\n\nCaveat on that claim, though; thirty queries is a small sample and I nearly fooled myself with it. Twice.\n\n## The metric couldn’t see anything[#](#the-metric-couldnt-see-anything)\n\nInitially, the headline metric was `recall@3`\n\n: did a relevant post make the top three? On a thirteen-post corpus, “top three of thirteen” is not a very high bar (random guessing clears it about a quarter of the time) and it turned out that almost every model cleared it on almost every query. Five different configurations tied at exactly 0.978. `Recall@3`\n\ncould not tell a 4 MB lookup table apart from a 23 MB transformer.\n\nWorse, it produced confident nonsense. Pure semantic search scored a *perfect* 1.000 on the exact-token queries; it apparently found `pydub`\n\nevery time. Except it didn’t:\n\n```\npydub  →  1. Free SSL on GitHub Pages       (wrong)\n          2. Let's Encrypt on GitHub Pages   (wrong)\n          3. Text-to-Speech with pydub       (correct)\n```\n\nThe right post is third, behind two completely unrelated ones, and `recall@3`\n\nscored that as a hit. The metric was rewarding the model for landing the right answer in a bucket wide enough that landing it there meant almost nothing. Switching to recall@**1** (“is the first result correct?”) separated the models more cleanly and told a different story. Lesson learned: a metric is worthless if the metric can’t see the thing you care about. I only caught it because the results looked *too* good.\n\n## What shipped, and how to steal it[#](#what-shipped-and-how-to-steal-it)\n\nThe search box on [the homepage](/) now runs keyword, semantic, and hybrid search, with a toggle so you can compare and watch them disagree. Type `pydub`\n\nand flip to semantic mode to see it get the answer wrong; flip to hybrid to see it get it right again. The whole thing is a 4 MB lookup table, a tiny document index, and about 300 lines of dependency-free JavaScript, lazy-loaded only when you focus the search box so the page itself pays nothing.\n\nThe build pipeline (the chunking, the quantization, the eval harness, all of it) is a Python package you can point at your own site:\n\n```\npip install static-site-search-eval\nsss-eval build --corpus content/post --outdir static/search \\\n  --model minishlab/potion-base-8M --dims 128 --chunk-size 600\n```\n\nThe [code is on GitHub](https://github.com/bartdegoede/static-site-search-eval), the thirty eval queries are in there too so you can try it out for yourself. If you run the benchmark above on a phone over cellular, you’ll likely feel the 23 MB, and understand why the lookup table was worth the trouble.\n\nAll time classic\n\n[https://www.destroyallsoftware.com/talks/wat](https://www.destroyallsoftware.com/talks/wat)[↩︎](#fnref:1)", "url": "https://wpnews.pro/news/semantic-hybrid-search-in-the-browser", "canonical_source": "https://bart.degoe.de/semantic-search-in-your-browser/", "published_at": "2026-07-11 13:04:35+00:00", "updated_at": "2026-07-11 13:35:04.130206+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-tools", "developer-tools"], "entities": ["Lunr.js", "Hugging Face", "Transformers.js", "ONNX", "WebAssembly", "PyTorch", "sentence-transformers", "model2vec"], "alternates": {"html": "https://wpnews.pro/news/semantic-hybrid-search-in-the-browser", "markdown": "https://wpnews.pro/news/semantic-hybrid-search-in-the-browser.md", "text": "https://wpnews.pro/news/semantic-hybrid-search-in-the-browser.txt", "jsonld": "https://wpnews.pro/news/semantic-hybrid-search-in-the-browser.jsonld"}}