# Building a Search Engine for 6,000 Personal Blogs > Source: > Published: 2026-08-03 00:00:00+00:00 TL;DR: Semsearch is an embedding-based search engine whose corpus is bootstrapped from [indieblog.page's blog list](https://indieblog.page/faq#data). It aims to make indie blogs more discoverable. Results are ranked by semantic relevance alone, without link popularity or SEO-derived signals. At the time of writing, the live instance is up at [semsearch.blog](https://semsearch.blog) with [roughly 6,000 unique blog sites, 154k pages, and 1.4B tokens](https://archive.today/2026.07.29-180627/https://semsearch.blog/status). The source code is at [github.com/yikerman/semantic-search](https://github.com/yikerman/semantic-search). The idea for this project emerged when I read about [The Great Blogging Collapse](https://danielstanica.com/posts/Great-Blogging-Collapse) ([HN discussion](https://news.ycombinator.com/item?id=48758802)). While the post itself has a couple of problems, its broader point resonated with me as a blogger: there are fewer ways to promote small and individual blogs. In the past, many visits began when Google led users to your blog. But Google, as shown in *The Great Blogging Collapse*, is doing so less and less for various reasons, such as the rise of AI summarization that completely eliminates visits to other sites. This aligns with other sources: [people (including me) prefer searching on Reddit](https://dkb.blog/p/google-search-is-dying) since Google's top results are cluttered by ads and SEO-optimized content farms. For example, a Google search for the old [Postgres vs MySQL](https://www.google.com/search?q=postgres+vs+mysql) debate returns mostly AWS and IBM articles that are of little use. (Interestingly, at the time of writing, it seems that Google has boosted Reddit's ranking and put it among the top results.) While the debate has largely settled, this would have been the perfect scenario to display some old-n-wise engineer writing about his experience with both in production, even if his webpage is styled uglier than [ The Motherfucking Website](https://motherfuckingwebsite.com/). Recent embedding models have made it practical to search by meaning rather than exact keywords (which also requires non-trivial tricks to work effectively), and vector databases have also proven to be practical and easy to use. While most existing embedding-based search systems are used to provide agentic LLMs with retrieval capabilities, I want to explore the possibility of using purely embedding-based search to help people discover indie blogs. I used FastAPI for the web app and [Postgres + VectorChord](https://hub.docker.com/r/tensorchord/vchord-suite) for the database. The set of blogs is bootstrapped from [indieblog.page's blog list](https://indieblog.page/faq#data). For each page discovered through either an RSS feed or a sitemap, I used [Trafilatura](https://github.com/adbar/trafilatura) to extract the text content, then chunked it into overlapping 384-token segments. For the embedding task, I chose [Qwen3-Embedding-4B](https://huggingface.co/Qwen/Qwen3-Embedding-4B) as the embedding model, since it topped the [MTEB leaderboard](http://archive.today/2026.08.01-143048/https://mteb-leaderboard.hf.space/models/Qwen/Qwen3-Embedding-4B) at the time of writing, is open source, and is relatively cheap to run. Every chunk is stored in the database as a 2,560-dimensional halfvec. At query time, Semsearch embeds the query once and runs designated retrievers. By default, only the dense retriever, based on the cosine similarity between the query embedding and chunk embeddings, is used, and the top 64 entries are returned. There is also an optional BM25 [1] retriever for both evaluation and combining results through RRF Deployment is a bit tricky, since the [HNSW index](https://github.com/pgvector/pgvector#hnsw) takes about 30 GB of RAM to load at the current scale, and VPS costs scale a lot with RAM. For reference, an `r6a.xlarge` instance on AWS costs $0.252/hr. I ended up getting a dedicated i7-6700 box from Hetzner's server auction for €55/mo. While the CPU is crappy by modern standards, it comes with 64 GB RAM and 2 × 512 GB NVMe SSDs, an absolute bargain in the RAMageddon era. I used [OpenRouter's API](https://openrouter.ai/qwen/qwen3-embedding-4b) to run the embedding model, since it is relatively cheap, namely $0.02/M tokens. Indexing the current corpus cost a modest $28.60, which is not bad. The latency and throughput are not ideal, though: I get roughly 6,000 tokens/sec on OpenRouter, and it bottlenecked the initial indexing process. So how do the results look? Going back to the [earlier query of Postgres vs MySQL](https://semsearch.blog/?q=Postgres+vs+MySQL&lang=en) ([archived](http://archive.today/2026.08.03-083542/https://semsearch.blog/?q=Postgres+vs+MySQL&lang=en)), the results are mixed and sit somewhere between useful, fun, and nonsense. After fiddling around, I found that it gives more insightful results when my query is an assertion a blogger might make in a post, and that longer queries resembling a sentence work better. Some extra interesting examples I tried: In conclusion, the search definitely works and offers a more personal side of the Internet from time to time. As expected, quality heavily depends on the corpus, while the search can only suggest relevance. The frontend can be confusing sometimes, since it only displays the title and the first few lines of a post, and the user may have a hard time locating the relevant chunk in the original post again. This is still an early proof of concept, and some quirks exist. A notable one is that the embedding model is not very robust to slightly nonstandard phrasing. For example, the query ["framework laptop review"](https://semsearch.blog/?q=framework+laptop+review&lang=en) ([archived](http://archive.today/2026.08.03-121718/https://semsearch.blog/?q=framework+laptop+review&lang=en)) returns gibberish results, while the properly capitalized query ["Framework Laptop review"](https://semsearch.blog/?q=Framework+Laptop+review&lang=en) ([archived](http://archive.today/2026.08.03-122117/https://semsearch.blog/?q=Framework+Laptop+review&lang=en)) returns relevant results. The website would also benefit from displaying relevant chunks in the search results as discussed above, and some community voting mechanism may be useful to filter out low-quality content (though it will introduce other problems). I plan to build a homelab and get a 3090 for self-hosting and further experiments, namely: