I built a privacy-focused search engine from scratch (and learned why search is hard)

A developer built Slick, an independent, privacy-focused search engine from scratch, learning that search is far more complex than it appears. The engine combines traditional keyword matching with semantic search using embeddings, and relies on Elasticsearch for indexing while currently using DuckDuckGo's index as it matures.

I built Slick https://slicksearchhq.com because I wanted a search engine that actually felt like it belonged to the user. Not another search wrapper. Not another app that just sends your query somewhere else. A real search engine. At first, I thought: "How hard could it be?" You crawl some websites, put them in an index, add a search box, and you're done... right? Turns out, not even close. Search is one of those things that looks simple until you try to build it yourself. Slick is an independent, privacy-focused search engine that I'm building from scratch. The goal is simple: Give users more control over how they search the web. Features include: The project started as an experiment, but quickly became a deep dive into crawling, information retrieval, machine learning, and infrastructure. A search engine is useless without an index. The first major component I built was the crawler. It needs to: The goal isn't to blindly crawl the entire internet. The goal is building a useful, high-quality index. For indexing, Slick uses Elasticsearch. By default we use DuckDuckGo's index, as Slick is still maturing. Each document contains information like: The initial search system uses traditional information retrieval techniques like BM25. But keyword matching has a problem. A user doesn't always search using the exact words found on a page. For example: "how do I stop websites tracking me" and: "privacy tools that prevent online surveillance" mean almost the same thing. A keyword-only search engine sees two different queries. Humans don't. To improve this, Slick uses semantic search. Pages and queries can be converted into embeddings that represent meaning instead of just text. The ranking pipeline combines: The idea is to combine the strengths of old-school search with modern machine learning. Getting results is easy. Getting the right results is hard. A search engine can return thousands of pages that technically match a query. But users don't want thousands of pages. They want the one that answers their question. Ranking involves constantly asking: This is where most of the engineering effort goes. The biggest surprise was how much infrastructure search requires. The UI is honestly the easy part. The difficult parts are: Large search engines have entire teams dedicated to one small part of this. Building even a small one makes you appreciate how much work happens behind every search box. Slick is still actively being developed. Some areas I'm working on: The long-term goal is not to make "another Google clone." It's to explore what search looks like when users have more control. If you want to try it: https://slicksearchhq.com/ https://slicksearchhq.com/ The project is still early, so any feedback, suggestions, or ideas are greatly appreciated. Thanks for checking it out