A local-first, private search engine for your Markdown knowledge base

A developer released PMS, an open-source local-first hybrid search engine and web clipper for personal Markdown knowledge bases, built in Python on SQLite FTS5 plus sqlite-vec and ONNX fastembed embeddings. PMS is positioned as a CPU-friendly alternative to qmd, replacing multi-gigabyte local GGUF LLMs with lightweight ONNX embeddings, structure-aware Markdown chunking that never splits code fences and prepends H1–H3 breadcrumbs, and reciprocal rank fusion of BM25 keyword and semantic results with no reranker or query expansion. The project runs fully offline with no LLM API or Ollama dependency and ships both a CLI and a local UI.

PMS PMS is a lightweight, local-first hybrid search engine and web clipper for personal Markdown knowledge bases, built in Python on top of SQLite FTS5 + sqlite-vec and ONNX fastembed . It is designed as a fast, minimal, CPU-friendly alternative to qmd https://github.com/tobi/qmd , trading multi-gigabyte local GGUF LLMs for lightweight ONNX embeddings, structure-aware Markdown chunking, and a hackable Python codebase. Repo here: https://github.com/benjaminetienne/PMS https://github.com/benjaminetienne/PMS Why PMS? Why build PMS? I needed a local-first knowledge base & web clipper for Markdown notes trafilatura → local .md files → hybrid search . It should work offline, and be able to read all my knowledge stored as markdown, so i can look up concepts, code snippets, etc. The advantage of it being local is that you can directly access your documents and keep sensitive data on your laptop. No LLM api here, no ollama, just lightweight models that can run in milliseconds on my laptop and still do the job. I came across qmd, and though a lot of good ideas were in it, i decided to go my own way for several reasons: - dependencies: qmd pulls a lot of dependencies to make it work. Bun, GGUF models, etc. I dont want to have ollama models, i want snappy search with lightweight models - no UI: the CLI is neat, but no one reads markdown files on a terminal. So yes qmd provides an API and a MCP, but this means having to build the UI. Shame there is no “qmd ui” to launch a local UI. - heavy pipeline: qmd does query expansion, reranking, etc. I am not sure it does bring a lot of value when documents are well written. I am ready to sacrifice a bit of accuracy in favor of lighter pipelines, and careful document crafting. PMS is not meant to be a production grade SaaS. It’s a local-first, lightweight and hackable code that anyone can adjust and tweak. Having a CLI and a UI also means its use can be versatile and offer many possibilities. It can also easily be ported to MCP with its friendly API. How It Works The tool is organized around collections which store documents. Documents are markdown files only. They can be fetched from urls with trafilatura , or written directly in any editor. Documents are chunked to preserve the structure as much as possible: - headers are aggregated to create “breadcrumb title” that are appended to the chunk - code fences are never interrupted - chunks are optimized to stay within a certain limit. Chunking is done on titles, code fences, or line breaks depending on which is more optimal. Once chunked, documents are added to a SQLite database that stores: - document metadata path, title, collection, hash - document chunks with a FTS5 search BM25 used for keyword search Embeddings are also computed for each chunk using the fastemb library. Retrieval follows a classic hybrid retrieval approach: keyword search + semantic search. A reciprocal rank fusion score is then given to the chunks to rank them. No reranker is applied, and no summary is generated. But both can easily be added. Web URL / Local .md files │ ▼ ┌────────────────────────────────────────────────────────┐ │ 1. Ingestion trafilatura / local disk │ │ Saves Markdown with YAML frontmatter + SHA-256 hash │ └────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────┐ │ 2. Structure-Aware Chunking pms.chunk │ │ • Respects code fences never splits code blocks │ │ • Splits on H1–H3 boundaries & paragraph breaks │ │ • Prepends H1 H2 H3 breadcrumb to embed text │ └────────────────────────────────────────────────────────┘ │ ├─────────────────────────────┐ ▼ ▼ ┌──────────────────────┐ ┌────────────────────────────┐ │ 3a. SQLite FTS5 │ │ 3b. FastEmbed + sqlite-vec │ │ Porter stemmer BM25 │ │ BAAI/bge-small-en-v1.5 │ │ full-text index │ │ 384-d float32 cosine KNN │ └──────────────────────┘ └────────────────────────────┘ │ │ └──────────────┬──────────────┘ ▼ ┌────────────────────────────────────────────────────────┐ │ 4. Reciprocal Rank Fusion RRF, k=60 │ │ Combines lexical BM25 + semantic rankings │ └────────────────────────────────────────────────────────┘ Python API The main entrypoint is KnowledgeBase , which initializes the SQLite database knowledgebase.db , loads sqlite-vec , and prepares the ONNX embedding model BAAI/bge-small-en-v1.5 . python from pms.kb import KnowledgeBase 1. Initialize the knowledge base SQLite + FastEmbed ONNX model kb = KnowledgeBase 2. Create or load a collection a directory of Markdown files agents col = kb.collections.create path="./knowledge/agents", name="agents", description="Articles and notes on AI agent architectures", 3. Clip a web article directly into the collection as Markdown agents col.add from url "https://developers.googleblog.com/build-long-running-ai-agents-that-pause-resume-and-never-lose-context-with-adk/" 4. Sync disk files with SQLite chunks, embeds, and indexes new documents kb.sync 5. Run hybrid search BM25 + Vector + Reciprocal Rank Fusion results = kb.search "what is prompt context pollution", k=5 for hit in results: print hit.text print "---" And that’s all. 5 steps to get your first document indexed and get results. Notice the .sync method to ensure newly added docs are added to the database. Collections map a logical name to a local directory containing .md files. You can drop Markdown files into the folder manually or ingest URLs directly: List all registered collections for col in kb.collections.list : print col.name, col.path, len col.files Fetch an existing collection by name col = kb.collections.get "agents" Ingest a URL into the collection saved as YYYY-MM-DD <slug .md with frontmatter col.add from url "https://www.langchain.com/blog/introducing-ambient-agents" Inspect all .md files in the collection print col.files Delete a collection from the database kb.collections.delete "agents" To ensure local content and DB content always stays up to date, we define a check sync status that tracks newly added files, edited files and deleted files. To compare content, PMS computes SHA-256 content hashes for all .md files across your collections and compares them against the documents table in SQLite: Preview pending changes without modifying the database to add, to delete, to update = kb.check sync status Prints: Sync status: +1 new, ~0 modified, -0 deleted Index new documents into chunks, chunks fts, and chunks vec kb.sync While kb.search query, k=10 returns the top fused chunk texts, you can also access kb.retrieval HybridRetrieval directly to inspect scores or compare BM25 against vector search: Get hits scored results = kb.retrieval.rrf "how do agents handle idle time?", k=5 for hit in scored results: print f" RRF: {hit.score:.4f} \n{hit.text}\n" Each Hit object contains the rrf score, as well as the bm25 and vector search ranks. To get all candidates from both methods, we can inspect candidates via Compare raw candidate chunk IDs from BM25 vs. Vector search bm25 ids, vector ids = kb.retrieval.candidates "what is prompt context pollution", k=10 print "BM25 top IDs: ", bm25 ids print "Vector top IDs:", vector ids Or run standalone BM25 search includes highlighted snippets bm25 rows = kb.retrieval. bm25 search "state machine checkpoint", k=5 for rowid, heading path, text, bm25 score, snippet in bm25 rows: print f" {rowid} {heading path} : {snippet}" Likewise, for vector search: Run standalone KNN cosine vector search vec rows = kb.retrieval. vector search "how to survive server restarts", k=5 for rowid, text, distance in vec rows: print f" {rowid} cosine distance={distance:.4f} " As the code is modular and hackable, it is also easy to inspect the chunking module and replace it with any custom chunker. The chunker is decoupled from the database and can be used, with or without the URL extractor, independently of the SQLite database: python from pms.extract import extract from url from pms.chunk import chunk markdown Extract clean Markdown + title from any web page doc = extract from url "https://www.omnara.com/blog/what-is-an-async-agent-really" Split into structure-aware chunks chunks = chunk markdown markdown=doc.content, title=doc.title or "", target chars=2000, overlap lines=10, for c in chunks :3 : print f"Chunk {c.seq} | Lines {c.start line}-{c.end line} | {c.heading path}" print c.embed text :200 print "---" Deepdives and lessons learned BM25 with SQLite Building a BM25 search engine in SQLite is very easy and straightforward, since it is already built-in sqlite. Assuming we create a chunks table in SQLite, we can derive a virtual table used to do keyword search like this CREATE VIRTUAL TABLE IF NOT EXISTS chunks fts USING fts5 heading path, text, content='chunks', content rowid='id', tokenize='porter unicode61' ; The tokenizer is important to match “approximate” words. For example this is how it finds singular names even if spelled plural, or vice-versa. At query time, the query to find matches is simply SELECT rowid as id, heading path, text, bm25 chunks fts as bm25 score, snippet chunks fts, 1, '<mark ', '</mark ', '...', 32 as snippet FROM chunks fts WHERE chunks fts MATCH ? ORDER BY bm25 chunks fts The last line is the key: MATCH ? will match the query, and ORDER BY bm25 will return the scores in ascending order, which is fine because lower means better for BM25. In practice, BM25 comes with a few caveats: in SQLite FTS5, passing what is AI uses an implicit AND “what AND is AND AI” . If no document has these exact 3 words, it returns nothing. One fix is to split the query in words and join them with the “OR” keyword: “what is AI” becomes “what OR is OR AI”. This way, BM25 will look for any of these 3 words. However there is still a drawback: the search will match any document that contains any of the three, including “what” and “is”, which are very common. There is no way to exclude stopwords directly in sqlite. Though BM25 relies on TF-IDF under the hood, a chunk that doesn’t mention “AI” at all can still rank 1 in BM25 just by repeating “what” and “is” many times. Chunking Chunking is an art, and though several methods exist, the following decisions are unavoidable: - which length ? too short and you miss the overall context, too long and details get drowned - overlap ? no overlap and you split related content in two, too much and you end up with chunks repeating themselves - rule-based, or llm-based ? Rule based is deterministic, LLM is not. Rule based can be explained, LLM cannot - etc. In PMS, chunking is inspired from qmd’s chunking strategy. The idea is to define a maximum window length, inside which there is room for multiple split choices: heading, line breaks, code fences, lists. Each choice comes with a score, meaning it is for example better to split on heading vs a line break. This hierarchy of split criteria is supposed to yield more coherent outputs. In practice, it still depends a lot of the window length and the overlap. Empirically, it seems that windows around 1000 characters work quite good. But this is clearly a critical choice on which the performance of the search engine depends. The good news: since we own the markdown, we can control the structure. For documents downloaded from a URL this might be more complicated, but for document i would write myself, writing clear markdown with nice headers, short paragraphs and well defined code fences is easy. Based on your personal preferences and writing style, you can adjust the window length and the overlap parameters in the chunking function Vector search & embeddings As said previously, chunking is an art, but so is choosing the right embedding model. Luckily, we have several options available when it comes to choosing an embedding model: - your own - open source models from Hugging Face - APIs like Vertex AI embeddings Since I wanted a local-first and CPU-friendly, the choice quickly ended up being HuggingFace models. But another question remained: how to balance performance and ressource utilization locally ? This ruled out models of several Gb memory, the sweetspot i was targeting being around several hundreds million parameters. I also went for specialized models instead of small general purpose LLMs like Qwen or SmolLM. Then came the question of the dependencies. I started with google/embeddinggemma-2 using the sentence-transformers lib from HF, but this meant adding pytorch to the dependencies, lots of Gb to be downloaded, etc. I pivoted to using the ONNX format which does not require pytorch to be installed, saving a lot of time in both libs needed but also in memory when loading the model. The fastembed python library turned out to provide exactly what i was looking for: small embeddings models with low memory footprint and using the ONNX format. I did not compare embedding models and chose the default one, BAAI/bge-small-en-v1.5 , but this is definitely worth trying. So in the end, how do both compare ? | Embedding | size on disk | Parameters | Loading avg on 10 | Encoding avg on 10 | |---|---|---|---|---| | BAAI/bge-small-en-v1.5 | 128Mb | 33M | ~0.2s | 4.1ms | | google/embeddinggemma-2 | 1.49Gb | 744M | ~3s | 38.6ms | If I also consider the dependencies needed torch, transformers, sentence-transformers, torchvision, etc , my choice is made. Now I had my embedding model, creating a “vector database” in sqlite was made easy with the sqlite-vec extension. Creating the vector table is also very straightforward: CREATE VIRTUAL TABLE IF NOT EXISTS chunks vec USING vec0 collection TEXT PARTITION KEY, embedding float 384 distance metric=cosine ; Nothing fancy here: cosine distance can be changed , and an embedding size of 384 corresponding to the embedding model size. The only specificity is the collection partition: when the document database starts being big, it is a good idea to leave the door open for a faster search on a specific part of the DB, …or so I thought — more on that disaster below. It is important, whenever doing vector search at scale to think “reduce search space, and then find neighbours”, and not do a full search followed by a filter. The second option might return empty results if the full search does not return neighbors from the given collection. When 17 documents became a 374MB database I opened my knowledge base directory one day and noticed knowledgebase.db was 374MB. For 17 documents. 245 chunks. Something was very wrong — 245 embeddings of 384 float32 each is only about 0.4MB of actual numbers. Where were the other 373? The vectors live in a sqlite-vec virtual table: CREATE VIRTUAL TABLE chunks vec USING vec0 collection TEXT PARTITION KEY, embedding float 384 distance metric=cosine ; sqlite-vec doesn’t store one row per vector the way a normal table does. It packs vectors into fixed blocks of 1024, and each block is pre-allocated at full size regardless of how many vectors actually land in it: 1024 × 384 × 4 bytes ≈ 1.5MB per block. 174 blocks for 245 vectors. Roughly one vector per 1.5MB block. 174 × 1.5MB ≈ 261MB, and the rest was fragmentation around it. The blocks should have held 1024 vectors each — all 245 of mine should have fit in a single block. The culprit was that PARTITION KEY. I added it thinking ahead to scale: partitioning vectors by collection lets you shrink the search space before doing KNN instead of filtering after . Good instinct so far, but wrong execution : blocks are allocated per partition value, and a NULL partition value doesn’t pack at all . I reproduced it in ten lines: 100 inserts, NULL partition key, even in a single transaction for i in range 100 : c.execute "INSERT INTO v rowid, collection, embedding VALUES ?,?,? ", i, None, vec c.execute "SELECT count FROM v vector chunks00" .fetchone - 100 blocks One hundred vectors = one hundred 1.5MB blocks. With a real partition value, the same 100 vectors pack into one. And that’s exactly what had happened: 172 of my 245 vectors had been written with collection = NULL legacy rows from before I wired the collection name into the vector insert. Each one had grabbed its own block. To make it worse, deleting a vector in sqlite-vec only flips a validity bit; it never frees the block. And VACUUM can’t reclaim them either, because to SQLite those shadow rows are perfectly live data. So i decided to read every vector back out, drop the table, and reinsert into a fresh, compact set of blocks with a MANDATORY collection field. New database size: 9.9Mb. Still a lot for less than 300 chunks. The partition trick eats a LOT of space and honestly, for a corpus of this size, the game wasn’t worth the fight. Final decision: remove the collections partition. Final DB size: 2.3Mb. In a nutshell: | Schema | DB size | Vector blocks | |---|---|---| | Partitioned, with NULL-key bug | 374 MB | 174 | | Partitioned, repacked | 9.9 MB | 6 one per collection | | No partition key | 2.3 MB | 1 | The lesson: partition keys are a scaling optimization, and at my scale I was paying the cost without ever collecting the benefit. sqlite-vec can brute-force 245 — or 25,000 — vectors fast enough that partitioning buys nothing but a 1.5MB-per-collection tax. I’d reach for it again if a single collection grew into the hundreds of thousands. Reciprocal Rank Fusion Since BM25 search and Vector search return scores on different scales, averaging or adding scores would not make sense. The standard approach in hybrid retrival is to use RRF, and average ranks instead. This is also a complex part where fine-tuning can have concrete consequences on the search performance. In the standard formula of RRF, a magical number of 60 is chosen to be added to the ranks: $$\text{RRF} d = \sum {m \in {\text{BM25}, \text{vec}}} \frac{1}{k + \text{rank} m d }$$ This leads to situations where differences between hits can be very small even if their respective rankings are very different. Eg: - rank 1 gets 1/61 = 0.0164 - rank 5 get 1/65 = 0.0154 That’s only a 6% difference, and can be wiped out easily if the other search yields different ranks If we lower the 60 to a 10 , differences become more obvious: - rank 1 gets 1/11 = 0.091 - rank 5 gets 1/15 = 0.067 That’s a 36% difference. So the “magical number” of the RRF formula really does have an impact. Then there is also the weights we give to each retrieval. In the current formula, each get the same weight, but we can well decide to give more importance to vector search, or to BM25. No golden rule here unfortunately. I ended up choosing a “normalized” version of the score so it would be between 0 and 1 and easier to compare: $$\text{RRF} {\text{norm}} d = \sum {m \in {\text{BM25}, \text{vec}}} 0. 5 \times \frac{k + 1}{k + \text{rank} m d }$$ Why no reranking or expanded queries qmd does provide an additional reranking step. To be honest, I haven’t tested it and was already happy with my results. Adding a reranking step with fastembed can easily be achieved using TextCrossEncoder . Expanded queries is something i don’t like very much, since it introduces another parameter which can influence the results. If expanded queries are not well formulated, they can downgrade the vector search. Plus i believe that if we introduce expanded queries for semantic search, we should be fair and use them for BM25 as well. Conclusion This little app is something i personally use to track notes, ideas, and archive my knowledge. Serves me daily, and i like the fact that i can search my own documents privately, without internet access, and get instant replies. The fact i can tweak models, rewrite documents and have full control on the retrieval is something i believe is genuinely useful and I hope this hackable codebase can be reused and extended by other uses. This project was also a great opportunity to show that building local search engines can be achieved very easily at no cost, since no infra or no LLM apis are required.