{"slug": "a-local-first-private-search-engine-for-your-markdown-knowledge-base", "title": "A local-first, private search engine for your Markdown knowledge base", "summary": "A developer released PMS, an open-source local-first hybrid search engine and web clipper for personal Markdown knowledge bases, built in Python on SQLite FTS5 plus sqlite-vec and ONNX fastembed embeddings. PMS is positioned as a CPU-friendly alternative to qmd, replacing multi-gigabyte local GGUF LLMs with lightweight ONNX embeddings, structure-aware Markdown chunking that never splits code fences and prepends H1–H3 breadcrumbs, and reciprocal rank fusion of BM25 keyword and semantic results with no reranker or query expansion. The project runs fully offline with no LLM API or Ollama dependency and ships both a CLI and a local UI.", "body_md": "# PMS\n\n**PMS** is a lightweight, local-first hybrid search engine and web clipper for personal Markdown knowledge bases, built in Python on top of **SQLite (`FTS5` + `sqlite-vec`)** and **ONNX (` fastembed`)**.\n\nIt is designed as a fast, minimal, CPU-friendly alternative to [`qmd`](https://github.com/tobi/qmd), trading multi-gigabyte local GGUF LLMs for lightweight ONNX embeddings, structure-aware Markdown chunking, and a hackable Python codebase. Repo here: [https://github.com/benjaminetienne/PMS](https://github.com/benjaminetienne/PMS)\n\n## Why PMS?\n\nWhy build PMS?\n\nI needed a local-first knowledge base & web clipper for Markdown notes (trafilatura → local .md files → hybrid search). It should work offline, and be able to read all my knowledge stored as markdown, so i can look up concepts, code snippets, etc. The advantage of it being local is that you can directly access your documents and keep sensitive data on your laptop. No LLM api here, no ollama, just lightweight models that can run in milliseconds on my laptop and still do the job.\n\nI came across qmd, and though a lot of good ideas were in it, i decided to go my own way for several reasons:\n\n- dependencies: qmd pulls a lot of dependencies to make it work. Bun, GGUF models, etc. I dont want to have ollama models, i want snappy search with lightweight models\n- no UI: the CLI is neat, but no one reads markdown files on a terminal. So yes qmd provides an API and a MCP, but this means having to build the UI. Shame there is no “qmd ui” to launch a local UI.\n- heavy pipeline: qmd does query expansion, reranking, etc. I am not sure it does bring a lot of value when documents are well written. I am ready to sacrifice a bit of accuracy in favor of lighter pipelines, and careful document crafting.\n\nPMS is not meant to be a production grade SaaS. It’s a local-first, lightweight and hackable code that anyone can adjust and tweak. Having a CLI and a UI also means its use can be versatile and offer many possibilities. It can also easily be ported to MCP with its friendly API.\n\n## How It Works\n\nThe tool is organized around collections which store documents. Documents are markdown files only. They can be fetched from urls with `trafilatura`, or written directly in any editor.\n\nDocuments are chunked to preserve the structure as much as possible:\n\n- headers are aggregated to create “breadcrumb title” that are appended to the chunk\n- code fences are never interrupted\n- chunks are optimized to stay within a certain limit. Chunking is done on titles, code fences, or line breaks depending on which is more optimal.\n\nOnce chunked, documents are added to a SQLite database that stores:\n\n- document metadata (path, title, collection, hash)\n- document chunks with a FTS5 search (BM25) used for keyword search\n\nEmbeddings are also computed for each chunk using the `fastemb` library.\n\nRetrieval follows a classic hybrid retrieval approach: keyword search + semantic search.\n\nA reciprocal rank fusion score is then given to the chunks to rank them.\n\nNo reranker is applied, and no summary is generated. But both can easily be added.\n\n```\nWeb URL / Local .md files\n        │\n        ▼\n┌────────────────────────────────────────────────────────┐\n│ 1. Ingestion (trafilatura / local disk)                │\n│    Saves Markdown with YAML frontmatter + SHA-256 hash │\n└────────────────────────────────────────────────────────┘\n        │\n        ▼\n┌────────────────────────────────────────────────────────┐\n│ 2. Structure-Aware Chunking (pms.chunk)           │\n│    • Respects code fences (never splits code blocks)   │\n│    • Splits on H1–H3 boundaries & paragraph breaks     │\n│    • Prepends [H1 > H2 > H3] breadcrumb to embed_text  │\n└────────────────────────────────────────────────────────┘\n        │\n        ├─────────────────────────────┐\n        ▼                             ▼\n┌──────────────────────┐    ┌────────────────────────────┐\n│ 3a. SQLite FTS5      │    │ 3b. FastEmbed + sqlite-vec │\n│ Porter stemmer BM25  │    │ BAAI/bge-small-en-v1.5     │\n│ full-text index      │    │ 384-d float32 cosine KNN   │\n└──────────────────────┘    └────────────────────────────┘\n        │                             │\n        └──────────────┬──────────────┘\n                       ▼\n┌────────────────────────────────────────────────────────┐\n│ 4. Reciprocal Rank Fusion (RRF, k=60)                  │\n│    Combines lexical (BM25) + semantic rankings         │\n└────────────────────────────────────────────────────────┘\n```\n\n## Python API\n\nThe main entrypoint is `KnowledgeBase`, which initializes the SQLite database (` knowledgebase.db`), loads `sqlite-vec`, and prepares the ONNX embedding model (` BAAI/bge-small-en-v1.5`).\n\n``` python\nfrom pms.kb import KnowledgeBase\n\n# 1. Initialize the knowledge base (SQLite + FastEmbed ONNX model)\nkb = KnowledgeBase()\n\n# 2. Create or load a collection (a directory of Markdown files)\nagents_col = kb.collections.create(\n    path=\"./knowledge/agents\",\n    name=\"agents\",\n    description=\"Articles and notes on AI agent architectures\",\n)\n\n# 3. Clip a web article directly into the collection as Markdown\nagents_col.add_from_url(\n    \"https://developers.googleblog.com/build-long-running-ai-agents-that-pause-resume-and-never-lose-context-with-adk/\"\n)\n\n# 4. Sync disk files with SQLite (chunks, embeds, and indexes new documents)\nkb.sync()\n\n# 5. Run hybrid search (BM25 + Vector + Reciprocal Rank Fusion)\nresults = kb.search(\"what is prompt context pollution\", k=5)           \nfor hit in results:                                                    \n    print(hit.text)                                                    \n    print(\"---\")\n```\n\nAnd that’s all. 5 steps to get your first document indexed and get results. Notice the `.sync()` method to ensure newly added docs are added to the database.\n\nCollections map a logical name to a local directory containing `.md` files. You can drop Markdown files into the folder manually or ingest URLs directly:\n\n```\n# List all registered collections\nfor col in kb.collections.list():\n    print(col.name, col.path, len(col.files))\n\n# Fetch an existing collection by name\ncol = kb.collections.get(\"agents\")\n\n# Ingest a URL into the collection (saved as YYYY-MM-DD_<slug>.md with frontmatter)\ncol.add_from_url(\"https://www.langchain.com/blog/introducing-ambient-agents\")\n\n# Inspect all .md files in the collection\nprint(col.files)\n\n# Delete a collection from the database\nkb.collections.delete(\"agents\")\n```\n\nTo ensure local content and DB content always stays up to date, we define a `check_sync_status()` that tracks newly added files, edited files and deleted files. To compare content, PMS computes SHA-256 content hashes for all `.md` files across your collections and compares them against the `documents` table in SQLite:\n\n```\n# Preview pending changes without modifying the database\nto_add, to_delete, to_update = kb.check_sync_status()\n# Prints: Sync status: +1 new, ~0 modified, -0 deleted\n\n# Index new documents into chunks, chunks_fts, and chunks_vec\nkb.sync()\n```\n\nWhile `kb.search(query, k=10)` returns the top fused chunk texts, you can also access `kb.retrieval` (`HybridRetrieval`) directly to inspect scores or compare BM25 against vector search:\n\n```\n# Get hits\nscored_results = kb.retrieval.rrf(\"how do agents handle idle time?\", k=5)\nfor hit in scored_results:\n    print(f\"[RRF: {hit.score:.4f}]\\n{hit.text}\\n\")\n```\n\nEach `Hit` object contains the rrf score, as well as the bm25 and vector search ranks. To get all candidates from both methods, we can inspect candidates via\n\n```\n# Compare raw candidate chunk IDs from BM25 vs. Vector search\nbm25_ids, vector_ids = kb.retrieval.candidates(\"what is prompt context pollution\", k=10)\nprint(\"BM25 top IDs:  \", bm25_ids)\nprint(\"Vector top IDs:\", vector_ids)\n```\n\nOr run standalone BM25 search (includes highlighted snippets)\n\n```\nbm25_rows = kb.retrieval._bm25_search(\"state machine checkpoint\", k=5)\nfor rowid, heading_path, text, bm25_score, snippet in bm25_rows:\n    print(f\"#{rowid} ({heading_path}): {snippet}\")\n```\n\nLikewise, for vector search:\n\n```\n# Run standalone KNN cosine vector search\nvec_rows = kb.retrieval._vector_search(\"how to survive server restarts\", k=5)\nfor rowid, text, distance in vec_rows:\n    print(f\"#{rowid} (cosine distance={distance:.4f})\")\n```\n\nAs the code is modular and hackable, it is also easy to inspect the chunking module and replace it with any custom chunker. The chunker is decoupled from the database and can be used, with or without the URL extractor, independently of the SQLite database:\n\n``` python\nfrom pms.extract import extract_from_url\nfrom pms.chunk import chunk_markdown\n\n# Extract clean Markdown + title from any web page\ndoc = extract_from_url(\"https://www.omnara.com/blog/what-is-an-async-agent-really\")\n\n# Split into structure-aware chunks\nchunks = chunk_markdown(\n    markdown=doc.content,\n    title=doc.title or \"\",\n    target_chars=2000,\n    overlap_lines=10,\n)\n\nfor c in chunks[:3]:\n    print(f\"Chunk #{c.seq} | Lines {c.start_line}-{c.end_line} | {c.heading_path}\")\n    print(c.embed_text[:200])\n    print(\"---\")\n```\n\n## Deepdives and lessons learned\n\n### BM25 with SQLite\n\nBuilding a BM25 search engine in SQLite is very easy and straightforward, since it is already built-in sqlite. Assuming we create a `chunks` table in SQLite, we can derive a virtual table used to do keyword search like this\n\n```\nCREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts USING fts5(\n                heading_path, \n                text, \n                content='chunks', \n                content_rowid='id',\n                tokenize='porter unicode61');\n```\n\nThe tokenizer is important to match “approximate” words. For example this is how it finds singular names even if spelled plural, or vice-versa. At query time, the query to find matches is simply\n\n```\nSELECT \n    rowid as id, \n    heading_path, \n    text, \n    bm25(chunks_fts) as bm25_score, \n    snippet(chunks_fts, 1, '<mark>', '</mark>', '...', 32) as snippet\nFROM chunks_fts \nWHERE chunks_fts MATCH ? ORDER BY bm25(chunks_fts)\n```\n\nThe last line is the key: `MATCH ?` will match the query, and `ORDER BY bm25` will return the scores in ascending order, which is fine because lower means better for BM25.\n\nIn practice, BM25 comes with a few caveats: in SQLite FTS5, passing what is AI uses an implicit AND (“what AND is AND AI”). If no document has these exact 3 words, it returns nothing.\n\nOne fix is to split the query in words and join them with the “OR” keyword: “what is AI” becomes “what OR is OR AI”. This way, BM25 will look for any of these 3 words.\n\nHowever there is still a drawback: the search will match any document that contains any of the three, including “what” and “is”, which are very common. There is no way to exclude stopwords directly in sqlite. Though BM25 relies on TF-IDF under the hood, a chunk that doesn’t mention “AI” at all can still rank #1 in BM25 just by repeating “what” and “is” many times.\n\n### Chunking\n\nChunking is an art, and though several methods exist, the following decisions are unavoidable:\n\n- which length ? too short and you miss the overall context, too long and details get drowned\n- overlap ? no overlap and you split related content in two, too much and you end up with chunks repeating themselves\n- rule-based, or llm-based ? Rule based is deterministic, LLM is not. Rule based can be explained, LLM cannot\n- etc.\n\nIn PMS, chunking is inspired from qmd’s chunking strategy. The idea is to define a maximum window length, inside which there is room for multiple split choices: heading, line breaks, code fences, lists. Each choice comes with a score, meaning it is for example better to split on heading vs a line break. This hierarchy of split criteria is supposed to yield more coherent outputs. In practice, it still depends a lot of the window length and the overlap. Empirically, it seems that windows around 1000 characters work quite good. But this is clearly a critical choice on which the performance of the search engine depends.\n\nThe good news: since we own the markdown, we can control the structure. For documents downloaded from a URL this might be more complicated, but for document i would write myself, writing clear markdown with nice headers, short paragraphs and well defined code fences is easy. Based on your personal preferences and writing style, you can adjust the window length and the overlap parameters in the chunking function\n\n### Vector search & embeddings\n\nAs said previously, chunking is an art, but so is choosing the right embedding model. Luckily, we have several options available when it comes to choosing an embedding model:\n\n- your own\n- open source models from Hugging Face\n- APIs like Vertex AI embeddings\n\nSince I wanted a local-first and CPU-friendly, the choice quickly ended up being HuggingFace models. But another question remained: how to balance performance and ressource utilization locally ? This ruled out models of several Gb memory, the sweetspot i was targeting being around several hundreds million parameters. I also went for specialized models instead of small general purpose LLMs like Qwen or SmolLM.\n\nThen came the question of the dependencies. I started with `google/embeddinggemma-2` using the `sentence-transformers` lib from HF, but this meant adding pytorch to the dependencies, lots of Gb to be downloaded, etc. I pivoted to using the ONNX format which does not require pytorch to be installed, saving a lot of time in both libs needed but also in memory when loading the model.\n\nThe `fastembed` python library turned out to provide exactly what i was looking for: small embeddings models with low memory footprint and using the ONNX format. I did not compare embedding models and chose the default one, `BAAI/bge-small-en-v1.5`, but this is definitely worth trying.\n\nSo in the end, how do both compare ?\n\n| Embedding | size on disk | Parameters | Loading (avg on 10) | Encoding (avg on 10) | \n|---|---|---|---|---|\n| BAAI/bge-small-en-v1.5 | 128Mb | 33M | ~0.2s | 4.1ms | \n| google/embeddinggemma-2 | 1.49Gb | 744M | ~3s | 38.6ms | \n\nIf I also consider the dependencies needed (torch, transformers, sentence-transformers, torchvision, etc), my choice is made.\n\nNow I had my embedding model, creating a “vector database” in sqlite was made easy with the sqlite-vec extension. Creating the vector table is also very straightforward:\n\n```\nCREATE VIRTUAL TABLE IF NOT EXISTS chunks_vec USING vec0(\n                collection TEXT PARTITION KEY, \n                embedding float[384] distance_metric=cosine);\n```\n\nNothing fancy here: cosine distance (can be changed), and an embedding size of 384 corresponding to the embedding model size. The only specificity is the collection partition: when the document database starts being big, it is a good idea to leave the door open for a faster search on a specific part of the DB, …or so I thought — more on that disaster below.\n\nIt is important, whenever doing vector search at scale to think “reduce search space, and then find neighbours”, and not do a full search followed by a filter. The second option might return empty results if the full search does not return neighbors from the given collection.\n\n### When 17 documents became a 374MB database\n\nI opened my knowledge base directory one day and noticed `knowledgebase.db` was 374MB. For 17 documents. 245 chunks. Something was very wrong — 245 embeddings of 384 float32 each is only about 0.4MB of actual numbers. Where were the other 373?\n\nThe vectors live in a sqlite-vec virtual table:\n\n```\nCREATE VIRTUAL TABLE chunks_vec USING vec0(\n    collection TEXT PARTITION KEY,\n    embedding float[384] distance_metric=cosine);\n```\n\nsqlite-vec doesn’t store one row per vector the way a normal table does. It packs vectors into fixed blocks of 1024, and each block is pre-allocated at full size regardless of how many vectors actually land in it: 1024 × 384 × 4 bytes ≈ 1.5MB per block.\n\n174 blocks for 245 vectors. Roughly one vector per 1.5MB block. 174 × 1.5MB ≈ 261MB, and the rest was fragmentation around it. The blocks should have held 1024 vectors each — all 245 of mine should have fit in a single block.\n\nThe culprit was that PARTITION KEY. I added it thinking ahead to scale:\n*partitioning vectors by collection lets you shrink the search space before doing KNN instead of filtering after*.\nGood instinct so far, but wrong execution :\nblocks are allocated per partition value, and a NULL partition\nvalue **doesn’t pack at all**. I reproduced it in ten lines:\n\n```\n# 100 inserts, NULL partition key, even in a single transaction\nfor i in range(100):\n    c.execute(\"INSERT INTO v(rowid, collection, embedding) VALUES (?,?,?)\",\n(i, None, vec()))\nc.execute(\"SELECT count(*) FROM v_vector_chunks00\").fetchone()  # -> 100 blocks\n```\n\nOne hundred vectors = one hundred 1.5MB blocks. With a real partition value, the same 100 vectors pack into one.\n\nAnd that’s exactly what had happened: 172 of my 245 vectors had been written with collection = NULL (legacy rows from before I wired the collection name into the vector insert. Each one had grabbed its own block. To make it worse, deleting a vector in sqlite-vec only flips a validity bit; it never frees the block. And VACUUM can’t reclaim them either, because to SQLite those shadow rows are perfectly live data.\n\nSo i decided to read every vector back out, drop the table, and reinsert into a fresh, compact set of blocks with a MANDATORY collection field. New database size: 9.9Mb. Still a lot for less than 300 chunks. The partition trick eats a LOT of space and honestly, for a corpus of this size, the game wasn’t worth the fight.\n\nFinal decision: remove the collections partition. Final DB size: 2.3Mb.\n\nIn a nutshell:\n\n| Schema | DB size | Vector blocks | \n|---|---|---|\n| Partitioned, with NULL-key bug | 374 MB | 174 | \n| Partitioned, repacked | 9.9 MB | 6 (one per collection) | \n| No partition key | 2.3 MB | 1 | \n\nThe lesson: partition keys are a scaling optimization, and at my scale I was paying the cost without ever collecting the benefit. sqlite-vec can brute-force 245 — or 25,000 — vectors fast enough that partitioning buys nothing but a 1.5MB-per-collection tax. I’d reach for it again if a single collection grew into the hundreds of thousands.\n\n### Reciprocal Rank Fusion\n\nSince BM25 search and Vector search return scores on different scales, averaging or adding scores would not make sense. The standard approach in hybrid retrival is to use RRF, and average ranks instead.\n\nThis is also a complex part where fine-tuning can have concrete consequences on the search performance. In the standard formula of RRF, a magical number of 60 is chosen to be added to the ranks:\n\n$$\\text{RRF}(d) = \\sum_{m \\in {\\text{BM25}, \\text{vec}}} \\frac{1}{k + \n\n\\text{rank}_m(d)}$$\n\nThis leads to situations where differences between hits can be very small even if their respective rankings are very different. Eg:\n\n- rank #1 gets 1/61 = 0.0164\n- rank #5 get 1/65 = 0.0154\n\nThat’s only a 6% difference, and can be wiped out easily if the other search yields different ranks\n\nIf we lower the `60` to a `10`, differences become more obvious:\n\n- rank #1 gets 1/11 = 0.091\n- rank #5 gets 1/15 = 0.067\n\nThat’s a 36% difference.\n\nSo the “magical number” of the RRF formula really does have an impact.\n\nThen there is also the weights we give to each retrieval. In the current formula, each get the same weight, but we can well decide to give more importance to vector search, or to BM25. No golden rule here unfortunately.\n\nI ended up choosing a “normalized” version of the score so it would be between 0 and 1 and easier to compare:\n\n$$\\text{RRF}*{\\text{norm}}(d) = \\sum*{m \\in {\\text{BM25}, \\text{vec}}} 0.\n5 \\times \\frac{k + 1}{k + \\text{rank}_m(d)}$$\n\n### Why no reranking or expanded queries\n\nqmd does provide an additional reranking step. To be honest, I haven’t tested it and was already happy with my results. Adding a reranking step with `fastembed` can easily be achieved using `TextCrossEncoder`.\n\nExpanded queries is something i don’t like very much, since it introduces another parameter which can influence the results. If expanded queries are not well formulated, they can downgrade the vector search. Plus i believe that if we introduce expanded queries for semantic search, we should be fair and use them for BM25 as well.\n\n## Conclusion\n\nThis little app is something i personally use to track notes, ideas, and archive my knowledge. Serves me daily, and i like the fact that i can search my own documents privately, without internet access, and get instant replies. The fact i can tweak models, rewrite documents and have full control on the retrieval is something i believe is genuinely useful and I hope this hackable codebase can be reused and extended by other uses.\n\nThis project was also a great opportunity to show that building local search engines can be achieved very easily at no cost, since no infra or no LLM apis are required.", "url": "https://wpnews.pro/news/a-local-first-private-search-engine-for-your-markdown-knowledge-base", "canonical_source": "https://benjaminetienne.com/blog/a-personal-markdown-search-engine/", "published_at": "2026-10-30 09:59:10+00:00", "updated_at": "2026-10-30 10:16:48.468323+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-search", "artificial-intelligence"], "entities": ["PMS", "qmd", "SQLite", "sqlite-vec", "ONNX", "fastembed", "trafilatura", "Benjamin Etienne"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-local-first-private-search-engine-for-your-markdown-knowledge-base", "markdown": "https://wpnews.pro/news/a-local-first-private-search-engine-for-your-markdown-knowledge-base.md", "text": "https://wpnews.pro/news/a-local-first-private-search-engine-for-your-markdown-knowledge-base.txt", "jsonld": "https://wpnews.pro/news/a-local-first-private-search-engine-for-your-markdown-knowledge-base.jsonld"}}