PMS is a lightweight, local-first hybrid search engine and web clipper for personal Markdown knowledge bases, built in Python on top of SQLite (FTS5 + sqlite-vec) and ONNX ( fastembed).
It is designed as a fast, minimal, CPU-friendly alternative to qmd, trading multi-gigabyte local GGUF LLMs for lightweight ONNX embeddings, structure-aware Markdown chunking, and a hackable Python codebase. Repo here: https://github.com/benjaminetienne/PMS
Why PMS? #
Why build PMS?
I needed a local-first knowledge base & web clipper for Markdown notes (trafilatura → local .md files → hybrid search). It should work offline, and be able to read all my knowledge stored as markdown, so i can look up concepts, code snippets, etc. The advantage of it being local is that you can directly access your documents and keep sensitive data on your laptop. No LLM api here, no ollama, just lightweight models that can run in milliseconds on my laptop and still do the job.
I came across qmd, and though a lot of good ideas were in it, i decided to go my own way for several reasons:
- dependencies: qmd pulls a lot of dependencies to make it work. Bun, GGUF models, etc. I dont want to have ollama models, i want snappy search with lightweight models
- no UI: the CLI is neat, but no one reads markdown files on a terminal. So yes qmd provides an API and a MCP, but this means having to build the UI. Shame there is no “qmd ui” to launch a local UI.
- heavy pipeline: qmd does query expansion, reranking, etc. I am not sure it does bring a lot of value when documents are well written. I am ready to sacrifice a bit of accuracy in favor of lighter pipelines, and careful document crafting.
PMS is not meant to be a production grade SaaS. It’s a local-first, lightweight and hackable code that anyone can adjust and tweak. Having a CLI and a UI also means its use can be versatile and offer many possibilities. It can also easily be ported to MCP with its friendly API.
How It Works #
The tool is organized around collections which store documents. Documents are markdown files only. They can be fetched from urls with trafilatura, or written directly in any editor.
Documents are chunked to preserve the structure as much as possible:
- headers are aggregated to create “breadcrumb title” that are appended to the chunk
- code fences are never interrupted
- chunks are optimized to stay within a certain limit. Chunking is done on titles, code fences, or line breaks depending on which is more optimal.
Once chunked, documents are added to a SQLite database that stores:
- document metadata (path, title, collection, hash)
- document chunks with a FTS5 search (BM25) used for keyword search
Embeddings are also computed for each chunk using the fastemb library.
Retrieval follows a classic hybrid retrieval approach: keyword search + semantic search.
A reciprocal rank fusion score is then given to the chunks to rank them.
No reranker is applied, and no summary is generated. But both can easily be added.
Web URL / Local .md files
│
▼
┌────────────────────────────────────────────────────────┐
│ 1. Ingestion (trafilatura / local disk) │
│ Saves Markdown with YAML frontmatter + SHA-256 hash │
└────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ 2. Structure-Aware Chunking (pms.chunk) │
│ • Respects code fences (never splits code blocks) │
│ • Splits on H1–H3 boundaries & paragraph breaks │
│ • Prepends [H1 > H2 > H3] breadcrumb to embed_text │
└────────────────────────────────────────────────────────┘
│
├─────────────────────────────┐
▼ ▼
┌──────────────────────┐ ┌────────────────────────────┐
│ 3a. SQLite FTS5 │ │ 3b. FastEmbed + sqlite-vec │
│ Porter stemmer BM25 │ │ BAAI/bge-small-en-v1.5 │
│ full-text index │ │ 384-d float32 cosine KNN │
└──────────────────────┘ └────────────────────────────┘
│ │
└──────────────┬──────────────┘
▼
┌────────────────────────────────────────────────────────┐
│ 4. Reciprocal Rank Fusion (RRF, k=60) │
│ Combines lexical (BM25) + semantic rankings │
└────────────────────────────────────────────────────────┘
Python API #
The main entrypoint is KnowledgeBase, which initializes the SQLite database ( knowledgebase.db), loads sqlite-vec, and prepares the ONNX embedding model ( BAAI/bge-small-en-v1.5).
from pms.kb import KnowledgeBase
kb = KnowledgeBase()
agents_col = kb.collections.create(
path="./knowledge/agents",
name="agents",
description="Articles and notes on AI agent architectures",
)
agents_col.add_from_url(
"https://developers.googleblog.com/build-long-running-ai-agents-that--resume-and-never-lose-context-with-adk/"
)
kb.sync()
results = kb.search("what is prompt context pollution", k=5)
for hit in results:
print(hit.text)
print("---")
And that’s all. 5 steps to get your first document indexed and get results. Notice the .sync() method to ensure newly added docs are added to the database.
Collections map a logical name to a local directory containing .md files. You can drop Markdown files into the folder manually or ingest URLs directly:
for col in kb.collections.list():
print(col.name, col.path, len(col.files))
col = kb.collections.get("agents")
col.add_from_url("https://www.langchain.com/blog/introducing-ambient-agents")
print(col.files)
kb.collections.delete("agents")
To ensure local content and DB content always stays up to date, we define a check_sync_status() that tracks newly added files, edited files and deleted files. To compare content, PMS computes SHA-256 content hashes for all .md files across your collections and compares them against the documents table in SQLite:
to_add, to_delete, to_update = kb.check_sync_status()
kb.sync()
While kb.search(query, k=10) returns the top fused chunk texts, you can also access kb.retrieval (HybridRetrieval) directly to inspect scores or compare BM25 against vector search:
scored_results = kb.retrieval.rrf("how do agents handle idle time?", k=5)
for hit in scored_results:
print(f"[RRF: {hit.score:.4f}]\n{hit.text}\n")
Each Hit object contains the rrf score, as well as the bm25 and vector search ranks. To get all candidates from both methods, we can inspect candidates via
bm25_ids, vector_ids = kb.retrieval.candidates("what is prompt context pollution", k=10)
print("BM25 top IDs: ", bm25_ids)
print("Vector top IDs:", vector_ids)
Or run standalone BM25 search (includes highlighted snippets)
bm25_rows = kb.retrieval._bm25_search("state machine checkpoint", k=5)
for rowid, heading_path, text, bm25_score, snippet in bm25_rows:
print(f"#{rowid} ({heading_path}): {snippet}")
Likewise, for vector search:
vec_rows = kb.retrieval._vector_search("how to survive server restarts", k=5)
for rowid, text, distance in vec_rows:
print(f"#{rowid} (cosine distance={distance:.4f})")
As the code is modular and hackable, it is also easy to inspect the chunking module and replace it with any custom chunker. The chunker is decoupled from the database and can be used, with or without the URL extractor, independently of the SQLite database:
from pms.extract import extract_from_url
from pms.chunk import chunk_markdown
doc = extract_from_url("https://www.omnara.com/blog/what-is-an-async-agent-really")
chunks = chunk_markdown(
markdown=doc.content,
title=doc.title or "",
target_chars=2000,
overlap_lines=10,
)
for c in chunks[:3]:
print(f"Chunk #{c.seq} | Lines {c.start_line}-{c.end_line} | {c.heading_path}")
print(c.embed_text[:200])
print("---")
Deepdives and lessons learned #
BM25 with SQLite
Building a BM25 search engine in SQLite is very easy and straightforward, since it is already built-in sqlite. Assuming we create a chunks table in SQLite, we can derive a virtual table used to do keyword search like this
CREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts USING fts5(
heading_path,
text,
content='chunks',
content_rowid='id',
tokenize='porter unicode61');
The tokenizer is important to match “approximate” words. For example this is how it finds singular names even if spelled plural, or vice-versa. At query time, the query to find matches is simply
SELECT
rowid as id,
heading_path,
text,
bm25(chunks_fts) as bm25_score,
snippet(chunks_fts, 1, '<mark>', '</mark>', '...', 32) as snippet
FROM chunks_fts
WHERE chunks_fts MATCH ? ORDER BY bm25(chunks_fts)
The last line is the key: MATCH ? will match the query, and ORDER BY bm25 will return the scores in ascending order, which is fine because lower means better for BM25.
In practice, BM25 comes with a few caveats: in SQLite FTS5, passing what is AI uses an implicit AND (“what AND is AND AI”). If no document has these exact 3 words, it returns nothing.
One fix is to split the query in words and join them with the “OR” keyword: “what is AI” becomes “what OR is OR AI”. This way, BM25 will look for any of these 3 words.
However there is still a drawback: the search will match any document that contains any of the three, including “what” and “is”, which are very common. There is no way to exclude stopwords directly in sqlite. Though BM25 relies on TF-IDF under the hood, a chunk that doesn’t mention “AI” at all can still rank #1 in BM25 just by repeating “what” and “is” many times.
Chunking
Chunking is an art, and though several methods exist, the following decisions are unavoidable:
- which length ? too short and you miss the overall context, too long and details get drowned
- overlap ? no overlap and you split related content in two, too much and you end up with chunks repeating themselves
- rule-based, or llm-based ? Rule based is deterministic, LLM is not. Rule based can be explained, LLM cannot
- etc.
In PMS, chunking is inspired from qmd’s chunking strategy. The idea is to define a maximum window length, inside which there is room for multiple split choices: heading, line breaks, code fences, lists. Each choice comes with a score, meaning it is for example better to split on heading vs a line break. This hierarchy of split criteria is supposed to yield more coherent outputs. In practice, it still depends a lot of the window length and the overlap. Empirically, it seems that windows around 1000 characters work quite good. But this is clearly a critical choice on which the performance of the search engine depends.
The good news: since we own the markdown, we can control the structure. For documents downloaded from a URL this might be more complicated, but for document i would write myself, writing clear markdown with nice headers, short paragraphs and well defined code fences is easy. Based on your personal preferences and writing style, you can adjust the window length and the overlap parameters in the chunking function
Vector search & embeddings
As said previously, chunking is an art, but so is choosing the right embedding model. Luckily, we have several options available when it comes to choosing an embedding model:
- your own
- open source models from Hugging Face
- APIs like Vertex AI embeddings
Since I wanted a local-first and CPU-friendly, the choice quickly ended up being HuggingFace models. But another question remained: how to balance performance and ressource utilization locally ? This ruled out models of several Gb memory, the sweetspot i was targeting being around several hundreds million parameters. I also went for specialized models instead of small general purpose LLMs like Qwen or SmolLM.
Then came the question of the dependencies. I started with google/embeddinggemma-2 using the sentence-transformers lib from HF, but this meant adding pytorch to the dependencies, lots of Gb to be downloaded, etc. I pivoted to using the ONNX format which does not require pytorch to be installed, saving a lot of time in both libs needed but also in memory when the model.
The fastembed python library turned out to provide exactly what i was looking for: small embeddings models with low memory footprint and using the ONNX format. I did not compare embedding models and chose the default one, BAAI/bge-small-en-v1.5, but this is definitely worth trying.
So in the end, how do both compare ?
| Embedding | size on disk | Parameters | (avg on 10) | Encoding (avg on 10) |
|---|---|---|---|---|
| BAAI/bge-small-en-v1.5 | 128Mb | 33M | ~0.2s | 4.1ms |
| google/embeddinggemma-2 | 1.49Gb | 744M | ~3s | 38.6ms |
If I also consider the dependencies needed (torch, transformers, sentence-transformers, torchvision, etc), my choice is made.
Now I had my embedding model, creating a “vector database” in sqlite was made easy with the sqlite-vec extension. Creating the vector table is also very straightforward:
CREATE VIRTUAL TABLE IF NOT EXISTS chunks_vec USING vec0(
collection TEXT PARTITION KEY,
embedding float[384] distance_metric=cosine);
Nothing fancy here: cosine distance (can be changed), and an embedding size of 384 corresponding to the embedding model size. The only specificity is the collection partition: when the document database starts being big, it is a good idea to leave the door open for a faster search on a specific part of the DB, …or so I thought — more on that disaster below.
It is important, whenever doing vector search at scale to think “reduce search space, and then find neighbours”, and not do a full search followed by a filter. The second option might return empty results if the full search does not return neighbors from the given collection.
When 17 documents became a 374MB database
I opened my knowledge base directory one day and noticed knowledgebase.db was 374MB. For 17 documents. 245 chunks. Something was very wrong — 245 embeddings of 384 float32 each is only about 0.4MB of actual numbers. Where were the other 373?
The vectors live in a sqlite-vec virtual table:
CREATE VIRTUAL TABLE chunks_vec USING vec0(
collection TEXT PARTITION KEY,
embedding float[384] distance_metric=cosine);
sqlite-vec doesn’t store one row per vector the way a normal table does. It packs vectors into fixed blocks of 1024, and each block is pre-allocated at full size regardless of how many vectors actually land in it: 1024 × 384 × 4 bytes ≈ 1.5MB per block.
174 blocks for 245 vectors. Roughly one vector per 1.5MB block. 174 × 1.5MB ≈ 261MB, and the rest was fragmentation around it. The blocks should have held 1024 vectors each — all 245 of mine should have fit in a single block.
The culprit was that PARTITION KEY. I added it thinking ahead to scale: partitioning vectors by collection lets you shrink the search space before doing KNN instead of filtering after. Good instinct so far, but wrong execution : blocks are allocated per partition value, and a NULL partition value doesn’t pack at all. I reproduced it in ten lines:
for i in range(100):
c.execute("INSERT INTO v(rowid, collection, embedding) VALUES (?,?,?)",
(i, None, vec()))
c.execute("SELECT count(*) FROM v_vector_chunks00").fetchone() # -> 100 blocks
One hundred vectors = one hundred 1.5MB blocks. With a real partition value, the same 100 vectors pack into one.
And that’s exactly what had happened: 172 of my 245 vectors had been written with collection = NULL (legacy rows from before I wired the collection name into the vector insert. Each one had grabbed its own block. To make it worse, deleting a vector in sqlite-vec only flips a validity bit; it never frees the block. And VACUUM can’t reclaim them either, because to SQLite those shadow rows are perfectly live data.
So i decided to read every vector back out, drop the table, and reinsert into a fresh, compact set of blocks with a MANDATORY collection field. New database size: 9.9Mb. Still a lot for less than 300 chunks. The partition trick eats a LOT of space and honestly, for a corpus of this size, the game wasn’t worth the fight.
Final decision: remove the collections partition. Final DB size: 2.3Mb.
In a nutshell:
| Schema | DB size | Vector blocks |
|---|---|---|
| Partitioned, with NULL-key bug | 374 MB | 174 |
| Partitioned, repacked | 9.9 MB | 6 (one per collection) |
| No partition key | 2.3 MB | 1 |
The lesson: partition keys are a scaling optimization, and at my scale I was paying the cost without ever collecting the benefit. sqlite-vec can brute-force 245 — or 25,000 — vectors fast enough that partitioning buys nothing but a 1.5MB-per-collection tax. I’d reach for it again if a single collection grew into the hundreds of thousands.
Reciprocal Rank Fusion
Since BM25 search and Vector search return scores on different scales, averaging or adding scores would not make sense. The standard approach in hybrid retrival is to use RRF, and average ranks instead.
This is also a complex part where fine-tuning can have concrete consequences on the search performance. In the standard formula of RRF, a magical number of 60 is chosen to be added to the ranks:
$$\text{RRF}(d) = \sum_{m \in {\text{BM25}, \text{vec}}} \frac{1}{k +
\text{rank}_m(d)}$$
This leads to situations where differences between hits can be very small even if their respective rankings are very different. Eg:
- rank #1 gets 1/61 = 0.0164
- rank #5 get 1/65 = 0.0154
That’s only a 6% difference, and can be wiped out easily if the other search yields different ranks
If we lower the 60 to a 10, differences become more obvious:
- rank #1 gets 1/11 = 0.091
- rank #5 gets 1/15 = 0.067
That’s a 36% difference.
So the “magical number” of the RRF formula really does have an impact.
Then there is also the weights we give to each retrieval. In the current formula, each get the same weight, but we can well decide to give more importance to vector search, or to BM25. No golden rule here unfortunately.
I ended up choosing a “normalized” version of the score so it would be between 0 and 1 and easier to compare:
$$\text{RRF}{\text{norm}}(d) = \sum{m \in {\text{BM25}, \text{vec}}} 0. 5 \times \frac{k + 1}{k + \text{rank}_m(d)}$$
Why no reranking or expanded queries
qmd does provide an additional reranking step. To be honest, I haven’t tested it and was already happy with my results. Adding a reranking step with fastembed can easily be achieved using TextCrossEncoder.
Expanded queries is something i don’t like very much, since it introduces another parameter which can influence the results. If expanded queries are not well formulated, they can downgrade the vector search. Plus i believe that if we introduce expanded queries for semantic search, we should be fair and use them for BM25 as well.
Conclusion #
This little app is something i personally use to track notes, ideas, and archive my knowledge. Serves me daily, and i like the fact that i can search my own documents privately, without internet access, and get instant replies. The fact i can tweak models, rewrite documents and have full control on the retrieval is something i believe is genuinely useful and I hope this hackable codebase can be reused and extended by other uses.
This project was also a great opportunity to show that building local search engines can be achieved very easily at no cost, since no infra or no LLM apis are required.