What’s the SQLite equivalent for full-text and vector search?

An embedded search engine is a library that runs inside your application’s process and keeps its index in local files, the way SQLite does for relational data. There is no server to deploy, no network hop on a query, and it works offline. For full-text search alone, SQLite’s own FTS5 extension already is that library. The question gets harder when the same application also needs vector search over embeddings, both kinds fused in one query, and SQL over the results.

This page compares the in-process options on those four jobs, and says when each one fits. It compares capabilities only, not speed.

Why search moves inside the application #

• Desktop and local-first apps. A notes app, an email client or a document tool has to search the user’s data on their machine, often offline. Shipping a search server with a desktop app is not realistic.
• Agents and RAG. An agent issues many retrievals per task. A library in the same process answers each one without a network round trip, and the whole thing runs in a notebook or a test without infrastructure.
• Edge and single-tenant deployments. One process per customer or per device, each with its own small index, is simpler as a file than as a cluster.

The trade-off is the one SQLite has always had. A library shares the application’s memory and CPU, and writes to one index go through one writer at a time. Reads scale with the processes that open the files. When many services write to the same large corpus and query it at a high sustained rate, a dedicated search server or vector database is the better shape.

How the embedded options compare #

All five are open source. Checked against each project’s documentation and release notes in September 2026. These projects move quickly, so check the current docs for the feature you depend on.

When is each one the right choice? #

• Infino when the application needs keyword, vector and SQL search over the same rows, fused in one query, with the data kept as Parquet on local disk or in object storage. Three limits today: it runs on macOS and Linux, not Windows. You bring your own embeddings. And it is built for batched writes, not a row at a time (see below).
• SQLite with FTS5 when the app needs keyword search only. It ships with most SQLite builds, so it is often already in the app, on every platform SQLite runs on.
• SQLite with sqlite-vec when the app already stores its data in SQLite and the vector corpus is small enough for exact search. The project says it is pre-v1, and its approximate indexes are still in alpha releases.
• DuckDB with fts and vss when the workload is analytics first and search second, on data already in DuckDB. Both extensions carry caveats: the vector index is experimental, and the full-text index has to be rebuilt after the table changes.
• LanceDB when the app works with vectors and multimodal data and wants built-in hybrid search, and filters are enough rather than full SQL.
• Chroma for the quickest start on a vector-only prototype, with Chroma Cloud when it needs ranked keyword and hybrid search.

For keyword search inside a Rust program, Tantivy is a full-text library in the style of Lucene. It ranks with BM25.

What this looks like in Infino #

Infino is an open source retrieval library. A table lives in a folder of superfiles, next to a small manifest that says which of them are live. A superfile is a search index that is a valid Parquet file: the data and the BM25 and vector indexes in one file. Point connect at a local path for a desktop app, or at an s3:// URI when the same table should live in a bucket. The code does not change.

import infino
import pyarrow as pa

db = infino.connect("./notes")   # a folder on local disk, no server

notes = db.create_table(
    "notes",
    pa.schema([
        pa.field("folder", pa.large_utf8(), nullable=False),
        pa.field("body", pa.large_utf8(), nullable=False),
        pa.field("embedding", pa.list_(pa.float32(), 384), nullable=False),
    ]),
    infino.IndexSpec().fts("body").vector("embedding", 384, "cosine"),
)
notes.append(rows)   # a batch of your text and embeddings: one commit

hits = notes.hybrid_search("body", "INV-2291", "embedding", query_vec, 10,
                           projection=["body"])

db.query_sql(f"""
    SELECT folder, COUNT(*) AS hits
    FROM hybrid_search('notes', 'body', 'overdue payment',
                       'embedding', '{vec_literal}', 10)
    GROUP BY folder ORDER BY hits DESC
""")

notes.optimize()

An identifier like INV-2291 is the kind of token embeddings blur. BM25 indexes it as inv and 2291, and the rare number carries the match. The vector half catches notes that say the same thing in other words. A WHERE on hybrid_search is applied to both halves before they are fused, so a filter ranks within the matching rows rather than trimming a list afterwards.

One thing to design for in a local app: every append is a commit that writes a new file, so write in batches rather than a row at a time. Nothing runs in the background inside your process. optimize() is where maintenance happens, so call it when the app is idle.

The quickstart runs the full version in Python, Node.js or Rust, and Integrating SQL analytics with vector and full-text search covers the filtering, grouping and joins over search results. For choosing a store for agent memory in particular, see Choosing a database for AI agents.

Common questions #

Is there an embedded vector database that runs without a server?

Yes. sqlite-vec, DuckDB’s vss extension, LanceDB, Chroma’s embedded client and Infino all run inside the application process and keep their data in local files. They differ in whether the vector search uses an approximate index or an exact scan, and in whether keyword search and SQL come with it.

Can SQLite do vector search?

With the sqlite-vec extension, yes. Its stable release does exact (brute-force) nearest-neighbor search, which is fine for small corpora. Approximate indexes are in its alpha releases, and the project describes itself as pre-v1. Combining it with FTS5 keyword search is SQL you write yourself.

How do I add offline search with embeddings to a desktop app?

Use a library that stores its index in local files and runs in the app’s process, then generate embeddings with a local model so nothing leaves the machine. For keyword search alone, SQLite FTS5 is enough. For keyword plus meaning, pick an engine with built-in hybrid search, such as LanceDB or Infino, and check that it ships for the operating systems you target. Batch writes where the engine commits per write, and run its maintenance step when the app is idle.

What is hybrid search in an embedded engine?

Running keyword (BM25) and vector search over the same records and merging the two ranked lists, usually with reciprocal rank fusion. Keyword search finds exact names and identifiers, vector search finds paraphrases, and the fused list keeps what either one alone would miss. What is hybrid search?.

Can I query my search index with SQL?

In SQLite and DuckDB, yes, since search lives inside a SQL database. LanceDB’s open source edition takes SQL filter expressions on queries. In Infino each search is a SQL table function, so search results can be filtered, grouped and joined in one statement. Its SQL is read-only, and writes go through the API. SQL reference.