Key Takeaways #
- Real-Time Search Freshness: Storing vectors alongside transactional metadata in TiDB eliminates data sync latency, ensuring RAG applications always retrieve up-to-date document context.
- Hybrid Engine Architecture: TiKV handles transactional writes while TiFlash powers HNSW vector indexes, maintaining complete searchability and high recall across both Delta and Stable storage layers.
- Unified SQL Querying: Consolidating vectors and operational data into one MySQL-compatible database enables single-query joins across embeddings, permissions, and tenant metadata without multi-system overhead.
When a RAG application keeps embeddings in a separate vector store, an edited support article can keep surfacing in its old form until the next sync job runs. The model then answers from stale context, and nobody notices until a customer does. Knowing how vector search works inside your database tells you how fresh those results are and how fast they come back.
This post walks through the write-to-query path of a vector in TiDB, from the transactional commit to the HNSW index, with the SQL to check each step.
How Vector Search Works #
Vector search converts content into embeddings, stores them, and returns the stored items closest to a query embedding by a distance function such as cosine or L2. An embedding model maps text, images, or audio to fixed-length arrays of floats, so items with similar meaning sit near each other. The database ranks stored vectors by distance and returns the top K.
Exact K-nearest neighbor (KNN) search compares the query with every row and slows down as the table grows. Approximate nearest neighbor (ANN) search walks an index such as HNSW and visits a small fraction of rows. TiDB’s documentation reports recall generally above 90% with its HNSW index.
Why AI Applications Need Vectors Next to Transactional Data #
An embedding is only useful with the row it describes: the document’s owner, its permissions, its current version. When vectors live in one system and that metadata lives in another, every retrieval needs a second lookup, and every write needs a sync pipeline that can fall behind.
Stock MySQL does not remove that split. MySQL Community 9.x stores a VECTOR type, but its DISTANCE() function is available only on MySQL HeatWave and MySQL AI, not in the Community or Commercial editions, as covered in this comparison of MySQL vector similarity search options. TiDB keeps vectors, metadata, and transactions in one MySQL-compatible database.
How TiDB Stores, Indexes, and Queries Vectors #
In TiDB, a vector is a column like any other: TiKV commits it transactionally, TiFlash replicates it, and TiFlash indexes it with HNSW. Writes commit through Raft in TiKV, the row store. TiFlash, the columnar engine, receives the same data as a Raft learner and holds the vector index. Together, the row store and columnar engine make TiDB an HTAP database. Before serving a read, TiFlash checks replication progress against the leader, so a search reflects committed data.
CREATE TABLE docs (
id BIGINT PRIMARY KEY,
tenant_id INT,
body TEXT,
embedding VECTOR(768),
VECTOR INDEX idx_embedding ((VEC_COSINE_DISTANCE(embedding)))
);
-- Check index build progress per table
SELECT TIDB_TABLE, ROWS_STABLE_INDEXED, ROWS_STABLE_NOT_INDEXED, ROWS_DELTA_NOT_INDEXED
FROM INFORMATION_SCHEMA.TIFLASH_INDEXES;
-- Merge recent writes into the indexed Stable layer
ALTER TABLE docs COMPACT;
TiFlash stores data in two layers. New and updated rows land in the Delta layer, which is never indexed; compaction merges them into the Stable layer, where the HNSW index covers them. Rows still in Delta remain searchable and results stay complete, but queries run slower until the index catches up. TiDB’s docs give a reference point: indexing a 500 MiB dataset of 768-dimension vectors can take up to 20 minutes.
Vector search runs on TiDB Cloud Starter, Essential, and Dedicated, and on TiDB Self-Managed. Dedicated and Self-Managed need v8.4.0 or later (v8.5.0 or later recommended). TiDB Cloud docs label the feature beta, and Self-Managed docs mark it experimental and not recommended for production. The vector search index docs list the full restrictions.
What This Design Means for RAG and Agent Retrieval #
Because vectors and metadata share one database, a retrieval can join embeddings to tenants, permissions, and document versions in a single SQL statement. Storage scales out by adding TiKV and TiFlash nodes, so a growing embedding corpus does not force a migration to a second system. TiDB has official integrations with LangChain and LlamaIndex, and you can generate embeddings with providers such as OpenAI, Hugging Face, and Jina AI, so vector search fits into an existing RAG or semantic search pipeline.
Dify, the open-source LLM application platform, uses TiDB Cloud Starter as its vector store alongside its document store and consolidated nearly half a million database containers into a single TiDB Cloud implementation.
The design has costs worth planning for:
- The vector index requires TiFlash; on Self-Managed, deploy TiFlash nodes first.
- A WHERE filter applied before the KNN step skips the index, so run KNN in a subquery and filter afterward; the filtered query can return fewer than K rows.
- Large backfills need compaction time before queries reach full index speed.
For agent-style workloads that keep per-agent context in the same database, see persistent context for AI agents, and for retrieval patterns more broadly, vector search and RAG on TiDB.
Build Your First Vector Search Workload on TiDB Cloud Starter
TiDB keeps embeddings in the same transactional database as the data they describe, so retrieval reflects committed writes without a sync pipeline. Start a free TiDB Cloud Starter cluster and run the table definition above against your own documents.
FAQ #
Are newly inserted vectors searchable right away in TiDB?
- Yes. Committed rows are searchable once replicated to TiFlash, and results stay complete.
- The HNSW index does not yet cover rows still in the Delta layer, so those queries run slower.
- ALTER TABLE … COMPACT merges Delta rows into the indexed Stable layer.
Which engine runs vector search in TiDB, TiKV or TiFlash?
- Writes go to TiKV; the HNSW index lives on TiFlash replicas.
- Defining a vector index at table creation creates the TiFlash replica automatically.
- An existing table needs ALTER TABLE … SET TIFLASH REPLICA 1 before you add the index.
How many dimensions does TiDB vector search support?
- Up to 16,383 dimensions per vector.
- The HNSW index requires a fixed dimension, such as VECTOR(768).
Spin up a database with 25 GiB free resources.
Related Resources #
AmzTrends’ Cloud-Native Transformation: 80% Cost Saving with TiDB Serverless
How Chainbase Reduced Web3 Infrastructure Costs Up to 50% with a Streamlined Tech Stack Powered by TiDB
Real Savings, Real Stories: Slash 30%-80% Database Costs with TiDB Serverless
TiDB Cloud Dedicated #
A fully-managed cloud DBaaS for predictable workloads
TiDB Cloud Starter #
A fully-managed cloud DBaaS for auto-scaling workloads