From "we have vectors" to "this actually scales, and doesn't bankrupt us."
👦 Nephew: Uncle, Phase 2 is done. I understand tokenization, embeddings, cosine similarity. But two things are bugging me. First — where do these vectors actually live, physically? Second — when we have 5 million of them, how does search stay fast without checking every single one? It feels like magic right now.
👨🦳 Uncle: Good — because it's not magic, and if you treat it like magic, you'll make bad decisions later. Which index to use, how much memory you'll need, why your search suddenly got slower after adding a million vectors, why your embedding bill is way higher than it should be — none of that makes sense until you understand what's actually happening underneath. Let's take this in order: first where things live, then how search stays fast, then — the part almost every tutorial skips — how you stop burning money doing any of this.
Phase 2: Embeddings & Semantic Search ✅
↓
TODAY ← WE ARE HERE
│
├─ Part 1: Where Embeddings Actually Get Saved (Schema Design)
├─ Part 2: Why You Can't Just "Check Everything" (The Brute-Force Wall)
├─ Part 3: IVF — The Neighborhood Approach
├─ Part 4: HNSW — The Highway Approach
├─ Part 5: Product Quantization — Compressing Vectors
├─ Part 6: Distance Metrics — What "Similar" Actually Means
├─ Part 7: Metadata Indexing — The Other Half Everyone Forgets
├─ Part 8: Putting It Together in pgvector
├─ Part 9: Choosing a Vector Database, Honestly
└─ Part 10: Token Economics — Stop Paying Twice for the Same Thing
↓
A System That Survives Real Traffic and a Real Bill
👨🦳 Uncle: Before we talk about indexing, let's settle the basics. A vector by itself is useless. What you actually store is a row — the vector, plus everything needed to use it later, find it again, and avoid paying for it twice.
Table: document_chunks
| Column | Purpose |
|---|---|
id |
|
| Unique identifier | |
chunk_text |
|
| The original text — kept as insurance for re-embedding with a new model later | |
embedding |
|
vector(1536) |
|
embedding_model |
|
e.g. 'text-embedding-3-small' — different models produce incompatible vector spaces |
|
content_hash |
|
SHA-256 of chunk_text — used for dedup, more on this in Part 10 |
|
metadata |
|
jsonb — { department, doc_type, uploaded_by, source_file } |
|
created_at |
|
| Timestamp |
👦 Nephew: Why is metadata a separate jsonb
column instead of just... more columns?
👨🦳 Uncle: Because metadata changes shape depending on the document. An HR policy chunk might have { department: "HR", doc_type: "policy" }
. A support ticket chunk might have { customer_id: 4521, priority: "high" }
. If you tried to make a rigid column for every possible field across every document type, you'd redesign your table every week. jsonb
gives you that flexibility — and, this is the part people miss — you can still index inside a jsonb
column, which we'll do properly in Part 7.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE document_chunks (
id BIGSERIAL PRIMARY KEY,
chunk_text TEXT NOT NULL,
embedding VECTOR(1536) NOT NULL,
embedding_model VARCHAR(50) NOT NULL,
content_hash VARCHAR(64) NOT NULL,
metadata JSONB DEFAULT '{}',
created_at TIMESTAMP DEFAULT now()
);
👦 Nephew: content_hash
again — that's the same SHA-256 idea from Phase 1's file dedup, right?
👨🦳 Uncle: Same idea, one level deeper. In Phase 1 we hashed the whole file to avoid storing the same file twice. Here we hash each individual chunk, to avoid embedding the same chunk twice — because two completely different files can contain the exact same paragraph: a shared boilerplate legal clause, a repeated disclaimer, a standard company intro paragraph copy-pasted into fifty documents. Hold this thought. It becomes real money in Part 10.
👨🦳 Uncle: Now, the search speed question. Suppose you have 5 million chunk vectors, each with 1536 dimensions. A user asks a question, you embed it into a query vector. What's the dumbest possible way to find the most similar chunks?
👦 Nephew: Compare the query vector against every single one of the 5 million vectors, calculate similarity for each, sort, take the top 5.
👨🦳 Uncle: Exactly — that's called a flat index, or exhaustive search. Let's see what it actually costs, in real numbers, not vague hand-waving.
5,000,000 vectors × 1536 dimensions each
For EACH query:
Compare against 5,000,000 vectors
Each comparison: 1536 multiplications + additions (cosine similarity)
Total operations per query:
5,000,000 × 1536 ≈ 7.68 BILLION operations
On a typical machine: ~20-30 seconds per query
👦 Nephew: 30 seconds?! Nobody waits 30 seconds for a chatbot answer.
👨🦳 Uncle: Exactly why brute force only works at small scale — a few thousand vectors, maybe. Past that, you need a fundamentally different strategy: don't check everything, only check the likely candidates. This trade-off has a name: Approximate Nearest Neighbor search, or ANN.
👦 Nephew: "Approximate"? So it might give the wrong answer?
👨🦳 Uncle: It might give you the 4th-closest match instead of the actual 1st-closest, occasionally. That trade is almost always worth it, because the alternative is checking all 5 million vectors exactly, on every single query, forever. You're trading a tiny bit of accuracy for a massive amount of speed.
👦 Nephew: Is this like the B-tree index we used on a normal database column, back when we built that email lookup?
👨🦳 Uncle: Same spirit — avoid scanning everything — but a completely different mechanism, and this distinction trips people up constantly. A B-tree index works because values have a natural sort order: "find email = x" can binary-search through sorted order, because "a" comes before "b" comes before "c". But vectors can't be sorted meaningfully in 1536-dimensional space. There's no "less than" between two directions in space.
So vector databases use an entirely different family of tricks. Every real ANN technique, no matter how fancy the name sounds, does one of two things: groups similar vectors together so you only search a small group, or builds a shortcut map that lets you jump toward the right neighborhood instead of walking through everyone. Let's go through both, properly.
👨🦳 Uncle: Remember Google Maps — it doesn't search the whole world to find Bangalore, it narrows down: Country → State → City. IVF works the same way for vectors.
Step 1 — Training: group vectors into "neighborhoods" (clusters)
Before any search happens, IVF looks at all your vectors and groups them into K clusters, using an algorithm like k-means. Imagine 5 million vectors get grouped into 1,000 clusters:
| Cluster | Centroid Topic | Vectors |
|---|---|---|
| 1 | Programming topics | 5,200 |
| 2 | HR policy topics | 4,800 |
| 3 | Financial topics | 5,100 |
| ... | ... | ... |
| 1000 | Legal topics | 4,900 |
Each cluster has a centroid — the "average" vector representing everything in that group.
Step 2 — Query time: only search the closest clusters
Query: "What is our notice period?"
↓
Compare query ONLY against the 1,000 centroids (fast — just 1,000 comparisons)
↓
Find the 5 closest centroids (say, clusters 3, 47, 112, 289, 501)
↓
Only search vectors INSIDE those 5 clusters (~25,000 vectors, not 5 million!)
↓
Return top-5 most similar from that much smaller set
// Conceptual illustration of IVF search (real implementations are C++/Rust,
// but this shows the logic clearly)
function ivfSearch(queryVector, centroids, clusters, nProbe = 5) {
// Step 1: Find nProbe closest centroids (cheap — only 1000 comparisons)
const centroidDistances = centroids.map((centroid, i) => ({
clusterId: i,
distance: cosineSimilarity(queryVector, centroid)
}));
const closestClusters = centroidDistances
.sort((a, b) => b.distance - a.distance)
.slice(0, nProbe);
// Step 2: Only search vectors WITHIN those clusters
let candidates = [];
for (const cluster of closestClusters) {
candidates.push(...clusters[cluster.clusterId]);
}
// Step 3: Full precise search, but only on the much smaller candidate set
return candidates
.map(v => ({ vector: v, score: cosineSimilarity(queryVector, v.embedding) }))
.sort((a, b) => b.score - a.score)
.slice(0, 5);
}
👦 Nephew: So instead of 5 million comparisons, we do 1,000 (to find clusters) + ~25,000 (inside the clusters) ≈ 26,000. That's roughly 200x fewer operations!
👨🦳 Uncle: Exactly the math. And notice the tuning knob — nProbe
, how many clusters you search:
| nProbe | Result |
|---|---|
| 1 | Fastest, but might miss the real best match (low recall) |
| 10 | Good balance, catches ~95% of true best matches |
| 1000 | Same as brute force (searches everything) |
This is the recall-vs-speed tradeoff, and it's the central tension in every vector index that exists.
One more practical detail: how many clusters (lists
, in pgvector's terminology) should you even create in the first place? A common rule of thumb:
lists ≈ sqrt(number of rows)
For 5 million rows: sqrt(5,000,000) ≈ 2,236, so roughly 2,000-2,500 lists
Too few lists → clusters are huge, barely faster than brute force.
Too many lists → clusters are tiny, and centroids get unreliable
without enough data per cluster.
CREATE INDEX ON document_chunks
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 2000);
👨🦳 Uncle: IVF groups things into neighborhoods. HNSW takes a completely different approach — it builds a multi-level shortcut map, like a highway system.
👦 Nephew: Highway system?
👨🦳 Uncle: Think about how you'd actually drive from Bangalore to a small village 800 km away. You don't take village roads the whole way.
Layer 2 (Highways): Bangalore ─────────────────→ Big City X
│
Layer 1 (State roads): Bangalore ──→ Town A ──→ Town B ──┤
│
Layer 0 (Local roads): Bangalore ──→ ... ──→ ... ──→ Village
👨🦳 Uncle: You take the highway to get close fast, then drop to smaller roads only for the final stretch. HNSW builds exactly this structure over your vectors — multiple layers, where the top layer has very few "long-distance" connections, and each layer below has progressively more, finer-grained, local connections.
How HNSW search actually works:
Layer 2 (few nodes, long connections):
Start at entry point → jump to nearest node in this sparse layer
↓ (drop down one layer, using that node as new starting point)
Layer 1 (more nodes, medium connections):
From current position → jump to nearest node in this layer
↓ (drop down again)
Layer 0 (ALL nodes, short/local connections):
From current position → carefully walk to the actual nearest neighbors
Total "hops" needed: roughly log(N) instead of N
For 5 million vectors: ~22 hops instead of 5 million comparisons!
// Conceptual illustration of HNSW search
function hnswSearch(queryVector, graph, entryPoint, topK = 5) {
let currentNode = entryPoint;
let currentLayer = graph.topLayer;
// Phase 1: Greedy descent through upper layers (the "highway" phase)
while (currentLayer > 0) {
currentNode = greedySearchInLayer(queryVector, currentNode, graph, currentLayer);
currentLayer--;
}
// Phase 2: Careful search in the bottom layer (the "local roads" phase)
const candidates = beamSearchInLayer(queryVector, currentNode, graph, 0, topK * 4);
return candidates
.sort((a, b) => b.score - a.score)
.slice(0, topK);
}
👦 Nephew: So HNSW is like... a skip list, but for vectors in high-dimensional space?
👨🦳 Uncle: That's a genuinely excellent way to think about it. If you've seen skip lists in data structures, HNSW is that exact idea, generalized from a sorted 1-D list to a graph in 1536-dimensional space — sparse at the top for big jumps, dense at the bottom for precision.
| IVF | HNSW | |
|---|---|---|
| Search speed | Fast | Usually faster |
| Build time | Faster to build | Slower to build (constructing the graph) |
| Memory usage | Lower | Higher (stores graph connections) |
| Accuracy (recall) | Good, tunable via nProbe |
|
Excellent, tunable via ef_search |
||
| Handling new inserts | Easy — assign to nearest cluster | Harder — inserting into a graph is costlier |
| Best for | Very large datasets, memory-constrained, heavy insert traffic | Best accuracy/speed balance — the modern default |
👦 Nephew: So if HNSW is usually faster and more accurate, why would anyone use IVF?
👨🦳 Uncle: Two real reasons. First, memory — at massive scale (hundreds of millions of vectors), HNSW's graph connections take noticeably more RAM than IVF's simpler cluster-list structure. Second, update patterns — if your dataset is constantly getting new vectors (a live chat system logging every message, for instance), IVF handles that more gracefully than rebuilding parts of an HNSW graph. For most RAG systems — a few hundred thousand to a few million chunks, updated periodically rather than constantly — HNSW is the common default today, and it's what most managed vector databases use out of the box.
👦 Nephew: Is there anything for when even HNSW's memory usage becomes a problem?
👨🦳 Uncle: Yes — Product Quantization, or PQ. This doesn't replace IVF or HNSW; it works alongside them, by compressing the vectors themselves.
Original vector: 1536 dimensions × 4 bytes each = 6,144 bytes per vector
Product Quantization:
Split the 1536 dimensions into, say, 96 sub-vectors of 16 dimensions each
For each sub-vector, find its closest match from a small pre-learned "codebook"
Store just the CODE (an index into the codebook), not the raw numbers
Compressed vector: 96 codes × 1 byte each = 96 bytes per vector
Compression: 6,144 bytes → 96 bytes = 64x smaller!
👨🦳 Uncle: The tradeoff, predictably, is a small loss in precision — you're now comparing compressed approximations, not exact vectors. But for 100 million+ vectors, that memory savings can be the difference between fitting in RAM and not fitting at all. Real large-scale systems often combine all three techniques: IVF to narrow down the neighborhood, PQ to keep everything compact in memory, and a final precise re-ranking step on the small candidate set to recover accuracy.
👦 Nephew: We've been saying "cosine similarity" this whole time. Is that the only way to measure closeness?
👨🦳 Uncle: No — three metrics show up constantly, and picking the wrong one for your use case genuinely hurts results.
| Metric | Measures | Best For | Formula |
|---|---|---|---|
| Cosine Similarity | |||
| Angle between vectors, ignores magnitude | Text embeddings (most common for RAG) | `(A · B) / (\ | |
| Euclidean Distance (L2) | |||
| Actual straight-line distance | Image embeddings, spatial data | {% raw %}sqrt(sum((A[i] - B[i])²)) |
|
| Dot Product | |||
| Like cosine, but also factors in magnitude | When magnitude itself is meaningful | A · B |
👨🦳 Uncle: For text embeddings from OpenAI, Cohere, or most modern embedding models — cosine similarity is the standard, because these models are trained so direction, not magnitude, carries the meaning. But always check your embedding provider's documentation — some models are specifically trained for dot-product comparison instead, and using the wrong metric silently gives you worse results, with no error message telling you why.
👨🦳 Uncle: Here's a problem vectors alone can never solve, no matter how good your ANN index is. Say your RAG system serves the whole company. HR documents, engineering docs, finance docs — all in one table, all embedded the same way. Someone from Finance asks:
"What is our expense reimbursement policy?"
👦 Nephew: Pure vector search would search all 5 million chunks — including HR chunks, engineering wiki pages, everything?
👨🦳 Uncle: Exactly the problem. Vector similarity alone doesn't know "this user only has access to Finance documents" or "only search documents from the last 6 months." That's not a meaning problem — it's a filtering problem. This is what metadata indexing solves, and it works completely independently from HNSW or IVF.
👨🦳 Uncle: There are two ways to combine "filter by metadata" with "search by vector similarity" — and picking the wrong one silently breaks your results, without ever throwing an error.
Post-filtering (the wrong default for most cases):
1. Run vector search → get top 5 similar chunks (from ALL departments)
2. THEN filter by department = 'Finance'
3. Problem: if none of the top 5 happened to be Finance,
you get ZERO results — even though relevant Finance
chunks exist further down the ranking!
Pre-filtering (usually correct):
1. FIRST filter: WHERE metadata->>'department' = 'Finance'
2. THEN run vector search, but ONLY within that filtered set
3. Correct Finance chunks are guaranteed to be considered
👦 Nephew: So we always want pre-filtering?
👨🦳 Uncle: Almost always, for access-control and correctness reasons like this. The catch is performance — pre-filtering needs the metadata filter itself to be fast, or you've just recreated the "scan everything" problem one step earlier. That's exactly why metadata needs its own index, separate from your HNSW or IVF index on the embedding column.
Indexing inside jsonb:
CREATE INDEX idx_metadata_department
ON document_chunks
USING GIN ((metadata -> 'department'));
-- If you'll filter by department very frequently, extract it into
-- its own indexed column instead — faster still:
ALTER TABLE document_chunks ADD COLUMN department VARCHAR(50)
GENERATED ALWAYS AS (metadata->>'department') STORED;
CREATE INDEX idx_department ON document_chunks (department);
The combined query — the real production pattern:
SELECT
chunk_text,
metadata,
1 - (embedding <=> $1) AS similarity
FROM document_chunks
WHERE department = 'Finance'
AND created_at > NOW() - INTERVAL '6 months'
ORDER BY embedding <=> $1
LIMIT 5;
👦 Nephew: That WHERE
clause runs first, using the metadata index — then the vector similarity ordering happens only on what's left?
👨🦳 Uncle: Exactly the mental model. Postgres's query planner uses the metadata index to shrink the candidate set first, then the vector index does its ANN search on that much smaller set.
Filter hard with metadata, search smart with vectors.That sentence is worth remembering more than any specific syntax we've covered today.
👨🦳 Uncle: Let's make everything concrete, end to end, since you're already comfortable with Postgres.
-- Enable the extension
CREATE EXTENSION IF NOT EXISTS vector;
-- Without an index: pgvector does exact brute-force search.
-- Fine for a few thousand rows, painful past that — exactly Part 2's problem.
-- Create an HNSW index (recommended default for most RAG use cases)
CREATE INDEX ON document_chunks
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);
-- m = how many connections per node (higher = more accurate, more memory)
-- ef_construction = how thorough the build process is (higher = better index, slower build)
// Tuning search-time accuracy vs speed for HNSW in pgvector
await db.query(`SET hnsw.ef_search = 100;`); // higher = more accurate, slower
// Default is 40. Push higher when a query needs better recall
// (say, a compliance-related question), lower for high-volume, low-stakes lookups.
👦 Nephew: So m
and ef_construction
affect how good the index is when it's built, and ef_search
affects how thorough each individual query is?
👨🦳 Uncle: Exactly that distinction, and it matters because of when you pay for it. You pay the ef_construction
cost once, when building the index. You pay the ef_search
cost on every single query, forever. Tune the build settings for overall quality; tune the search setting per-query if some queries deserve more accuracy than others.
vector_cosine_ops
in that CREATE INDEX
line tells Postgres which similarity math to build the index around. pgvector also supports vector_l2_ops
(Euclidean) and vector_ip_ops
(dot product) — match the index's ops to the metric you actually query with (Part 6), or the index silently won't help at all.
👦 Nephew: Pinecone, Qdrant, Weaviate, pgvector, Milvus — how do I actually pick?
👨🦳 Uncle: Same instinct as picking a PDF parser back in Phase 1 — match the tool to your actual scale and constraints, don't pick based on hype.
| pgvector | Qdrant | Pinecone | Weaviate | |
|---|---|---|---|---|
| Type | ||||
| Postgres extension | Standalone (self-host or cloud) | Fully managed cloud | Standalone or managed | |
| Best for | ||||
| Already on Postgres, moderate scale (<10M vectors) | Self-hosted control, strong filtering | Zero-ops, massive scale | Hybrid search (keyword + vector) | |
| Index types | ||||
| HNSW, IVFFlat | HNSW | Proprietary (managed) | HNSW | |
| Cost model | ||||
| Free (just your Postgres bill) | Free self-hosted, paid cloud tier | Pay per vector + query volume | Free self-hosted, paid cloud tier | |
| Operational overhead | ||||
| Low — it's just Postgres | Medium — separate service | None — fully managed | Medium — separate service |
👨🦳 Uncle: My honest default: if you're already running Postgres and under roughly 5-10 million vectors, pgvector is usually the right starting choice — one less system to operate, and modern pgvector with HNSW is genuinely fast. Reach for a dedicated vector database when you outgrow that, need advanced filtering at real scale, or need to scale vector search independently from your main database.
👦 Nephew: Uncle, one more thing bugging me. Embeddings don't go through an LLM chat, so what "tokens" are we even saving on this side?
👨🦳 Uncle: Two different token costs hide in a RAG pipeline, and most beginners only think about one of them.
| Cost | What it is |
|---|---|
| Embedding tokens | |
| Every time you call the embedding API, you pay per token of the text you're embedding — 5M chunks × ~200 tokens each = 1 billion tokens | |
| LLM generation tokens | |
| Every retrieved chunk stuffed into the prompt costs tokens again, every single query, forever |
👨🦳 Uncle: This is exactly why we stored content_hash
back in Part 1. Before calling the embedding API, check first.
async function getOrCreateEmbedding(chunkText, embeddingModel) {
const hash = crypto.createHash("sha256").update(chunkText).digest("hex");
const existing = await db.query(
`SELECT embedding FROM document_chunks
WHERE content_hash = $1 AND embedding_model = $2`,
[hash, embeddingModel]
);
if (existing.rows.length > 0) {
console.log("Skipped embedding call — already exists");
return existing.rows[0].embedding;
}
const response = await openai.embeddings.create({
model: embeddingModel,
input: chunkText,
});
return response.data[0].embedding;
}
👦 Nephew: So if the same disclaimer paragraph appears in 200 different uploaded PDFs...
👨🦳 Uncle: You call the embedding API exactly once for it, not 200 times. At scale, across thousands of documents with shared boilerplate, this alone can meaningfully cut embedding costs — some real-world document sets have 20-30% content overlap once you actually measure it.
// Expensive: 1000 separate API calls, 1000x network overhead
for (const chunk of chunks) {
await openai.embeddings.create({ model, input: chunk.text });
}
// Correct: ONE call, same total tokens, far less overhead
const response = await openai.embeddings.create({
model: "text-embedding-3-small",
input: chunks.map(c => c.text), // array of texts
});
👨🦳 Uncle: The token cost is roughly the same either way — you're still paying for the same total text. But batching slashes the number of requests, which matters for rate limits, latency, and — if your provider has any per-request overhead — real cost too.
👨🦳 Uncle: This connects straight back to Phase 2's Top-K idea. Every chunk you retrieve gets sent to the LLM as context, and you pay generation-side tokens for every single one, on every single query, forever. Retrieving Top-10 "just to be safe" instead of Top-5 doesn't just cost double — it also adds noise that can make the LLM's answer worse, not better.
👨🦳 Uncle: If many users ask near-identical questions — "what is the notice period" asked 500 times a month across the company — cache the final answer (or at least the retrieved chunks) keyed by a normalized version of the question, so identical questions don't re-run the whole embed → search → generate pipeline every time.
PHASE 1: DOCUMENT INGESTION ✅
─────────────────────────────
PDF Upload → File Hash Check → Parse & Clean → Chunking
→ Deduplication → Store Chunk Text
PHASE 2: EMBEDDINGS & SEMANTIC SEARCH ✅
─────────────────────────────
Chunk Text → Tokenization → Embedding Layer → Vector
→ Cosine Similarity → Top-K Retrieval
TODAY: STORAGE, INDEXING & TOKEN ECONOMICS ← YOU ARE HERE
─────────────────────────────
Vector + Metadata
↓
Store in Postgres (pgvector): chunk_text, embedding,
content_hash, embedding_model, metadata (jsonb)
↓
Vector Indexing (HNSW / IVF, +PQ at massive scale) → fast, approximate similarity search
↓
Metadata Indexing (GIN / generated columns) → fast, correct filtering
↓
Combined Query: filter FIRST (metadata) → search SMART (vectors)
↓
Token Economics: dedup via content_hash, batch embedding calls,
keep Top-K small, cache repeated queries
PHASE 4: QUERY TIME & PRODUCTION SAFETY (next)
─────────────────────────────
User Question → Embed → Pre-filter by metadata → ANN vector search
→ Top-K chunks → Groundedness checks → LLM → Cited Answer
| Problem | Before Today | After Today |
|---|---|---|
| Search accuracy (meaning) | ✅ Solved in Phase 2 | ✅ Unchanged, still relies on it |
| Search speed at scale | ❌ Checks everything, 30 sec | ✅ ANN index (HNSW/IVF), milliseconds |
| Memory at massive scale | ❌ Not addressed | ✅ Product Quantization |
| Access control / filtering | ❌ Not addressed | ✅ Metadata indexing, pre-filtering |
| Duplicate embedding cost | ❌ Not addressed | ✅ content_hash dedup |
| Request overhead | ❌ Not addressed | ✅ Batched embedding calls |
| Repeated identical questions | ❌ Not addressed | ✅ Query-level caching |
nProbe
, with lists ≈ sqrt(row count)
as a starting rule of thumbef_construction
(build time) and ef_search
(query time); the common default for new production systems today👦 Nephew: So when someone says "our vector search is slow," the real question isn't "which tool" — it's which index, which parameters, how much data, and whether the slowness is even coming from the vector side at all, versus an unindexed metadata filter?
👨🦳 Uncle: Exactly the right instinct. "Vector search" was never one thing — it's a whole family of tradeoffs between speed, accuracy, memory, and cost, sitting right alongside an equally important filtering problem that has nothing to do with vectors at all. The right answer always depends on your actual scale and constraints, not on which tool sounds most impressive in a job description.
Next: Query-Time Architecture & Production Safety — wiring pre-filtering and ANN search together, handling embedding model migrations safely, guardrails for when retrieval finds nothing relevant, and rate limiting for a real production RAG API.
Remember: less noise, more action. Today is where a demo RAG project turns into a system that survives real traffic, a real access-control policy, and a real bill.