vector database intro

A vector database stores data as high-dimensional mathematical embeddings rather than rows and columns, allowing you to perform "similarity searches" to find items that are conceptually related rather than exactly matching.

Think of a standard SQL database like a giant filing cabinet. If you search for "crimson sneakers," and the database only has "red shoes," you get zero results. It's binary. A vector database doesn't see words; it sees coordinates in a space with hundreds or thousands of dimensions. In that space, the vector for "crimson" is physically very close to "red."

The mechanical shift from keywords to embeddings

To get data into a vector DB, you run your text, image, or code through an embedding model (like text-embedding-3-small

from OpenAI). This model spits out a vector—essentially a long list of numbers like [0.12, -0.59, 0.88, ...]

.

The magic is in the distance calculation. Most vector databases use Cosine Similarity or Euclidean Distance to figure out how far apart two points are.

| Database Type | Search Method | Result Type | Best Use Case |

| :--- | :--- | :--- | :--- |

| Relational (PostgreSQL) | Exact Match / B-Tree | Boolean (Yes/No) | User accounts, Orders |

| Vector (Milvus/Pinecone) | Nearest Neighbor (ANN) | Probability/Score | RAG, Recommendation engines |

| Key-Value (Redis) | Key Lookup | Exact Value | Caching, Session state |

I tried building a simple RAG (Retrieval-Augmented Generation) pipeline last month using just a JSON file and a loop to calculate cosine similarity manually. With 50 documents, it was fast. Once I hit 5,000 chunks of documentation, my latency jumped to 1.2 seconds per query. That's where the "Approximate Nearest Neighbor" (ANN) algorithms in real vector databases come in. They don't check every single vector; they use indexing (like HNSW) to narrow the search area instantly.

Implementing a basic vector flow

If you're coding this, the workflow usually looks like this:

1. Chunking: You can't embed a 100-page PDF as one vector; you'll lose all the nuance. Break it into 500-token chunks.

2. Embedding: Pass those chunks to your model.

3. Upserting: Push the vector and the original text (as metadata) into the database.

4. Querying: Embed the user's question → Search DB for top 3 closest vectors → Feed those 3 chunks to Claude or GPT-4 as context.

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer('all-MiniLM-L6-v2')

![vector database intro](/uploads/articles/81ebb413397be7c8.webp)

docs = ["Python is a coding language", "The sky is blue", "AI agents are cool"]
embeddings = model.encode(docs)

query = "Tell me about programming"
query_vec = model.encode([query])

similarities = np.dot(embeddings, query_vec.T).flatten()
best_match_idx = np.argmax(similarities)

print(f"Closest match: {docs[best_match_idx]}") 

Why this matters for prompt engineering

Most people think prompt engineering is just about the words you use in the chat box. It's not. The real "engineering" happens in the retrieval step. If your vector database returns irrelevant chunks (noise), the LLM will hallucinate, no matter how good your system prompt is.

I've found that tweaking the "top-k" value (how many documents you retrieve) is a balancing act. Too few, and the AI misses the answer. Too many, and you hit the context window limit or distract the model. I usually settle on k=4 for technical docs, but for legal text, I've had to go up to k=10 to get enough context.

If you're struggling to find the right way to structure your retrieval queries, looking at Prompt Sharing can give you a head start on how others are framing their RAG prompts to handle retrieved data more effectively.

The "Hidden" cost of vector DBs

Vector databases aren't free in terms of complexity. You have to manage the "embedding drift." If you update your embedding model from version 1 to version 2, every single vector in your database becomes useless. You have to re-embed your entire dataset.

It's a pain. I spent four hours last Tuesday re-indexing a dataset because I decided to switch models for slightly better accuracy.

Joining the PromptCube ecosystem

Learning this stuff in a vacuum is slow. You can read the docs for Pinecone or Weaviate all day, but seeing how another dev structured their metadata filters to reduce hallucination is where the actual growth happens.

PromptCube is basically a clubhouse for people obsessed with the intersection of LLMs and production code. Instead of guessing if a certain prompt structure works with a vector retrieval flow, you can see what's actually performing. It's less about "trying things" and more about using proven patterns.

You can join the community by signing up on the site. It's the fastest way to move from "I think this works" to "I have the benchmarks to prove this works."

Next Building a Hinglish voice mentor with Gemini and LiveKit is a →

a library of Claude prompt techniques, with plenty of directly applicable cases.

All Replies （0） #

No replies yet — be the first!