If you've been exploring AI and large language models, you've probably heard about vector databases. They're very important for llm behind semantic search, recommendation systems, and Retrieval-Augmented Generation (RAG). ChromaDB is one of the most beginner-friendly options out there—it's open-source, runs locally, and requires no cloud account to get started. Let's dive in.
Traditional databases search by matching keywords exactly. Vector databases work differently—they store data as embeddings (lists of numbers that capture meaning) and find results based on semantic similarity rather than exact word matches.
Here's a simple example: if you search for "pets," a vector database can return documents about dogs and cats, even if the word "pets" never appears in them. This capability powers modern AI applications, such as chatbots that can reason about your specific documents.
ChromaDB is particularly popular for its low learning curve. You can run it entirely on your machine, making it perfect for prototyping and learning before moving to production systems.
Installing ChromaDB is straightforward with pip:
pip install chromadb
ChromaDB works with Python 3.8 or higher, and the package includes everything needed for local usage.
For server-mode deployment (better suited for production), you can install the server extras:
pip install chromadb[server]
This gives you HTTP API support for connecting from other applications.
Let's build a simple example from scratch. This code creates a persistent vector database, adds documents, and performs semantic search.
A persistent client saves your data to disk so it's not lost when your program ends:
import chromadb
client = chromadb.PersistentClient(path="./chroma_db")
A collection is like a table in a traditional database—it stores your documents, embeddings, and metadata together:
collection = client.get_or_create_collection(name="my_documents")
ChromaDB can automatically generate embeddings for your text—you don't need to manage vectors manually:
collection.add(
ids=["doc1", "doc2", "doc3"],
documents=[
"Dogs make wonderful companions and are known for their loyalty.",
"Cats are independent animals that make great pets for busy people.",
"I love driving fast cars on open highways."
]
)
Each document needs a unique ID, and Chroma handles the embedding process automatically.
Now for the magic—querying by meaning rather than keywords:
results = collection.query(
query_texts=["Tell me about animal companions"],
n_results=2 # Return the top 2 most similar results
)
print(results)
Chroma returns documents that are semantically closest to your query. In this case, you'd likely get the dog and cat documents—not the car one.
Behind the scenes, ChromaDB converts your documents into embeddings—numerical vectors that capture meaning. When you query, it calculates the distance between your query embedding and all stored embeddings using metrics like cosine similarity or Euclidean distance.
ChromaDB uses an indexing algorithm called HNSW (Hierarchical Navigable Small World) to find similar vectors quickly, even with millions of documents. This makes semantic search practical at scale.
Once you've got the fundamentals down, here are some directions to explore:
where
filtersIf you are planning to handle more than 10 million users, you should use another vector database, such as Pinecone or Milvus. In this case, Milvus is free, while Pinecone is a paid service.