Generally, when a user asks a query, the system searches for the relevant chunks stored in the vector database using cosine similarity. The better we can filter the data, the smaller the search space becomes, resulting in faster and more efficient retrieval.
Suppose we have a book with 10 chapters. If we want to search for a particular topic, all the points in the vector database are compared with the user query, and only the closest points are retrieved. This process is called KNN (K-Nearest Neighbors).
Another algorithm is ANN (Approximate Nearest Neighbors). Instead of checking all the points in the vector database, ANN searches only within a smaller region based on the proximity of the data. As the name suggests, it does not always return the exact result, but it provides the most preferred or approximate results much faster.
Is there any other method we can use to make the search more effective?
Metadata Filtering
Metadata means data about the data.
Metadata is stored along with each chunk. It can contain information related to the chunk, such as the chapter name, topic description, author, or any other relevant details.
When the user query contains information related to the metadata (for example, a chapter name or topic), the system can directly filter the relevant chunks before performing vector similarity search. This technique is called metadata filtering.
Metadata filtering is supported by:
FAISS does not provide built-in support for metadata filtering.
Documents are first split into chunks, and each chunk is converted into vectors and stored in the vector database.
When a user query arrives, it is converted into a vector and searched against the vector database to retrieve the closest chunks. However, we do not know whether the retrieved documents are actually the most relevant to the query. It is not always true that the closest vectors represent the most relevant documents.
The documents retrieved from the vector database are passed to a cross-encoder along with the user query.
The cross-encoder assigns a relevance score that indicates how closely each document matches the query. The documents are then displayed in ascending or descending order based on these scores.
The results produced by the cross-encoder are called reranked results.
The retrieved documents remain the same as those returned by the vector database, but their order changes. Documents with higher relevance scores appear before those with lower scores.
A cross-encoder is a neural ranking model. Instead of encoding the query and documents separately, it takes both the query and the document together as input to a transformer model and generates a relevance score for each document.
There are transformer models specifically designed for reranking tasks. The encoder understands the meaning of both the query and the document and reranks the documents accordingly.
Reranking is an important step in the RAG pipeline.
It is especially useful when working with documents that contain images or other multimodal content.
Suppose the user asks:
"Show me the front view of the truck."
The vector database may retrieve multiple images related to trucks because they are semantically similar.
The reranker analyzes both the query and the retrieved images (or their associated text descriptions) and assigns relevance scores.
As a result, the image showing the front view of the truck receives a higher score than the other truck images, making it appear first in the final results.