Hallucination means making an assumption or making up something when the LLM does not know the answer.
Example:
Suppose we have a PDF file that contains information about Python, but it does not have any details about Decorators.
Here, the document is split into chunks, and the chunks are stored in the vector database.
Suppose we ask the query:
"What is a decorator?"
The LLM should not give any response because the information about decorators is not available in the database.
But the LLM may give a response from its own knowledge, which may be correct, but it is not from the database.
This is called hallucination in RAG.
The response given by the LLM should be supported by the information given in the context.
If the response is not supported by the context, it is considered hallucinated. Compare the embedding of the context with the embedding of the LLM response to check how closely they are related.
LettuceDetect is a BERT-based model used to find where hallucinations could occur at the character level.
It checks the response word by word to identify possible hallucinations.
Give the response to another LLM and ask it to detect hallucinations.
The drawback here is that the LLM may hallucinate again, and it can also be expensive unless we use a local model.
RAGAS is also called an evaluation framework. It provides different metrics that can be used to evaluate RAG systems and identify potential hallucinations.
Some of the important metrics are:
Faithfulness – Checks whether the results are backed by the provided context. This can be used to check hallucination.
Answer Relevancy – Suppose the context and answer are relevant. This metric checks how relevant the answer is to the query.
Context Precision – Checks how many of the retrieved documents are useful and relevant.
Context Recall – Checks whether the necessary documents have been retrieved for the query.
Answer Correctness – Checks the response against the ground truth.
Answer Similarity – Checks the semantic similarity between the expected answer and the generated answer.
RAGAS can be used in a CI/CD pipeline.
We can write unit test cases and compare the actual answer with the expected answer using the above metrics.
DeepEval can also be used to evaluate RAG systems and detect hallucinations.