# Building rag-critic: Factual Grounding & Adversarial Hallucination Auditing in LangGraph

> Source: <https://dev.to/samjanjua6/building-rag-critic-factual-grounding-adversarial-hallucination-auditing-in-langgraph-24hf>
> Published: 2026-10-23 20:58:29+00:00

When building production Retrieval-Augmented Generation (RAG) pipelines, the primary failure mode isn't retrieval latency or vector index size—**it's silent hallucination**. 

LLMs frequently generate fluent, highly plausible-sounding claims that are completely unsupported by the retrieved reference chunks. In multi-agent systems and mission-critical workflows, accepting an ungrounded draft can corrupt downstream decision-making.

To solve this in cyclic multi-agent graphs, I built and published [**`rag-critic`**](https://pypi.org/project/rag-critic/) on PyPI—a lightweight, adversarial hallucination auditor and citation grounding engine architected specifically for **LangGraph**.

Standard RAG architectures evaluate answers at a holistic paragraph level using an LLM-as-a-judge. This approach has three major flaws:

📦 Project Links & Open-Source Repositories

PyPI Package: [pypi.org/project/rag-critic](//pypi.org/project/rag-critic)

GitHub Repository: [github.com/samjanjua6/rag-critic](//github.com/samjanjua6/rag-critic)

Author Engineering Portfolio: [https://shaheerulhassan.com](//shaheerulhassan.com)

Install directly from PyPI:

``` python
bash
pip install rag-critic

from rag_critic import RAGCritic

critic = RAGCritic()

# Evaluate a draft answer against retrieved source chunks
result = critic.evaluate(
    draft_response="Python was released in 1991 by Guido van Rossum. It was originally named Monty.",
    retrieved_chunks=[
        "Python is an interpreted programming language released in 1991 by Guido van Rossum."
    ]
)

print(f"Status:              {result.status}")              # REVISE / APPROVED
print(f"Grounding Score:      {result.grounding_score:.2f}") # e.g. 0.50
print(f"Hallucination Index:  {result.hallucination_index:.2f}")

# Inspect claim-level verification
for claim in result.claims:
    print(f"[{claim.status.value}] {claim.text}")
    # [SUPPORTED] Python was released in 1991 by Guido van Rossum.
    # [UNGROUNDED] It was originally named Monty.

from langgraph.graph import StateGraph, END
from rag_critic import create_critic_node, critic_router

builder = StateGraph(MyRAGState)

# Add standard generator and retriever nodes
builder.add_node("retrieve", retrieve_docs)
builder.add_node("generate", generate_draft)

# Add rag-critic as an adversarial validation checkpoint
builder.add_node("critic", create_critic_node())

builder.add_edge("retrieve", "generate")
builder.add_edge("generate", "critic")

# Dynamic cyclic routing based on hallucination thresholds
builder.add_conditional_edges(
    "critic",
    critic_router,
    {
        "approved": END,              # Output is grounded; return to user
        "rewrite": "generate",        # Minor gaps; instruct re-writer
        "re_retrieve": "retrieve",    # Heavy hallucination; expand search
    }
)
```