Your LLM gave you an answer. Should your application trust it?
I built BOOTH, a lightweight checkpoint layer for LLM outputs.
The idea is simple: don't automatically pass every model response downstream. Check it first.
For example:
Evidence: Returns are allowed within 45 days.
LLM: Returns are allowed within 90 days.
The answer sounds confident. It's also unsupported by the evidence.
BOOTH's check_with_evidence() lets you check an LLM response against evidence your RAG pipeline has already retrieved.
result = booth.check_with_evidence(
answer=llm_answer,
evidence=retrieved_docs,
compare_fn=your_comparison_function,
)
No need to replace your existing RAG pipeline or commit to a particular LLM provider.
Zero runtime dependencies. Provider-agnostic. Small API.
pip install boothpy
GitHub: https://github.com/Vedantgitbot/booth
How are you currently deciding whether an LLM output is safe to pass downstream?
Beta · 2K+ PyPI downloads · 300+ tests · CI passing · MIT · Python 3.9+