NxN E-valuation: Hypothesis Certification via a Conformal CRT Null

Researchers propose NxN E-valuation, an e-value-based hypothesis-certification algorithm that verifies hypotheses without building case-specific null hypotheses, using a large dataset where samples serve as nulls for each other. The method, detailed in arXiv:2608.06621v1, is designed for LLM-based exploration systems to counter hallucination and can replace circular verification and held-out testing.

arXiv:2608.06621v1 Announce Type: new Abstract: We propose NxN E-valuation, a handy, e-value-based hypothesis-certification algorithm that lets a hypothesis be verified without building any case-specific certification procedure---such as constructing a dedicated null hypothesis---as long as a large enough dataset is available. The method is especially suited to LLM-based exploration systems, where LLMs are remarkably good at proposing hypotheses but suffer badly from hallucination; this hallucination prevents us from harvesting LLM outputs directly, and existing remedies each fall short. The most common solutions include letting the LLM verify or correct itself circular verification and held-out testing where false hypotheses can still pass via spurious correlations , among other remedies detailed in the introduction. To resolve this, NxN E-valuation exploits the naturally existing large training set and lets different samples serve as null hypotheses for one another. This design directly realizes a conditional randomization test CRT that certifies each hypothesis. The approach can be a universally better replacement for at least LLM circular verification and held-out-data testing, provided the LLM's generations are hypotheses that apply to each individual sample.