LLM Hallucination Detection via Low-rank Multilinear Decomposition

A new arXiv paper (2610.36191v1) proposes LOTUS (Low-rank Order-statistic TUcker-based Subspace learning), a multilinear low-rank decomposition framework for detecting hallucinations in large language models by extracting compact representations from hidden states. LOTUS summarizes variable-length token representations with statistical descriptors capturing peak activations, activation distributions and cross-layer changes, then applies low-rank multilinear decomposition to train a lightweight hallucination classifier while reducing dependence on labeled data. The authors report improved hallucination detection over recent state-of-the-art baselines across multiple LLMs and question-answering benchmarks.

arXiv:2610.36191v1 Announce Type: new Abstract: Large language models LLMs frequently generate fluent yet factually incorrect responses, raising concerns about their reliability. Recent hallucination detectors increasingly exploit internal representations, but extracting informative signals from high-dimensional hidden states remains challenging. Existing approaches often rely on selected activations or pooling-based compression, potentially overlooking discriminative information distributed across Transformer layers, feature modes, and tokens. We propose LOTUS Low-rank Order-statistic TUcker-based Subspace learning , a multilinear low-rank decomposition framework for hallucination detection that extracts compact representations from LLM hidden states while reducing dependence on labeled data. Our approach first summarizes variable-length token representations using complementary statistical descriptors that capture peak activations, activation distributions, and cross-layer changes. We then leverage low-rank multilinear decomposition to extract compact representations that capture the underlying structure of LLM hidden states, which are subsequently used to train a lightweight hallucination classifier. Experiments across multiple LLMs and question-answering benchmarks demonstrate improved hallucination detection performance over recent state-of-the-art baselines.