Scoped Semantic Invariance as a Foundation for LLM Reliability

A new arXiv paper (2610.33267v1) formalizes LLM consistency as semantic invariance over equivalence classes of queries and proves that invariance under arbitrary meaning-preserving transformations is undecidable in general, motivating a scoped version evaluated against bounded semantic domains. Testing across the MITRE D3FEND ontology and eight Wikidata-derived domains, the authors find inconsistencies are structured rather than random, concentrating in localized hotspot subgraphs organized around recurring TREE, STAR and CYCLE patterns that stay stable across models and domains. Selective graph grounding improved correctness and cross-model consistency while indiscriminate grounding caused regressions, and a layered grounding strategy produced the strongest monotonicity profile, with hotspot structures from simple consistency probes transferring to more realistic compositional reasoning tasks.

arXiv:2610.33267v1 Announce Type: new Abstract: LLMs often answer semantically equivalent queries inconsistently, even when the underlying facts are known. We formalize consistency as semantic invariance over equivalence classes of queries and show that invariance under arbitrary meaning-preserving transformations is undecidable in general. This motivates scoped semantic invariance, where model behavior is evaluated relative to bounded semantic domains. We operationalize this framework using knowledge graphs KGs , which act as explicit semantic control structures defining local semantic constraints and grounding contexts. Across the MITRE D3FEND ontology and eight Wikidata- derived domains, we show that inconsistencies are structured rather than random, concentrating in localized hotspot subgraphs organized around recurring TREE, STAR, and CYCLE patterns that remain stable across models and domains. We further show that selective graph grounding improves correctness and cross-model consistency, while indiscriminate grounding introduces regressions. A layered grounding strategy achieves the strongest monotonicity profile, preserving correct knowledge while reducing inconsistency. Finally, we show that hotspot structures discovered from simple consistency probes transfer to more realistic compositional reasoning tasks.