Natural Language Autoencoders Identify Without Explaining

A paper posted to arXiv as 2610.32691v1 argues that Natural Language Autoencoders (NLAs), proposed as interpretability tools for AI alignment, fail to establish descriptive faithfulness because their round-trip reconstruction criterion only tests whether an explanation carries enough information to identify an activation, not whether it correctly describes what the activation does. The author compares the flaw to the chain-of-thought faithfulness problem, where verbal output trained to accompany a process is mistaken for a report on that process, and to the documented gap between human introspective reports and actual governing processes. The paper concludes that NLAs reintroduce, through unverified explanations, the epistemic problem they were intended to resolve.

arXiv:2610.32691v1 Announce Type: new Abstract: Natural Language Autoencoders NLAs express a model's internal activations in the form of readable text. The verbalizer turns an activation into an English explanation; the reconstructor then converts that explanation back into an activation, and the explanation is considered successful if the round trip brings back the original activation. We claim that this criterion evaluates whether the information is sufficient and does not address descriptive faithfulness. A sentence can contain sufficient information to make it possible to identify an activation even if it wrongly explains what the activation does: in the case of reconstruction, the primary aim is to accurately identify the activations and not to describe them. This issue has the same structure as the chain-of-thought faithfulness problem, in which verbal output that has been trained to go with a process is taken as a report on that process, and it parallels the dissociation between an individual's introspective report and the actual governing process as documented in human psychology. Although proposed as interpretability tools for the purpose of alignment, NLAs reintroduce, through unverified explanations, the epistemic problem that they were intended to resolve.