# Zero-Dimensional Attention Topology for Linguistic Acceptability

> Source: <https://www.machinebrief.com/news/zero-dimensional-attention-topology-for-linguistic-acceptabi-m105>
> Published: 2026-11-18 05:00:00+00:00

**Recommendation:** When evaluating attention-derived sentence representations, report out-of-domain Matthews correlation against both simple attention statistics and the full BERT baseline, and check whether features encode token count before claiming a topological advantage.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

arXiv:2611.22312v1 Announce Type: new 
Abstract: Topological summaries of Transformer attention provide sentence representations for linguistic acceptability classification. We examine a zero-dimensional persistent-homology ($H_0$) representation that summarizes minimum-spanning-tree merge values from 144 symmetrized BERT attention graphs into 1,440 features. The retrospective evaluation uses 8,517 downstream training sentences and 521 in-domain and 497 out-of-domain CoLA validation sentences after data-integrity filtering. The recorded out-of-domain Matthews correlation coefficients are 0.438 for the topological representation and 0.508 for attention statistics, compared with 0.545 for BERT. Attention-feature results are retained aggregate measurements; BERT predictions are rescored and length controls are refitted from the accompanying records. Analytical results show that the representation retains token-count information and discards a class of directional attention differences. Source resampling of BERT predictions also demonstrates the sensitivity of uncertainty estimates to the sampling unit. The reported results do not demonstrate a predictive advantage for this fixed topological representation over simpler attention summaries. They identify representation properties and evaluation controls relevant to subsequent studies of attention topology.
