{"slug": "zero-dimensional-attention-topology-for-linguistic-acceptability", "title": "Zero-Dimensional Attention Topology for Linguistic Acceptability", "summary": "A zero-dimensional persistent-homology (H0) representation built from 144 symmetrized BERT attention graphs into 1,440 features failed to beat simpler attention summaries on linguistic acceptability classification, according to arXiv paper 2611.22312v1. The topological representation recorded an out-of-domain Matthews correlation coefficient of 0.438 versus 0.508 for attention statistics and 0.545 for BERT, evaluated on 8,517 downstream training sentences and 521 in-domain plus 497 out-of-domain CoLA validation sentences after data-integrity filtering. The authors report that the representation retains token-count information and discards a class of directional attention differences, and that their results do not demonstrate a predictive advantage for the fixed topological representation over simpler attention summaries.", "body_md": "arXiv:2611.22312v1 Announce Type: new \nAbstract: Topological summaries of Transformer attention provide sentence representations for linguistic acceptability classification. We examine a zero-dimensional persistent-homology ($H_0$) representation that summarizes minimum-spanning-tree merge values from 144 symmetrized BERT attention graphs into 1,440 features. The retrospective evaluation uses 8,517 downstream training sentences and 521 in-domain and 497 out-of-domain CoLA validation sentences after data-integrity filtering. The recorded out-of-domain Matthews correlation coefficients are 0.438 for the topological representation and 0.508 for attention statistics, compared with 0.545 for BERT. Attention-feature results are retained aggregate measurements; BERT predictions are rescored and length controls are refitted from the accompanying records. Analytical results show that the representation retains token-count information and discards a class of directional attention differences. Source resampling of BERT predictions also demonstrates the sensitivity of uncertainty estimates to the sampling unit. The reported results do not demonstrate a predictive advantage for this fixed topological representation over simpler attention summaries. They identify representation properties and evaluation controls relevant to subsequent studies of attention topology.", "url": "https://wpnews.pro/news/zero-dimensional-attention-topology-for-linguistic-acceptability", "canonical_source": "https://www.machinebrief.com/news/zero-dimensional-attention-topology-for-linguistic-acceptabi-m105", "published_at": "2026-11-18 05:00:00+00:00", "updated_at": "2026-11-18 05:16:31.767203+00:00", "lang": "en", "topics": ["natural-language-processing", "machine-learning", "ai-research"], "entities": ["BERT", "CoLA", "arXiv"], "also_reported_by": [], "takeaway": "A 1,440-feature H0 topological summary of 144 symmetrized BERT attention graphs scored 0.438 out-of-domain Matthews correlation on CoLA versus 0.508 for attention statistics and 0.545 for BERT, and it retains token-count information while discarding directional attention differences.", "recommendation": {"advice": "When evaluating attention-derived sentence representations, report out-of-domain Matthews correlation against both simple attention statistics and the full BERT baseline, and check whether features encode token count before claiming a topological advantage.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/zero-dimensional-attention-topology-for-linguistic-acceptability", "markdown": "https://wpnews.pro/news/zero-dimensional-attention-topology-for-linguistic-acceptability.md", "text": "https://wpnews.pro/news/zero-dimensional-attention-topology-for-linguistic-acceptability.txt", "jsonld": "https://wpnews.pro/news/zero-dimensional-attention-topology-for-linguistic-acceptability.jsonld"}}