Diagnosing Compositional Generalization in Transformers on ReCOGS with Compositional Graph Similarity
Researchers Bruno Franco and Edson Scalabrin introduced Compositional Graph Similarity (CGS), a graph-based metric for evaluating Transformer models on the ReCOGS benchmark, and found that the lowest-scoring categories a…