Why Does Graph Learning Fail to Fully Benefit from a Text Teacher?

Researchers investigating a multimodal graph neural network (GNN) model that combines self-supervised pretraining with alternating optimization of a language model and GNN found that the combined approach failed to sufficiently improve predictive performance, identifying six factors including a strength-safety trade-off in the E-step anchor and conflicting optimization forces. The study, released on arXiv (2608.25741v1), provides experimental evidence for these limitations.

arXiv:2608.25741v1 Announce Type: cross Abstract: Graph neural networks GNNs are widely used to represent complex interactions and relationships among entities. We investigate a multimodal model that combines two complementary ideas: a self-supervised method that enables a GNN encoder pretrained on one dataset to operate directly on another dataset with a different node-feature dimensionality, without rebuilding the model or realigning the data; and an alternating optimization method that updates a language-model module in an E-step and a GNN module in an M-step, rather than jointly training a large language model and a GNN end to end on a large graph. Despite expectations, the combined model did not sufficiently improve predictive performance. We identify six factors: 1 an external anchor in the E-step has a strength-safety trade-off: a weak anchor has little effect, whereas an overly strong anchor can damage the graph representation; 2 the knowledge of the E-step teacher is not injected directly into the GCN embedding Z; 3 the representation space constructed in the M-step is not optimized for the same objective as the E-step teacher space, resulting in a compromise representation for target classification; 4 GCN propagation averages a node's own textual information with information from its neighbors; 5 cosine alignment does not guarantee axes that are discriminative for classification, so stronger geometric alignment with the E-step text anchor need not sufficiently improve the target decision boundary or classification performance; and 6 the force that preserves the source-side self-supervised geometry in the M-step conflicts with the force that moves the representation toward the E-step teacher. We support these observations through a staged set of experiments that varies the influence of the E-step.