Elamur Research⌕
☰
home / papers / 2025 / Text-Attributed Graph Learning with Coupled Augmentations

Text-Attributed Graph Learning with Coupled Augmentations

Paper · 2025 · Chuang Zhou, Jiahe Du, Huachi Zhou et al.

Summary

Text-attributed graphs are networks whose nodes carry text, such as product descriptions or paper abstracts. They are usually modeled from one side only: a language model reads the node text, or a GNN exploits connectivity. Chaining both is computationally expensive. The authors show that this produces complementary errors. On Amazon, about 9% of nodes are classified correctly only by the graph model ("text bias"), and 11% only by the text model ("graph bias").

They propose CTGL, which trains a text model and a GNN side by side and links them through Coupled Text & Graph Augmentation (CTGA). Positive views mix a node's text with text from its graph neighbors, and negative views mix it with text from non-neighbors. Each model gets an extra contrastive loss that pulls original and positive embeddings together and pushes negative ones apart. This loss is weighted by λ (text) and γ (graph) and added to each model's usual objective, which also lets the framework use unlabeled nodes. On three datasets CTGL beats text-only and graph-only baselines. This suggests that lightweight parallel coupling can reduce both biases without serial architectures.

Key points

  • Bias analysis on Amazon: both models are correct for about 70% of nodes, only the graph model for 9%, and only the text model for 11%.
  • On Wiki, the grid search over the loss weights gave the best accuracy at λ = 1.5 and γ = 1. Accuracy drops when both are set to zero, and a broad plateau suggests low hyperparameter sensitivity.
  • In the augmentation ablation, accuracy rose from .649 (random positive sampling, no negatives) to .668 (distance-normalized sampling plus negative samples). The dataset for this table is not named in the extracted text.
  • The paper reports overhead as modest: offline CPU sampling takes under 5 minutes per 100,000 nodes, and full training finishes within about 30 minutes.
  • Of the three datasets, one is an existing open resource and two were built by the authors with anonymized consumer data. The contributions list nonetheless calls all three "public."

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Chuang Zhou, Jiahe Du, Huachi Zhou, Hao Chen, Feiran Huang and Xiao Huang. 2025. Text-Attributed Graph Learning with Coupled Augmentations. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zhou2025text,
  title = {{Text-Attributed Graph Learning with Coupled Augmentations}},
  author = {Chuang Zhou, Jiahe Du, Huachi Zhou, Hao Chen, Feiran Huang and Xiao Huang},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.722/}
}

References

34 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. William Brannon, Wonjune Kang, Suyash Fulay, Hang Jiang, Brandon Roy, Deb Roy, and Jad Kabbara. 2023. Congrat: Self-supervised contrastive pretraining for joint graph and text embeddings. arXiv preprint arXiv:2305.14321. find
  2. Shaked Brody, Uri Alon, and Eran Yahav. 2022. How attentive are graph attention networks? In International Conference on Learning Representations. find
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  4. Benjamin Paul Chamberlain, James Rowbottom, Maria I. Gorinova, Stefan D Webb, Emanuele Rossi, and Michael M. Bronstein. 2021. GRAND: Graph 1 https://www.aclweb.org/portal/content/ acl-code-ethics Jie Chen, Tengfei Ma, and Cao Xiao. 2018. FastGCN: Fast learning with graph convolutional networks via i… ↗
  5. Shengyuan Chen, Qinggang Zhang, Junnan Dong, Wen Hua, Qing Li, and Xiao Huang. 2024. Entity alignment with noisy annotations from large language models. arXiv preprint arXiv:2405.16806. — [arXiv:2405.16806](https://arxiv.org/abs/2405.16806) find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019a. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019b. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  8. Junnan Dong, Qinggang Zhang, Chuang Zhou, Hao Chen, Daochen Zha, and Xiao Huang. 2024a. Cost-efficient knowledge-based question answering with large language models. arXiv preprint arXiv:2405.17337. — [arXiv:2405.17337](https://arxiv.org/abs/2405.17337) find
  9. Junnan Dong, Qinggang Zhang, Huachi Zhou, Daochen Zha, Pai Zheng, and Xiao Huang. 2024b. Modality-aware integration with large language models for knowledge-based visual question answering. arXiv preprint arXiv:2402.12728. find
  10. Yaakov HaCohen-Kerner, Daniel Miller, and Yair Yigal. 2020. The influence of preprocessing on text classification using a bag-of-words representation. PloS one, 15(5):e0232525. find
  11. William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, page 1025–1035.
  12. Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web, page 507–517. find
  13. Zhongyu Huang, Yingheng Wang, Chaozhuo Li, and Huiguang He. 2022. Going deeper into permutation-sensitive graph neural networks. In International Conference on Machine Learning, pages 9377–9409. PMLR. find
  14. Bowen Jin, Yu Zhang, Yu Meng, and Jiawei Han. 2023. Edgeformers: Graph-empowered transformers for representation learning on textual-edge networks. In The Eleventh International Conference on Learning Representations. find
  15. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations.
  16. Yichuan Li, Kaize Ding, and Kyumin Lee. 2023. Grenade: Graph-centric language model for self-supervised representation learning on text-attributed graphs. arXiv preprint arXiv:2310.15109. find
  17. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Interspeech, volume 2, pages 1045–1048. Makuhari. find
  18. Giannis Nikolentzos, Antoine Tixier, and Michalis Vazirgiannis. 2020. Message passing attention networks for document understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8544–8551. find
  19. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing, pages 1532–1543.
  20. Qiaoyu Tan, Xin Zhang, Jiahe Du, and Xiao Huang. 2023. Graph neural networks with non-recursive message passing. In 2023 IEEE International Conference on Data Mining Workshops (ICDMW), pages 506–514. IEEE. find
  21. Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: Extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, page 990–998. find
  22. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In International Conference on Learning Representations. find
  23. Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep graph infomax. In International Conference on Learning Representations. find
  24. Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In International conference on machine learning, pages 6861–6871. find
  25. Michihiro Yasunaga, Jure Leskovec, and Percy Liang. 2022. LinkBERT: Pretraining language models with document links. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8003–8016, Dublin, Ireland. Association for Computational Lin…
  26. Anthony Zaknich. 1998. Introduction to the modified probabilistic neural network for general signal processing applications. IEEE Transactions on Signal Processing, 46(7):1980–1990. find
  27. Peiyan Zhang, Jiayan Guo, Chaozhuo Li, Yueqi Xie, Jae Boum Kim, Yan Zhang, Xing Xie, Haohan Wang, and Sunghun Kim. 2023. Efficiently leveraging multi-level user intent for session-based recommendation via atten-mixer network. In Proceedings of the Sixteenth ACM International Conference on Web Search… find
  28. Yifei Zhang, Hao Zhu, Zixing Song, Piotr Koniusz, and Irwin King. 2022. Costa: Covariance-preserving feature augmentation for graph contrastive learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 2524–2534. find
  29. Yin Zhang, Rong Jin, and Zhi-Hua Zhou. 2010. Understanding bag-of-words model: a statistical framework. International journal of machine learning and cybernetics, 1:43–52. find
  30. Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. 2023. Learning on large-scale text-attributed graphs via variational inference. In The Eleventh International Conference on Learning Representations. find
  31. Chuang Zhou, Junnan Dong, Xiao Huang, Zirui Liu, Kaixiong Zhou, and Zhaozhuo Xu. 2024. Quest: Efficient extreme multi-label text classification with large language models on commodity hardware. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3929–3940. find
  32. Huachi Zhou, Hao Chen, Junnan Dong, Daochen Zha, Chuang Zhou, and Xiao Huang. 2023. Adaptive popularity debiasing aggregator for graph collaborative filtering.(2023). A APPENDICES A, 1. find
  33. Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, page 2069–2080. find
  34. Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit S, Guangzhong Sun, and Xing Xie. 2021. Graphformers: GNN-nested transformers for representation learning on textual graph. In Advances in Neural Information Processing Systems. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2025-text-attributed-graph-learning-coupled/