Elamur Research⌕
☰
home / papers / 2022 / JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection

JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection

Summary

The paper addresses zero-shot stance detection (ZSSD): deciding whether a text supports, opposes or is neutral toward a target that never appeared in training. Earlier work used attention, adversarial training or commonsense-knowledge graphs to learn stance from context. Those methods did not exploit the idea that an unseen target can be understood through its relation to known targets. The authors propose JointCL, which couples two contrastive objectives on top of a BERT encoder.

  • Context-aware objective: a supervised stance contrastive loss pulls together examples with the same stance label and pushes apart the others.
  • Target-aware objective: prototypes are obtained by clustering the training representations. A prototypical graph is then built for each instance, so the prototypes act as a bridge between seen and unseen targets. An edge-oriented graph contrastive loss makes similar instances develop similar graph structures.

Across three benchmark datasets, including VAST and SEM16, JointCL reports state-of-the-art ZSSD results. It also performs well in few-shot and cross-target settings, which suggests the approach is a broadly reusable way to transfer stance knowledge to new topics.

Key points

  • The encoder is pre-trained BERT with input "[CLS] text [SEP] target [SEP]". The [CLS] vector serves as the instance representation.
  • The stance contrastive loss works within each mini-batch. It uses cosine similarity with a temperature, treating same-label pairs as positives.
  • The prototype construction follows prototypical contrastive learning (Li et al., 2021). The new edge-oriented loss operates on attention-score matrices, i.e., graph edges, rather than on node embeddings.
  • The authors also tested few-shot detection on VAST and cross-target detection on SEM16, where JointCL consistently beat the comparison methods. A t-SNE plot of 200 VAST test instances per label shows clearer Pro/Con separation than BERT-GCN.
  • Code is released at github.com/HITSZ-HLT/JointCL.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

46 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 37 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.acl-long.7

@inproceedings{liang2022jointcl,
  title = {{JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection}},
  author = {Liang, Bin and Zhu, Qinglin and Li, Xiang and Yang, Min and Gui, Lin and He, Yulan and Xu, Ruifeng},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.7},
  url = {https://doi.org/10.18653/v1/2022.acl-long.7}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Emily Allaway and Kathleen McKeown. 2020. Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8913– 8931, Online. Association for Computational Linguistics. find our card
  2. Eduardo Graells-Garrido, Ricardo Baeza-Yates, and Mounia Lalmas. 2020. Representativeness of abortion legislation debate on twitter: A case study in argentina and chile. In Companion Proceedings of the Web Conference 2020, page 765–774. find
  3. Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. 2021. Supervised contrastive learning for pre-trained language model fine-tuning. In International Conference on Learning Representations. on Semantic Evaluation (SemEval-2016), pages 31– 41, San Diego, California. Association for Comput… find our card
  4. Mitra Mohtarami, Ramy Baly, James Glass, Preslav Nakov, Lluís Màrquez, and Alessandro Moschitti. 2018. Automatic stance detection using end-to-end memory networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tec… find our card
  5. Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. IEEE. find
  6. Mohammad Kachuee, Hao Yuan, Young-Bum Kim, and Sungjin Lee. 2021. Self-supervised contrastive learning for efficient user satisfaction prediction in conversational agents. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… find
  7. Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, and Jie Zhou. 2021. ERICA: Improving entity and relation understanding for pre-trained language models via contrastive learning. In Proceedings of the 59th Annual Meeting of the Association for Computa… find
  8. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS… find
  9. Yao Qiu, Jinchao Zhang, and Jie Zhou. 2021. Improving gradient-based adversarial training for text classification by contrastive learning and auto-encoder. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1698–1707, Online. Association for Computational Linguistic… find
  10. Junnan Li, Pan Zhou, Caiming Xiong, and Steven Hoi. 2021. Prototypical contrastive learning of unsupervised representations. In International Conference on Learning Representations. find
  11. T. Y.S.S. Santosh, Srijan Bansal, and Avirup Saha. 2019. Can siamese networks help in stance detection? In Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, page 306–309. find
  12. Bin Liang, Yonghao Fu, Lin Gui, Min Yang, Jiachen Du, Yulan He, and Ruifeng Xu. 2021a. Target-adaptive graph for cross-target stance detection. In the Web Conference 2021 (WWW ’21). find
  13. Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4077–4087. find
  14. Bin Liang, Wangda Luo, Xiang Li, Lin Gui, Min Yang, Xiaoqi Yu, and Ruifeng Xu. 2021b. Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning, page 3242–3247. Association for Computing Machinery, New York, NY, USA. find
  15. Rui Liu, Zheng Lin, Yutong Tan, and Weiping Wang. 2021. Enhancing zero-shot and few-shot stance detection with commonsense knowledge graph. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3152–3157, Online. Association for Computational Linguistics. find our card
  16. Parinaz Sobhani, Diana Inkpen, and Xiaodan Zhu. 2017. A dataset for multi-target stance detection. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 551–557, Valencia, Spain. Association for Computational Lin… find our card
  17. Yixin Liu and Pengfei Liu. 2021. SimCLS: A simple framework for contrastive learning of abstractive summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Pa… find our card
  18. Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124, Los Angeles, CA. Association for Computational Linguistics. find
  19. Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive multiview coding. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, pages 776–794. Springer. find
  20. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Re… find
  21. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5419–5430, Online. Association for Computational Linguistics. find
  22. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. find
  23. Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, and Jie Zhou. 2021. CLEVE: Contrastive Pre-training for Event Extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on N… find
  24. Zhirong Wu, Yuanjun Xiong, Stella X. Yu, and Dahua Lin. 2018. Unsupervised feature learning via non-parametric instance discrimination. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 3733–3742. IEEE Computer Society. find
  25. Chang Xu, Cécile Paris, Surya Nepal, and Ross Sparks. 2018. Cross-target stance classification with self-attention networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 778–783, Melbourne, Australia. Association for Compu… find
  26. Nan Yang, Furu Wei, Binxing Jiao, Daxing Jiang, and Linjun Yang. 2021. xMoCo: Cross momentum contrastive learning for open-domain question answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lang… find
  27. Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812– 5823. find
  28. Bowen Zhang, Min Yang, Xutao Li, Yunming Ye, Xiaofei Xu, and Kuai Dai. 2020. Enhancing cross-target stance detection with transferable semantic-emotion knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3188–3197, Online. Association for Comp… find our card
  29. Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nal-lapati, Andrew O. Arnold, and Bing Xiang. 2021. Supporting clustering with contrastive learning. In find

↑ back to top

This page in Markdown →