Elamur Research⌕
☰
home / papers / 2021 / Enhancing Zero-shot and Few-shot Stance Detection with Commonsense Knowledge Graph

Enhancing Zero-shot and Few-shot Stance Detection with Commonsense Knowledge Graph

Summary

The paper addresses stance detection (labelling a text as Pro, Con or Neutral toward a topic) when a topic has no labelled examples (zero-shot) or only a handful (few-shot). Two difficulties stand out. Models trained only on annotated data generalize poorly, and authors often express stance without ever naming the topic. The authors propose CKE-Net to bridge document and topic with commonsense knowledge. A document–topic pair is encoded jointly with BERT. In parallel, concepts mentioned in the document and in the topic are matched to ConceptNet. The two-hop paths connecting them form a relational subgraph, which is then encoded with CompGCN so that both nodes and relation types are embedded. Attention-pooled graph features are concatenated with the text features and passed to an MLP classifier. On the VAST benchmark, CKE-Net beats BiLSTM-based models, fine-tuned BERT variants and a plain-GCN baseline in macro-F1. It also does best on hard cases such as implicit topics, quotations and sarcasm. This suggests that structured external knowledge reduces the need for annotation when many topics must be covered.

Key points

  • Data: experiments use VAST (VAried Stance Topics, Allaway & McKeown 2020), which covers thousands of topics.
    • Training: 13,477 examples, with 4,003 zero-shot and 638 few-shot topics.
    • Test: 3,006 examples, with 600 zero-shot and 159 few-shot topics.
    • Evaluation metric: macro-averaged F1.
  • Graph construction:
    • Concepts are matched to ConceptNet, which has 34 relation types.
    • The subgraph is built from two-hop directed paths between document concepts and topic concepts, with reverse edges added.
    • Node and relation features are initialized with TransE.
    • The subgraph is encoded by 2 CompGCN layers that use a subtraction-based composition.
  • Main results: CKE-Net reaches macro-F1 of 0.702 on zero-shot topics, 0.701 on few-shot topics and 0.701 overall. The strongest baseline, BERT-GCN, scores 0.686, 0.697 and 0.692 respectively. The original TGA-Net, with frozen BERT, scores 0.665 overall.
  • Challenging phenomena: CKE-Net achieves the highest accuracy on all five challenge subsets. Examples:
    • Implicit topic (Imp): 0.625, compared with 0.619 for BERT-GCN.
    • Quotations (Qte): 0.695, compared with 0.668 for BERT-GCN.
  • Setup: BERT-base, Adam optimizer, learning rate 4e-5, batch size 64. Documents are truncated to 200 words and topics to 5 words. Reported results are averaged over three random seeds.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

48 papers of our library cite it (of the 140673 whose bibliographies we read); 15 of them have a page here, 33 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-acl.278

@inproceedings{liu2021enhancing,
  title = {{Enhancing Zero-shot and Few-shot Stance Detection with Commonsense Knowledge Graph}},
  author = {Liu, Rui and Lin, Zheng and Tan, Yutong and Wang, Weiping},
  booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.278},
  url = {https://doi.org/10.18653/v1/2021.findings-acl.278}
}

References

22 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Emily Allaway and Kathleen McKeown. 2020. Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8913–8931, Online. Association for Computational Linguistics. find our card
  2. Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016a. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876– 885, Austin, Texas. Association for Computational Lingu… find our card
  3. Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016b. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 876– 885… find our card
  4. Antoine Bordes, Nicolas Usunier, Alberto Garcı́a-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of… find
  5. Erik Cambria, Soujanya Poria, Devamanyu Hazarika, and Kenneth Kwok. 2018. Senticnet 5: Discovering conceptual primitives for sentiment analysis by means of context embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications… find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference Jiachen Du, Ruifeng Xu, Yulan He, and Lin Gui. 2017. Stance classification with target-specific neural atten… find
  7. Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. The argument reasoning comprehension task: Identification and reconstruction of implicit warrants. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… find
  8. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. find
  9. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. find our card
  10. Jonathan Kobbe, Ioana Hulpus, and Heiner Stucken-schmidt. 2020. Unsupervised stance detection for arguments from consequences. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 50–60. Association for Computation… find our card
  11. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiao-Dan Zhu, and Colin Cherry. 2016a. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016, San Diego, CA, USA, June 16-17, 2016, pages 31– 41. The Asso… find
  12. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016b. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31– 41, San Diego, California. Association for Computational Lin… find
  13. Saif M. Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. 2017. Stance and sentiment in tweets. ACM Trans. Internet Techn., 17(3):26:1–26:23. find
  14. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 4444–4451. AAAI Press. find
  15. Bowen Zhang, Min Yang, Xutao Li, Yunming Ye, Xiaofei Xu, and Kuai Dai. 2020. Enhancing cross-target stance detection with transferable semantic-emotion knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3188–3197, Online. Association for Comp… find our card
  16. Jannis Vamvas and Rico Sennrich. 2020. X -stance: A multilingual multi-target dataset for stance detection. In Proceedings of the 5th Swiss Text Analytics Conference and the 16th Conference on Natural Language Processing, SwissText/KONVENS 2020, Zurich, Switzerland, June 23-25, 2020 [online only], v… find
  17. Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. 2020. Composition-based multi-relational graph convolutional networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. find
  18. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017,… find
  19. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenRe-view.net. find
  20. Penghui Wei and Wenji Mao. 2019. Modeling transferable topics for cross-target stance detection. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, pages 1173–1176. ACM. find
  21. Penghui Wei, Wenji Mao, and Guandan Chen. 2019. A topic-aware reinforced model for weakly supervised stance detection. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI… find
  22. Chang Xu, Cécile Paris, Surya Nepal, and Ross Sparks. 2018. Cross-target stance classification with self-attention networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers, pages 77… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/liu-2021-enhancing-zero-shot-few-shot/