Elamur Research⌕
☰
home / papers / 2023 / Stance Detection on Social Media with Background Knowledge

Stance Detection on Social Media with Background Knowledge

Summary

Social media posts often take a stance on a target without spelling out the context needed to read them. That context can be facts about the target or the meaning of hashtags, acronyms, nicknames and slang. The authors split this missing context into episodic knowledge and discourse knowledge and propose KASD (Knowledge-Augmented Stance Detection).

For episodic knowledge, they build a Wikipedia knowledge base for each target and fit an LDA topic model on that target's posts. A topic-weighted TF-IDF score then selects candidate passages, and ChatGPT removes the irrelevant parts. For discourse knowledge, ChatGPT rewrites each post so that implicit references become explicit. The enriched input goes either to a fine-tuned encoder such as RoBERTa or to ChatGPT itself.

On four benchmarks, KASD reports state-of-the-art results in both in-target and zero-shot settings. Knowledge-augmented small models even outperform ChatGPT. This suggests that an LLM's understanding and knowledge can be transferred into much cheaper classifiers.

Key points

  • Evaluation covers four datasets: Sem16, P-Stance, Covid19 (tweets) and VAST, in both in-target and zero-shot stance detection.
  • The knowledge base uses the top 10 Wikipedia pages per target, split by section into documents of roughly 400 words. Retrieval combines LDA topics (sklearn, online variational Bayes) with heuristic TF-IDF matching.
  • In zero-shot P-Stance, KASD-RoBERTa-large reaches an average of 83.10, compared with 81.50 for ChatGPT and 73.35 for plain RoBERTa-large. It does so with about 500–1000× fewer parameters.
  • Three evaluators judged 500 sampled outputs. ChatGPT-generated knowledge was rated good in over 95% of cases: 96.00% for episodic knowledge, 95.13% for the redundancy filtering and 96.87% for discourse knowledge.
  • Code is released at github.com/HITSZ-HLT/KA-Stance-Detection. One limitation is that the knowledge source is limited to Wikipedia and does not use timelier sources such as news.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

26 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 23 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.emnlp-main.972

@inproceedings{li2023stance,
  title = {{Stance Detection on Social Media with Background Knowledge}},
  author = {Li, Ang and Liang, Bin and Zhao, Jingqian and Zhang, Bowen and Yang, Min and Xu, Ruifeng},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.972},
  url = {https://doi.org/10.18653/v1/2023.emnlp-main.972}
}

References

33 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Emily Allaway and Kathleen R. McKeown. 2020. Zero-shot stance detection: A dataset and model using generalized topic representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 8913–8931. Association for… find our card
  2. Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876–885, Austin, Texas. Association for Computational Linguist… find our card
  3. Pengyuan Chen, Kai Ye, and Xiaohui Cui. 2021. Integrating n-gram features into pre-trained model: a novel ensemble model for multi-target stance detection. In International Conference on Artificial Neural Networks, pages 269–279. Springer. find
  4. Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2022. Incorporating stock market signals for twitter stance detection. In Proceedings of https://developer.twitter.com/en/docs/twitter-api the 60th Annual Meeting of the Association fo… ↗
  5. Javid Ebrahimi, Dejing Dou, and Daniel Lowd. 2016. Weakly supervised tweet stance classification by relational bootstrapping. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1012–1017. find
  6. Tianqing Fang, Hongming Zhang, Weiqi Wang, Yangqiu Song, and Bin He. 2021. DISCOS: bridging the gap between discourse knowledge and commonsense knowledge. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pages 2648–2659. ACM / IW3C2. find
  7. Shalmoli Ghosh, Prajwal Singhania, Siddharth Singh, Koustav Rudra, and Saptarshi Ghosh. 2019. Stance detection in web and social media: A comparative study. In Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Lu… find
  8. Kyle Glandt, Sarthak Khanal, Yingjie Li, Doina Caragea, and Cornelia Caragea. 2021. Stance detection in COVID-19 tweets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2… find our card
  9. Kazuaki Hanawa, Akira Sasaki, Naoaki Okazaki, and Kentaro Inui. 2019. Stance detection attending external knowledge from wikipedia. J. Inf. Process., 27:499–506. find
  10. Kazi Saidul Hasan and Vincent Ng. 2013. Stance classification of ideological debates: Data, models, features, and constraints. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 1348–1356. find
  11. Zihao He, Negar Mokhberian, and Kristina Lerman. 2022. Infusing knowledge from Wikipedia to enhance stance detection. In Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, pages 71–77, Dublin, Ireland. Association for Computational Lingui… find our card
  12. Matthew D. Hoffman, David M. Blei, and Francis R. Bach. 2010. Online learning for latent dirichlet allocation. In Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010. Proceedings of a meeting held 6-9 December 2010, Vancouver, Br… find
  13. Hu Huang, Bowen Zhang, Yangyang Li, Baoquan Zhang, Yuxi Sun, Chuyao Luo, and Cheng Peng. 2023. Knowledge-enhanced prompt-tuning for stance detection. ACM Transactions on Asian and Low-Resource Language Information Processing. find
  14. Yang Li and Jiawei Yuan. 2022. Generative data augmentation with contrastive learning for zero-shot stance detection. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 6985–6995. Associat… find our card
  15. Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Jayara-man Nair, Diana Inkpen, and Cornelia Caragea. 2021. P-stance: A large dataset for stance detection in political domain. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJC… find our card
  16. Bin Liang, Zixiao Chen, Lin Gui, Yulan He, Min Yang, and Ruifeng Xu. 2022a. Zero-shot stance detection via contrastive learning. In Proceedings of the ACM Web Conference 2022, pages 2738–2747. find
  17. Bin Liang, Qinglin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, and Ruifeng Xu. 2022b. Jointcl: A joint contrastive learning framework for zero-shot stance detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin,… find our card
  18. Rui Liu, Zheng Lin, Yutong Tan, and Weiping Wang. 2021. Enhancing zero-shot and few-shot stance detection with commonsense knowledge graph. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3152–3157, Online. Association for Computational Linguistics. find our card
  19. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. find
  20. Yunpu Ma, Volker Tresp, and Erik A. Daxberger. 2019. Embedding models for episodic knowledge graphs. J. Web Semant., 59. find
  21. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiao-Dan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016, San Diego, CA, USA, June 16-17, 2016, pages 31–41. The Ass… find
  22. Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. Bertweet: A pre-trained language model for english tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2020 -Demos, Online, November 16-20, 2020, pages 9–14. Association… find our card
  23. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 1… find our card
  24. Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124, Los Angeles, CA. Association for Computational Linguistics. find
  25. Peter Stefanov, Kareem Darwish, Atanas Atanasov, and Preslav Nakov. 2020. Predicting the topical stance and political leaning of media using tweets. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 527–537, Online. Association for Computational Lingui… find
  26. Qingying Sun, Zhongqing Wang, Qiaoming Zhu, and Guodong Zhou. 2018. Stance detection with hierarchical attention network. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2399–2409, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card
  27. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. CoRR, ab… find
  28. Hanzi Xu, Slobodan Vucetic, and Wenpeng Yin. 2022. Openstance: Real-world zero-shot stance detection. CoRR, abs/2210.14299. find our card
  29. Bowen Zhang, Xianghua Fu, Daijun Ding, Hu Huang, Yangyang Li, and Liwen Jing. 2023. Investigating chain-of-thought with chatgpt for stance detection on social media. CoRR, abs/2304.03087. find
  30. Bowen Zhang, Min Yang, Xutao Li, Yunming Ye, Xiaofei Xu, and Kuai Dai. 2020. Enhancing cross-target stance detection with transferable semantic-emotion knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3188–3197, Online. Association for Comp… find our card
  31. Kai Zheng, Qingfeng Sun, Yaming Yang, and Fei Xu. 2022. Knowledge stimulated contrastive prompting for low-resource stance detection. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 1168–1178. Association for Compu… find
  32. Lixing Zhu, Zheng Fang, Gabriele Pergola, Robert Proc-ter, and Yulan He. 2022a. Disentangled learning of stance and aspect topics for vaccine attitude detection in social media. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Huma… find
  33. Qinglin Zhu, Bin Liang, Jingyi Sun, Jiachen Du, Lanjun Zhou, and Ruifeng Xu. 2022b. Enhancing zero-shot stance detection via targeted background knowledge. In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 202… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2023-stance-detection-social-media-background/