Elamur Research⌕
☰
home / papers / 2023 / A New Direction in Stance Detection: Target-Stance Extraction in the Wild

A New Direction in Stance Detection: Target-Stance Extraction in the Wild

Summary

Most stance detection systems assume the opinion's target is supplied with the text. On social media, however, targets are often only implied, and labeling them by hand at scale is impractical. The authors therefore define Target-Stance Extraction (TSE), which predicts both the target and the stance from raw text alone.

To benchmark TSE they build a two-stage pipeline:

  • Stage one identifies the target in one of two ways: classifying the text into a fixed target set, or generating keyphrases with BART fine-tuned on keyphrase corpora and mapping them onto the predefined list.
  • Stage two classifies stance toward the predicted target, with target prediction added as an auxiliary task in a multi-task setup.

Evaluation covers an in-target setting built by merging four stance corpora and a newly assembled zero-shot set of six unseen targets. Both include an extra Unrelated class.

The task proves difficult. In zero-shot TSE, stance scores with generated targets are roughly half those obtained with gold targets. The multi-task objective helps consistently. The contribution is mainly a more realistic problem formulation, and the code is publicly released.

Key points

  • In-target data: SemEval-2016, AM, COVID-19 (C19) and P-Stance are merged into 18 targets. Abortion targets are unified, and Climate Change is held out for zero-shot.
  • Zero-shot data: Each of six targets has 500 samples, for 3,000 test examples in total: the MeToo Movement, the Disney–Fox merger, the New York State lockdown, Creationism, Gay Rights, and the view that climate change is a concern. The Unrelated class is kept at a 5:1 ratio of targets of interest to Unrelated.
  • Target generation: BART is fine-tuned separately on OpenKP, KPTimes and FullTextKP using a One2Seq approach. KPTimes-based generation gave the best downstream stance results.
  • Zero-shot results: the best model, TGA-Net with KPTimes, reaches 22.83 F1, versus 40.94 F1 with ground-truth targets. The stance classifiers compared are BiLSTM, BiCond, TAN, CrossNet, TGA-Net and BERTweet.
  • Multi-task gains: the auxiliary target-prediction task raises average F (F_avg) by 2.35 on SemEval-2016, 0.80 on AM, 2.95 on COVID-19 and 0.92 on P-Stance. The largest gains appear on the smaller datasets. Code: github.com/chuchun8/TSE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

15 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 13 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-long.560

@inproceedings{li2023new,
  title = {{A New Direction in Stance Detection: Target-Stance Extraction in the Wild}},
  author = {Li, Yingjie and Garg, Krishna and Caragea, Cornelia},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.560},
  url = {https://doi.org/10.18653/v1/2023.acl-long.560}
}

References

55 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abeer ALDayel and Walid Magdy. 2021. Stance detection on social media: State of the art and trends. International Journal on Information Processing and Management, 58(4). find
  2. Emily Allaway and Kathleen McKeown. 2020. Zero-shot stance detection: A dataset and model using generalized topic representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8913– 8931. find our card
  3. Rabah Alzaidy, Cornelia Caragea, and C. Lee Giles. 2019. Bi-lstm-crf sequence labeling for keyphrase extraction from scholarly documents. In The World Wide Web Conference, WWW ’19, page 2551–2557, New York, NY, USA. Association for Computing Machinery. find
  4. Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876–885. find our card
  5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  6. Cornelia Caragea, Florin Adrian Bulgarov, Andreea Godea, and Sujatha Das Gollapalli. 2014. Citation-enhanced keyphrase extraction from research papers: A supervised approach. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1435–1446, Doha, Qat… find
  7. Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2020. Will-they-won’t-they: A very large dataset for stance detection on Twitter. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1715… find our card
  8. Kareem Darwish, Walid Magdy, and Tahar Zanouda. 2017. Trump vs. Hillary: What went viral during the 2016 US presidential election. In Social Informatics, pages 143–161. find
  9. Leon Derczynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. 2017. SemEval-2017 task 8: RumourEval: Determining rumour veracity and support for rumours. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 69–76… find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  11. Jiachen Du, Ruifeng Xu, Yulan He, and Lin Gui. 2017. Stance classification with target-specific neural attention networks. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pages 3988–3994. find
  12. Corina Florescu and Cornelia Caragea. 2017. Posi-tionRank: An unsupervised approach to keyphrase extraction from scholarly documents. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1105–1115, Vancouver, Canada. Association fo… find
  13. Ygor Gallina, Florian Boudin, and Beatrice Daille. 2019. KPTimes: A large-scale dataset for keyphrase generation on news documents. In Proceedings of the 12th International Conference on Natural Language Generation, pages 130–135. find
  14. Krishna Garg, Jishnu Ray Chowdhury, and Cornelia Caragea. 2022. Keyphrase generation beyond the boundaries of title and abstract. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5809–5821, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  15. Akash Gautam, Puneet Mathur, Rakesh Gosangi, Debanjan Mahata, Ramit Sawhney, and Rajiv Ratn Shah. 2020. \#MeTooMA: Multi-aspect annotations of tweets related to the MeToo movement. Proceedings of the International AAAI Conference on Web and Social Media, 14(1):209–216. find
  16. Kyle Glandt, Sarthak Khanal, Yingjie Li, Doina Caragea, and Cornelia Caragea. 2021. Stance detection in COVID-19 tweets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Lo… find our card
  17. Genevieve Gorrell, Elena Kochkina, Maria Liakata, Ahmet Aker, Arkaitz Zubiaga, Kalina Bontcheva, and Leon Derczynski. 2019. SemEval-2019 task 7: RumourEval, determining rumour veracity and support for rumours. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 845–854. find
  18. Lara Grimminger and Roman Klinger. 2021. Hate towards the political opponent: A Twitter corpus study of the 2020 US elections on the basis of offensive speech and stance detection. In Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysi… find our card
  19. Momchil Hardalov, Arnav Arora, Preslav Nakov, and Isabelle Augenstein. 2021. Cross-domain label-adaptive stance detection. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9011–9028. find our card
  20. Momchil Hardalov, Arnav Arora, Preslav Nakov, and Isabelle Augenstein. 2022. A survey on stance detection for mis- and disinformation identification. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1259–1277. find our card
  21. Kazi Saidul Hasan and Vincent Ng. 2014. Why are you taking this stance? Identifying and classifying reasons in ideological debates. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 751–762. find
  22. Muhammad Naeem Khan, Muhammad Azeem Ashraf, Donald Seinen, Kashif Ullah Khan, and Rizwan Ahmed Laar. 2021. Social media for knowledge acquisition and dissemination: The impact of the COVID-19 pandemic on collaborative learning driven social media adoption. Frontiers in Psychology, 12. find
  23. Dilek Küçük and Fazli Can. 2020. Stance detection: A survey. ACM Comput. Surv., 53(1):1–37. find
  24. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the… find our card
  25. Yingjie Li and Cornelia Caragea. 2019. Multi-task stance detection with sentiment and stance lexicons. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6298–6304. find our card
  26. Yingjie Li and Cornelia Caragea. 2021a. A multi-task learning framework for multi-target stance detection. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2320–2326. find our card
  27. Yingjie Li and Cornelia Caragea. 2021b. Target-aware data augmentation for stance detection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1850–1860. find our card
  28. Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Ja-yaraman Nair, Diana Inkpen, and Cornelia Caragea. 2021a. P-Stance: A large dataset for stance detection in political domain. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2355–2365. find our card
  29. Yingjie Li, Chenye Zhao, and Cornelia Caragea. 2021b. Improving stance detection with multi-dataset learning and knowledge distillation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6332–6345. find our card
  30. Yingjie Li, Chenye Zhao, and Cornelia Caragea. 2023. Tts: A target-based teacher-student framework for zero-shot stance detection. In Proceedings of the ACM Web Conference 2023, page 1500–1509. find
  31. Bin Liang, Yonghao Fu, Lin Gui, Min Yang, Jiachen Du, Yulan He, and Ruifeng Xu. 2021. Target-adaptive graph for cross-target stance detection. In Proceedings of the Web Conference 2021, page 3453–3464. find
  32. Bin Liang, Qinglin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, and Ruifeng Xu. 2022. JointCL: A joint contrastive learning framework for zero-shot stance detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 81–91. find our card
  33. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. find
  34. Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017. Deep keyphrase generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 582–592, Vancouver, Canada. Association for Computational Lingui… find
  35. Lin Miao, Mark Last, and Marina Litvak. 2020. Twitter data augmentation for monitoring public opinion on COVID-19 intervention measures. In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020. find
  36. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016a. A dataset for detecting stance in tweets. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3945– 3952. find
  37. Christian Stab, Tristan Miller, Benjamin Schiller, Pranav Rai, and Iryna Gurevych. 2018. Cross-topic argument mining from heterogeneous sources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3664– 3674. find
  38. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016b. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41. find
  39. Lucas Sterckx, Cornelia Caragea, Thomas Demeester, and Chris Develder. 2016. Supervised keyphrase extraction as positive unlabeled learning. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1924–1929, Austin, Texas. Association for Computational Lingui… find
  40. Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. BERTweet: A pre-trained language model for English tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 9–14. find our card
  41. Krutarth Patel and Cornelia Caragea. 2019. Exploring word embeddings in crf-based keyphrase extraction from research papers. In Proceedings of the 10th International Conference on Knowledge Capture, K-CAP ’19, page 37–44, New York, NY, USA. Association for Computing Machinery. find
  42. Delip Rao and Dean Pomerleau. 2017. Fake news challenge. find
  43. Jishnu Ray Chowdhury, Cornelia Caragea, and Doina Caragea. 2019. Keyphrase extraction from disaster-related tweets. In The World Wide Web Conference, WWW ’19, page 1555–1566, New York, NY, USA. Association for Computing Machinery. find
  44. Jishnu Ray Chowdhury, Seo Yeon Park, Tuhin Kundu, and Cornelia Caragea. 2022. KPDROP: Improving absent keyphrase generation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4853–4870, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  45. Benjamin Schiller, Johannes Daxenberger, and Iryna Gurevych. 2021. Stance detection benchmark: How robust is your stance detection? KI - Künstliche Intelligenz. find
  46. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681. find
  47. Parinaz Sobhani, Diana Inkpen, and Xiaodan Zhu. 2017. A dataset for multi-target stance detection. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 551–557. find our card
  48. Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124. find
  49. Jannis Vamvas and Rico Sennrich. 2020. X-Stance: A multilingual multi-target dataset for stance detection. In Proceedings of the 5th Swiss Text Analytics Conference (SwissText) & 16th Conference on Natural Language Processing (KONVENS). find
  50. Wikipedia. Wikipedia:list of controversial issues. [Online; accessed 10-December-2012]. find
  51. Lee Xiong, Chuan Hu, Chenyan Xiong, Daniel Campos, and Arnold Overwijk. 2019. Open domain web keyphrase extraction beyond language modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proces… find
  52. Chang Xu, Cécile Paris, Surya Nepal, and Ross Sparks. 2018. Cross-target stance classification with self-attention networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 778– 783. find
  53. Jiacheng Ye, Tao Gui, Yichao Luo, Yige Xu, and Qi Zhang. 2021. One2Set: Generating diverse keyphrases as a set. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers… find
  54. Xingdi Yuan, Tong Wang, Rui Meng, Khushboo Thaker, Peter Brusilovsky, Daqing He, and Adam Trischler. 2020. One size does not fit all: Generating and evaluating variable number of keyphrases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7961–7975. find
  55. Bowen Zhang, Min Yang, Xutao Li, Yunming Ye, Xiaofei Xu, and Kuai Dai. 2020. Enhancing cross-target stance detection with transferable semantic-emotion knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3188–3197. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2023-new-direction-stance-detection-target/