Elamur Research⌕
☰
home / papers / 2020 / Unsupervised stance detection for arguments from consequences

Unsupervised stance detection for arguments from consequences

Summary

Most stance detection systems are trained separately for each debate topic, so every new topic needs fresh annotated data. Kobbe, Hulpuș and Stuckenschmidt instead target a common type of reasoning, the argument from consequences, in which a claim supports or opposes something by pointing to its good or bad effects. Their method needs no training data and is not tied to any topic. Using dependency-based rules, it pulls an "effect triple" out of each claim:

  • the target, and whether the claim argues for increasing or reducing it;
  • a predicate that acts on an object;
  • that object, and whether it is generally seen as good or bad.

Effect and sentiment lexicons supply the polarities. The system then compares the claim's target with the topic's target to decide whether the claim is for or against the topic. For evaluation, the authors crowdsourced stance and consequence labels on Amazon Mechanical Turk for 1894 Debatepedia arguments. The approach performs on par with a fine-tuned BERT model and sometimes better. It also varies less across folds and produces explanations that humans can follow. This suggests that methods built around specific argumentation schemes are a workable alternative to topic-specific classifiers.

Key points

  • Effect lexicon (ECF): The authors built an extended connotation frames lexicon. They started from the 947 verbs of Rashkin et al.'s connotation frames, dropped near-neutral entries to keep 845, and expanded these through WordNet synsets to 2508 effect words. ECF consistently beat +/-EffectWordNet (EWN), whose larger coverage came with lower accuracy.
  • Sentiment lexicon: Polarities come from a combination of the MPQA lexicon, Hu & Liu's opinion lexicon and the Toledo-Ronen et al. (2018) lexicon.
  • Robustness: On the same 10 folds, the system's F1-macro standard deviation stayed between .03 (debate) and .07 (conseq), whereas BERT varied much more.
  • Coverage: A potential consequence pattern (TPO/TP/PO) was found for about .6 of the conseq arguments. Depending on the dataset, 40–61% of instances still fell back on a heuristic. The main error sources were target detection, missing patterns and gaps in the lexicons.
  • Availability: The data and code are released as StArCon (github.com/dwslab/StArCon).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

13 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 12 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.4

@inproceedings{kobbe2020unsupervised,
  title = {{Unsupervised stance detection for arguments from consequences}},
  author = {Kobbe, Jonathan and Hulpuș, Ioana and Stuckenschmidt, Heiner},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.4},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.4}
}

References

36 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adam Faulkner. 2014. Automated classification of stance in student essays: An approach using stance target information and the wikipedia link-based measure. Proceedings of the 27th International Florida Artificial Intelligence Research Society Conference, FLAIRS 2014, pages 174–179. find
  2. Aseel Addawood, Jodi Schneider, and Masooda Bashir. 2017. Stance classification of twitter debates: The encryption debate as a use case. In Proceedings of the 8th International Conference on Social Media & Society, \#SMSociety17, New York, NY, USA. Association for Computing Machinery. find
  3. Christiane Fellbaum. 2010. About wordnet. find
  4. Khalid Al-Khatib, Yufang Hou, Henning Wachsmuth, Charles Jochim, Francesca Bonin, and Benno Stein. 2020. End-to-end argumentation knowledge graph construction. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020). find
  5. Princeton university: Shalmoli Ghosh, Prajwal Singhania, Siddharth Singh, Koustav Rudra, and Saptarshi Ghosh. 2019. Stance detection in web and social media: A comparative study. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 75–87, Cham. Springer International Publi… find
  6. Pranav Anand, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani, and Michael Minor. 2011. Cats rule and dogs drool!: Classifying stance in online debate. In Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2.011), pages 1– 9, Portl… find
  7. Subrata Ghosh, Konjengbam Anand, Sailaja Rajanala, A Bharath Reddy, and Manish Singh. 2018. Unsupervised Stance Classification in Online Debates. In Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, CoDS-COMAD ’18, pages 30–36, New York, NY, USA. ACM… find
  8. Kazi Saidul Hasan and Vincent Ng. 2013. Extra-linguistic constraints on stance recognition in ideological debates. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 816–821, Sofia, Bulgaria. Association for Computational Lingui… find
  9. Roy Bar-Haim, Indrajit Bhattacharya, Francesco Din-uzzo, Amrita Saha, and Noam Slonim. 2017a. Stance classification of context-dependent claims. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 251–261, Valen… find
  10. Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013. Learning whom to trust with MACE. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120–1130, Atlanta, Georgia. Associati… find our card
  11. Roy Bar-Haim, Lilach Edelstein, Charles Jochim, and Noam Slonim. 2017b. Improving claim stance classification with lexical knowledge expansion and context utilization. In Proceedings of the 4th Workshop on Argument Mining. Association for Computational Linguistics. find
  12. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, page 168–177, New York, NY, USA. Association for Computing Machinery. find
  13. Yoonjung Choi and Janyce Wiebe. 2014. +/-EffectWordNet: Sense-level lexicon acquisition for opinion inference. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1181–1191, Doha, Qatar. Association for Computational Linguistics. find
  14. Anand Konjengbam, Subrata Ghosh, Nagendra Kumar, and Manish Singh. 2018. Debate stance classification using word embeddings. In Big Data Analytics and Knowledge Discovery, pages 382–395, Cham. Springer International Publishing. find
  15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. find
  16. Kuntal Dey, Ritvik Shrivastava, and Saroj Kaushik. 2018. Topical stance detection for twitter: A two-phase lstm model using attention. In Advances in Information Retrieval, pages 529–536, Cham. Springer International Publishing. find
  17. Dilek Küçük and Fazli Can. 2020. Stance detection: A survey. ACM Comput. Surv., 53(1). find
  18. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP Natural Language Processing Toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60, Baltimore, Ma… find our card
  19. Jiachen Du, Ruifeng Xu, Yulan He, and Lin Gui. 2017. Stance classification with target-specific neural attention. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 3988–3994. find
  20. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31– 41, San Diego, California. Association for Computational Ling… find
  21. Dhanya Sridhar, Lise Getoor, and Marilyn Walker. 2014. Collective stance classification of posts in online debate forums. In Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media, pages 109–117, Baltimore, Maryland. Association for Computational Linguistics. find
  22. Akiko Murakami and Rudy Raymond. 2010. Support or oppose? classifying positions in online debates from reply activities and opinion expressions. In Coling 2010: Posters, pages 869–875, Beijing, China. find
  23. Coling 2010 Organizing Committee. find
  24. Qingying Sun, Zhongqing Wang, Qiaoming Zhu, and Guodong Zhou. 2018. Stance detection with hierarchical attention network. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2399–2409, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card
  25. Pavithra Rajendran, Danushka Bollegala, and Simon Parsons. 2016. Contextual stance classification of opinions: A step towards enthymeme reconstruction in online reviews. In Proceedings of the Third Workshop on Argument Mining (ArgMining2016), pages 31–39, Berlin, Germany. Association for Computation… find
  26. Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 327–335, Sydney, Australia. Association for Computational Lin… find
  27. Hannah Rashkin, Sameer Singh, and Yejin Choi. 2016. Connotation Frames: A Data-Driven Investigation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 311–321. Association for Computational Linguistics. Event-place: Berlin, Germ… find
  28. Orith Toledo-Ronen, Roy Bar-Haim, Alon Halfon, Charles Jochim, Amir Menczel, Ranit Aharonov, and Noam Slonim. 2018. Learning sentiment composition from sentiment lexicons. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2230–2241, Santa Fe, New Mexico, USA. As… find
  29. Paul Reisert, Naoya Inoue, Tatsuki Kuribayashi, and Kentaro Inui. 2018. Feasible Annotation Scheme for Capturing Policy Argument Reasoning using Argument Templates. In Proceedings of the 5th Workshop on Argument Mining, pages 79–89, Brussels, Belgium. Association for Computational Linguistics. find
  30. Douglas Walton, Christopher Reed, and Fabrizio Macagno. 2008. Argumentation Schemes. Cambridge University Press. find
  31. Rui Wang, Deyu Zhou, Mingmin Jiang, Si Jiasheng, and Yang Yang. 2019. A survey on opinion mining: from stance to product aspect. IEEE Access, PP:1– 1. find
  32. Benjamin Schiller, Johannes Daxenberger, and Iryna Gurevych. 2020. Stance detection benchmark: How robust is your stance detection? find
  33. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing Contextual Polarity in Phrase-level Sentiment Analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT ’05, pages 347–354, Stroudsburg, PA, USA. Association for… find
  34. Parinaz Sobhani, Saif Mohammad, and Svetlana Kiritchenko. 2016. Detecting stance in tweets and analyzing its interaction with sentiment. In Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics, pages 159–169, Berlin, Germany. Association for Computational Linguistics. find
  35. Swapna Somasundaran and Janyce Wiebe. 2009. Recognizing Stances in Online Debates. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1, ACL ’09, pages 226– 234, Stroud… find
  36. Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124, Los Angeles, CA. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/kobbe-2020-unsupervised-stance-detection-arguments-consequences/