Elamur Research⌕
☰
home / papers / 2018 / Cross-topic Argument Mining from Heterogeneous Sources

Cross-topic Argument Mining from Heterogeneous Sources

Summary

Argument search aims to retrieve, for a controversial topic, text passages that give evidence or reasoning for or against it. Most existing argument mining systems target a single genre and transfer poorly to mixed Web content. The authors define a flat, sentence-level task: each sentence is labelled as a supporting argument, an opposing argument, or a non-argument relative to a topic. A sentence that only states a stance without justification counts as a non-argument. The scheme is simple enough for crowd workers on Amazon Mechanical Turk to apply. With it, the authors built a corpus of over 25,000 sentences from heterogeneous Web documents on eight topics.

To generalise to unseen topics, they feed a topic vector into the input and candidate gates of a contextual BiLSTM (biclstm). This beats a vanilla BiLSTM and attention-based variants by more than 3 F1 points in cross-topic tests. Adding topic-relevance data through multi-task learning improves results further. The work matters because it supplies the first resource for cross-topic, cross-genre argument mining and supports a public argument search engine.

Key points

  • Data collection: For each of 8 controversial topics (e.g., abortion, gun control, nuclear energy), the authors took the top 50 Google results archived by the Wayback Machine. They cleaned these with Apache Tika and Stanford CoreNLP, which left 27,520 raw sentences.
  • Annotation reliability: Two experts reached Cohen's κ of 0.721. Crowd labels were aggregated with MACE and compared against the expert gold standard.
  • Best model: mtl-biclstm-dip uses multi-task learning with the DIP2016 corpus. It beats the strongest baseline by 5.9 F1 points with two labels and by 4.6 with three labels. Transfer from SemEval stance data mostly hurt performance.
  • Adapting to a new topic: With only about 20% of target-topic training data, topic-aware models gain a lot of argument recall without losing precision. The plain BiLSTM's recall stays near 30% until more than 60% of target-topic data is added.
  • Error analysis: Typical errors include off-topic sentences predicted as arguments and missed arguments that rely on background knowledge (e.g., the Second Amendment for "gun control"). The model also misreads words like "against" and "oppose".

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

94 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 83 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1402

@inproceedings{stab2018cross,
  title = {{Cross-topic Argument Mining from Heterogeneous Sources}},
  author = {Stab, Christian and Miller, Tristan and Schiller, Benjamin and Rai, Pranav and Gurevych, Iryna},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1402},
  url = {https://doi.org/10.18653/v1/d18-1402}
}

References

42 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yamen Ajjour, Wei-Fan Chen, Johannes Kiesel, Henning Wachsmuth, and Benno Stein. 2017. Unit segmentation of argumentative texts. In Proceedings of the 4th Workshop on Argument Mining, pages 118– 128. Association for Computational Linguistics. find
  2. Khalid Al-Khatib, Henning Wachsmuth, Matthias Hagen, Jonas Köhler, and Benno Stein. 2016. Cross-domain mining of argumentative text through distant supervision. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… find
  3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473. find our card
  4. Jamal Bentahar, Bernard Moulin, and Micheline Bélanger. 2010. A taxonomy of argumentation models used for knowledge representation. Artificial Intelligence Review, 33(3):211–259. find
  5. Jean Carletta. 1996. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2):249–254. find our card
  6. Xinchi Chen, Zhan Shi, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial multi-criteria learning for chinese word segmentation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 1, pages 1193–1203. Association for Computational Linguistics. find
  7. Johannes Daxenberger, Steffen Eger, Ivan Habernal, Christian Stab, and Iryna Gurevych. 2017. What is the essence of a claim? Cross-domain claim identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2045–2056. Association for Computational Li… find
  8. Steffen Eger, Johannes Daxenberger, and Iryna Gurevych. 2017. Neural end-to-end learning for computational argumentation mining. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11–22. Association for Computational Linguistics. find our card
  9. Felix A. Gers and Jürgen Schmidhuber. 2000. Recurrent nets that time and count. In Proceedings of the IEEE–INNS–ENNS International Joint Conference on Neural Networks, volume 3, pages 189–194. IEEE. find
  10. Shalini Ghosh, Oriol Vinyals, Brian Strope, Scott Roy, Tom Dean, and Larry P. Heck. 2016. Contextual LSTM (CLSTM) models for large scale NLP tasks. CoRR, abs/1602.06291. — [arXiv:1602.06291](https://arxiv.org/abs/1602.06291) find
  11. Theodosis Goudas, Christos Louizos, Georgios Petasis, and Vangelis Karkaletsis. 2014. Argument extraction from news, blogs, and social media. In Artificial Intelligence: Methods and Applications, volume 8445 of Lecture Notes in Computer Science, pages 287–299. Springer International Publishing. find
  12. Ivan Habernal, Judith Eckle-Kohler, and Iryna Gurevych. 2014. Argumentation mining on the Web from information seeking perspective. In Proceedings of the Workshop on Frontiers and Connections between Argumentation Theory and Natural Language Processing, volume 1341 of CEUR Workshop Proceedings, page… find
  13. Ivan Habernal, Maria Sukhareva, Fiana Raiber, Anna Shtok, Oren Kurland, Hadar Ronen, Judit Bar-Ilan, and Iryna Gurevych. 2016. New Collection Announcement: Focused Retrieval Over the Web. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retriev… find
  14. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems, pages 1693–1701. find our card
  15. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. find
  16. Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013. Learning whom to trust with MACE. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120–1130. Association for Computation… find our card
  17. Xinyu Hua and Lu Wang. 2017. Understanding and detecting diverse supporting arguments of diverse types. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 2, pages 203–208. Association for Computational Linguistics. find
  18. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations. find
  19. Keiichi Kobayashi. 2009. Comprehension of relations among controversial texts: Effects of external strategy use. Instructional Science, 37(4):311–324. find
  20. Ran Levy, Yonatan Bilu, Daniel Hershcovich, Ehud Aharoni, and Noam Slonim. 2014. Context dependent claim detection. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1489– 1500. Dublin City University and Association for Computatio… find our card
  21. Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial multi-task learning for text classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 1, pages 1–10. find our card
  22. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60. Associ… find our card
  23. Chris A. Mattmann and Jukka Zitting. 2011. Tika in Action. Manning Publications, Greenwich, CT, USA. find
  24. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, pages 3111–3119. Curran Associates, Inc. find
  25. Raquel Mochales-Palau and Marie-Francine Moens. 2009. Argumentation mining: The detection, classification and structure of arguments in text. In Proceedings of the 12th International Conference on Artificial Intelligence and Law, pages 98–107. ACM. find
  26. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 Task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation, pages 31–41. Association for Computational Linguistics. find our card
  27. Huy Nguyen and Diane Litman. 2016. Context-aware argumentative relation mining. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, volume 1, pages 1127–1137. Association for Computational Linguistics. find
  28. Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345–1359. find
  29. Nils Reimers and Iryna Gurevych. 2017. Reporting score distributions makes a difference: Performance study of LSTM-networks for sequence tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 338–348. Association for Computational Linguistics. find our card
  30. Ruty Rinott, Lena Dankin, Carlos Alzate Perez, Mitesh M. Khapra, Ehud Aharoni, and Noam Slonim. 2015. Show me your evidence – An automatic method for context dependent evidence detection. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 440–450. Associ… find our card
  31. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. 2015. Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664. find our card
  32. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958. find
  33. Henning Wachsmuth, Martin Potthast, Khalid Al Khatib, Yamen Ajjour, Jana Puschmann, Jiani Qu, Jonas Dorsch, Viorel Morari, Janek Bevendorff, and Benno Stein. 2017. Building an argument search engine for the web. In Proceedings of the 4th Workshop on Argument Mining, pages 49–59. Association for Comp… find
  34. Shuohang Wang and Jing Jiang. 2016. Learning natural language inference with LSTM. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1442–1451. Association for Computational Linguistics. find our card
  35. Ursula Wingate. 2012. ‘Argument!’ Helping students understand what essay writing is about. Journal of English for Academic Purposes, 11(2):145–154. find
  36. Adam Wyner, Raquel Mochales-Palau, Marie-Francine Moens, and David Milward. 2010. Approaches to text mining arguments from legal cases. In Enrico Francesconi, Simonetta Montemagni, Wim Peters, and Daniela Tiscornia, editors, Semantic Processing of Legal Texts: Where the Language of Law Meets the Law… find
  37. Christian Stab. 2017. Argumentative Writing Support by Means of Natural Language Processing. Dr.-Ing. thesis, Technische Universität Darmstadt. find
  38. Christian Stab, Johannes Daxenberger, Chris Stahlhut, Tristan Miller, Benjamin Schiller, Christopher Tauchmann, Steffen Eger, and Iryna Gurevych. 2018. ArgumenText: Searching for arguments in heterogeneous sources. In Proceedings of the 16th Annual Conference of the North American Chapter of the Ass… find
  39. Christian Stab and Iryna Gurevych. 2014. Identifying argumentative discourse structures in persuasive essays. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 46–56. Association for Computational Linguistics. find our card
  40. Christian Stab and Iryna Gurevych. 2017. Parsing argumentation structures in persuasive essays. Computational Linguistics, 43(3):619–659. find our card
  41. Chris Stahlhut. 2018. Searching arguments in German with ArgumenText. In Proceedings of the First Biennial Conference on Design of Experimental Search & Information Retrieval Systems, volume 2167 of CEUR Workshop Proceedings, page 104. find
  42. Ola Svenson. 1979. Process descriptions of decision making. Organizational Behavior and Human Performance, 23(1):86–112. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/stab-2018-cross-topic-argument-mining-heterogeneous/