Elamur Research⌕
☰
home / papers / 2018 / Retrieval of the Best Counterargument without Prior Topic Knowledge

Retrieval of the Best Counterargument without Prior Topic Knowledge

Summary

The paper addresses how to retrieve the most fitting counterargument for an arbitrary argument when nothing is known in advance about its topic. The authors' hypothesis is that a strong counterargument talks about the same aspects as the original argument but takes the opposing side. They turn this into an unsupervised ranking model. The model scores each candidate in two ways at once. A topic-similarity component rewards overlap with the argument. A stance-dissimilarity component penalizes candidates that are too close to either the argument's conclusion or its premises. Both components use word-based and embedding-based similarities.

For evaluation, they built the ArguAna Counterargs corpus from idebate.org, where each point is paired with a dedicated counter. From this corpus they defined eight retrieval tasks that differ in which candidates compete. Their best configuration consistently outperforms standard similarity baselines. Even when every test argument is a candidate, it ranks the true counter first roughly one time in three. This suggests that useful counterarguments can often be found without topic knowledge, which matters for debating technologies and argument search engines such as args.

Key points

  • Corpus: ArguAna Counterargs contains 6753 argument–counterargument pairs from 1069 idebate.org debates, grouped into 15 themes, plus millions of derived false pairs. The corpus and Java code are released at arguana.com.
  • Gains over baselines: The proposed variants (we, we↓, we↑) beat plain word and embedding similarity by about 8–10 accuracy points in every task. They also beat the strongest baseline in each task by at least 3 points.
  • Same-debate setting: Among opposing-stance arguments from the same debate (7.6 candidates on average), the we variant ranks the true counter first in 60.3% of cases. The we↑ variant reaches 74.5% when only opposing counters are candidates.
  • Full test set: When the candidate pool includes the entire set of 2801 test arguments, the we↑ variant achieves 32.4% accuracy@1, a mean rank of 15, and an MRR of 0.5. The best baseline (w+) reaches 28.6% accuracy@1 with a mean rank of 33.
  • Variation by theme: Results differ notably across themes. On the full-portal task, we↑ ranges from 12.1% accuracy@1 for free speech debates, whose arguments tend to be unusually long, to 46.7% for sport, whose topics are very specific.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

94 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 85 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1023

@inproceedings{wachsmuth2018retrieval,
  title = {{Retrieval of the Best Counterargument without Prior Topic Knowledge}},
  author = {Wachsmuth, Henning and Syed, Shahbaz and Stein, Benno},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1023},
  url = {https://doi.org/10.18653/v1/p18-1023}
}

References

38 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yamen Ajjour, Wei-Fan Chen, Johannes Kiesel, Henning Wachsmuth, and Benno Stein. 2017. Unit segmentation of argumentative texts. In Proceedings of the 4th Workshop on Argument Mining, pages 118– 128. Association for Computational Linguistics. find
  2. Khalid Al-Khatib, Henning Wachsmuth, Matthias Hagen, Jonas Köhler, and Benno Stein. 2016. Cross-domain mining of argumentative text through distant supervision. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… find
  3. Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010. SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC’10). European Languages Resources Association… find our card
  4. Roy Bar-Haim, Indrajit Bhattacharya, Francesco Din-uzzo, Amrita Saha, and Noam Slonim. 2017. Stance classification of context-dependent claims. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 251–261. Associ… find
  5. Oana Cocarascu and Francesca Toni. 2017. Identifying attack and support argumentative relations using deep learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1374–1379. Association for Computational Linguistics. find
  6. T. Edward Damer. 2009. Attacking Faulty Reasoning: A Practical Guide to Fallacy-Free Arguments, 6th edition. Wadsworth, Cengage Learning, Belmont, CA. find
  7. Vanessa Wei Feng and Graeme Hirst. 2011. Classifying arguments by scheme. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 987–996. Association for Computational Linguistics. find
  8. Trudy Govier. 2010. A Practical Study of Argument, 7th edition. Wadsworth, Cengage Learning, Belmont, CA. find
  9. Ivan Habernal and Iryna Gurevych. 2016. What makes a convincing argument? Empirical analysis and detecting attributes of convincingness in web argumentation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1214–1223. Association for Computational Ling… find
  10. Beata Beigman Klebanov, Eyal Beigman, and Daniel Diermeier. 2010. Vocabulary choice as an indicator of perspective. In Proceedings of the ACL 2010 Conference Short Papers, pages 253–257. Association for Computational Linguistics. find
  11. Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. Before name-calling: Dynamics and triggers of ad hominem fallacies in web argumentation. In 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Ass… find
  12. Filip Boltužić and Jan Šnajder. 2014. Back up your stance: Recognizing arguments in online discussions. In Proceedings of the First Workshop on Argumentation Mining, pages 49–58. Association for Computational Linguistics. find
  13. Yufang Hou and Charles Jochim. 2017. Argument relation classification using a joint inference model. In Proceedings of the 4th Workshop on Argument Mining, pages 60–66. Association for Computational Linguistics. find
  14. Liora Braunstain, Oren Kurland, David Carmel, Idan Szpektor, and Anna Shtok. 2016. Supporting human answers for advice-seeking questions in CQA sites. In Proceedings of the 38th European Conference on IR Research, pages 129–141. find
  15. Elena Cabrio and Serena Villata. 2012. Combining textual entailment and argumentation theory for supporting online debates interactions. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 208–212. Association for Computational L… find
  16. Sung-Hyuk Cha. 2007. Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions. International Journal of Mathematical Models and Methods in Applied Sciences, 1(4):300–307. find
  17. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computation… find
  18. Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, and Kilian Q. Weinberger. 2015. From word embeddings to document distances. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37, pages 957–966. find
  19. John Lawrence and Chris Reed. 2017. Mining argumentative structure from natural language text using automatically generated premise-conclusion topic models. In Proceedings of the 4th Workshop on Argument Mining, pages 39–48. Association for Computational Linguistics. find
  20. Stefano Menini, Federico Nanni, Simone Paolo Ponzetto, and Sara Tonelli. 2017. Topic-based agreement and disagreement in us electoral manifestos. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2938–2944. Association for Computational Linguistics. find
  21. Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2017. Advances in pre-training distributed word representations. CoRR. find our card
  22. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems -Volume 2, pages 3111–3119. find
  23. Vlad Niculae, Joonsuk Park, and Claire Cardie. 2017. Argument mining with structured SVMs and RNNs. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 985–995. Association for Computational Linguistics. find
  24. Andreas Peldszus and Manfred Stede. 2015. Towards detecting counter-considerations in text. In Proceedings of the 2nd Workshop on Argumentation Mining, pages 104–109. Association for Computational Linguistics. find
  25. Peter Prettenhofer and Benno Stein. 2010. Cross-language text classification using structural correspondence learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1118–1127. Association for Computational Linguistics. find
  26. Ruty Rinott, Lena Dankin, Carlos Alzate Perez, M. Mitesh Khapra, Ehud Aharoni, and Noam Slonim. 2015. Show me your evidence — An automatic method for context dependent evidence detection. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 440–450. Associ… find our card
  27. Haggai Roitman, Shay Hummel, Ella Rabinovich, Benjamin Sznajder, Noam Slonim, and Ehud Aharoni. 2016. On the retrieval of wikipedia articles containing claims on controversial topics. In Proceedings of the 25th International Conference on World Wide Web, Companion Volume, pages 991–996. find
  28. Parinaz Sobhani, Diana Inkpen, and Stan Matwin. 2015. From argumentation mining to stance classification. In Proceedings of the 2nd Workshop on Argumentation Mining, pages 67–77. Association for Computational Linguistics. find
  29. Robert Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pages 4444–4451. find
  30. Christian Stab and Iryna Gurevych. 2016. Recognizing the absence of opposing arguments in persuasive essays. In Proceedings of the Third Workshop on Argument Mining (ArgMining2016), pages 113–118. Association for Computational Linguistics. find
  31. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International World Wide Web Conference, pages 613–624. find
  32. Stephen E. Toulmin. 1958. The Uses of Argument. Cambridge University Press. find
  33. Henning Wachsmuth, Nona Naderi, Yufang Hou, Yonatan Bilu, Vinodkumar Prabhakaran, Tim Alberdingk Thijm, Graeme Hirst, and Benno Stein. 2017a. Computational argumentation quality assessment in natural language. In Proceedings of the 15th Conference of the European Chapter of the Association for Compu… find our card
  34. Henning Wachsmuth, Martin Potthast, Khalid Al Khatib, Yamen Ajjour, Jana Puschmann, Jiani Qu, Jonas Dorsch, Viorel Morari, Janek Bevendorff, and Benno Stein. 2017b. Building an argument search engine for the web. In Proceedings of the 4th Workshop on Argument Mining, pages 49–59. Association for Com… find
  35. Henning Wachsmuth, Benno Stein, and Yamen Ajjour. 2017c. “PageRank” for Argument Relevance. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1117–1127. Association for Computational Linguistics. find
  36. Douglas Walton. 2006. Fundamentals of Critical Argumentation. Cambridge University Press. find
  37. Douglas Walton. 2009. Objections, rebuttals and refutations. pages 1–10. find
  38. Justine Zhang, Ravi Kumar, Sujith Ravi, and Cristian Danescu-Niculescu-Mizil. 2016. Conversational flow in Oxford-style debates. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 136–141. Associati… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wachsmuth-2018-retrieval-best-counterargument-without-prior/