Elamur Research⌕
☰
home / papers / 2019 / Argument Generation with Retrieval, Planning, and Realization

Argument Generation with Retrieval, Planning, and Realization

Paper · 2019 · Xinyu Hua, Zhe Hu and Lu Wang

Summary

The paper tackles counter-argument generation: given a statement on a contentious topic, the system writes a paragraph that argues the opposing side. Earlier retrieval-based approaches stitched together sentences from unrelated sources, so their output lacked coherence and stylistic control. The authors propose CANDELA, which has two parts. The first is a retrieval stage. A search engine over roughly 12 million Wikipedia and news articles returns passages, ranks them by relevance and stance, and condenses them into keyphrases. The second is a two-decoder generator. A text planning decoder chooses, for each sentence, which keyphrases to cover and what argumentative function it serves, either substantive content or a stylistic "filler" such as a concession. A content realization decoder then writes the text based on those plans. Training and evaluation use argument–counterargument pairs from Reddit /r/ChangeMyView. CANDELA scores significantly higher on BLEU, ROUGE and METEOR than the authors' earlier system and other baselines. Human judges rated its output more appropriate and richer in content, with fluency close to human-written text. The work shows that explicit planning combined with diverse external evidence leads to more informative generated arguments.

Key points

  • The retrieval index is built with Elasticsearch and holds 11.9M articles split into about 120M passages. Sources are a Wikipedia dump plus four news outlets (NYT, WaPo, Reuters, WSJ) that span different ideological leanings. News articles were gathered via Common Crawl and cleaned with jusText.
  • Passages are windows of three sentences, shifted by two sentences each step, with a minimum of 50 words. For each query, BM25 returns the top 20 passages per medium.
  • Input keyphrases come from topic signature words, expanded with WordNet synonyms, hyponyms, hypernyms and antonyms. Candidate noun and verb phrases are extracted with the help of the Stanford POS tagger.
  • Human evaluation covered 40 test examples rated by three annotators, with Krippendorff's α of 0.44, 0.58 and 0.49 across the three criteria. CANDELA beat the human or retrieval text in 39.2% of judgments. The corresponding figures were 34.2% for Seq2seqAug and 13.3% for the earlier H&W model.
  • The authors list several weaknesses: excessive use of negation, weaker organization than human arguments, and coherence that degrades toward the end of longer outputs. Code and data are released publicly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Northeastern UniversityBoston · United States28 works · 6,164 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPhilosophy · 211 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 145 works

Cite

Cite as Xinyu Hua, Zhe Hu and Lu Wang. 2019. Argument Generation with Retrieval, Planning, and Realization. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1255

@inproceedings{hua2019argument,
  title = {{Argument Generation with Retrieval, Planning, and Realization}},
  author = {Xinyu Hua, Zhe Hu and Lu Wang},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1255},
  url = {https://doi.org/10.18653/v1/p19-1255}
}

References

44 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations (ICLR).
  2. Roy Bar-Haim, Indrajit Bhattacharya, Francesco Din-uzzo, Amrita Saha, and Noam Slonim. 2017. Stance classification of context-dependent claims. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 251–261. Associ…
  3. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137–1155. premises in an online persuasive forum. In Proceedings of the 4th Workshop on Argument Mining, pages 11–21. Association for Compu…
  4. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168–177. ACM. find
  5. Xinyu Hua and Lu Wang. 2018. Neural argument generation augmented with externally retrieved evidence. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 219–230. Association for Computational Linguistics. find
  6. Benoit Lavoie and Owen Rambow. 1997. A fast and portable realizer for text generation systems. In Fifth Conference on Applied Natural Language Processing.
  7. Giuseppe Carenini and Johanna Moore. 2000. A strategy for generating evaluative arguments. In INLG’2000 Proceedings of the First International Conference on Natural Language Generation, pages 47–54, Mitzpe Ramon, Israel. Association for Computational Linguistics. find
  8. Dieu-Thu Le, Cam Tu Nguyen, and Kim Anh Nguyen. 2018. Dave the debater: a retrieval-based and generative argumentative dialogue agent. In Proceedings of the 5th Workshop on Argument Mining, pages 121–130. Association for Computational Linguistics. find
  9. T Edward Damer. 2012. Attacking faulty reasoning. Cengage Learning. find
  10. Ran Levy, Ben Bogin, Shai Gretz, Ranit Aharonov, and Noam Slonim. 2018. Towards an argumentative content search engine using weak supervision. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2066–2081. Association for Computational Linguistics. find
  11. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, Maryland, USA. Association for Computational Linguistics.
  12. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul):2121–2159. find
  13. Yarin Gal and Zoubin Ghahramani. 2016. A theoretically grounded application of dropout in recurrent neural networks. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 1019–1027. Curran Associates, Inc.
  14. Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. Bottom-up abstractive summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4098–4109, Brussels, Belgium. Association for Computational Linguistics.
  15. Clinton Gormley and Zachary Tong. 2015. Elasticsearch: The definitive guide: A distributed real-time search and analytics engine. " O’Reilly Media, Inc.". find
  16. Christopher Hidey, Elena Musi, Alyssa Hwang, Smaranda Muresan, and Kathy McKeown. 2017. Analyzing the semantic types of claims and Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of t…
  17. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out.
  18. Chin-Yew Lin and Eduard Hovy. 2000. The automated acquisition of topic signatures for text summarization. In COLING 2000 Volume 1: The 18th International Conference on Computational Linguistics.
  19. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60, Baltimore, Ma…
  20. George A. Miller. 1994. Wordnet: A lexical database for english. In HUMAN LANGUAGE TECHNOL-OGY: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
  21. Eric W Noreen. 1989. Computer-intensive methods for testing hypotheses. Wiley New York. find
  22. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Lin…
  23. Adam Paszke, Sam Gross, Soumith Chintala, and Gregory Chanan. 2017. Pytorch: Tensors and dynamic neural networks in python with strong gpu acceleration. PyTorch: Tensors and dynamic neural networks in Python with strong GPU acceleration. find
  24. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  25. Jan Pomikálek. 2011. Removing boilerplate and duplicate content from web corpora. Ph.D. thesis, Masaryk university, Faculty of informatics, Brno, Czech Republic. find
  26. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The penn discourse treebank 2.0. In LREC. Citeseer.
  27. Chris Reed, Derek Long, and Maria Fox. 1996. An architecture for argumentative dialogue planning. In International Conference on Formal and Applied Practical Reasoning, pages 555–566. Springer. find
  28. Paul Reisert, Naoya Inoue, Naoaki Okazaki, and Kentaro Inui. 2015. A computational approach for generating toulmin model argumentation. In Proceedings of the 2nd Workshop on Argumentation Mining, pages 45–55, Denver, CO. Association for Computational Linguistics. find
  29. Ehud Reiter and Robert Dale. 2000. Building natural language generation systems. Cambridge university press. find
  30. Ruty Rinott, Lena Dankin, Carlos Alzate Perez, Mitesh M. Khapra, Ehud Aharoni, and Noam Slonim. 2015. Show me your evidence - an automatic method for context dependent evidence detection. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 440–450, Lisbon…
  31. Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995. Okapi at trec-3. Nist Special Publication Sp, 109:109.
  32. Misa Sato, Kohsuke Yanai, Toshinori Miyoshi, Toshihiko Yanase, Makoto Iwayama, Qinghua Sun, and Yoshiki Niwa. 2015. End-to-end argument generation system in debating. In Proceedings of ACL-IJCNLP 2015 System Demonstrations, pages 109– 114, Beijing, China. Association for Computational Linguistics an… find
  33. Christian Stab, Johannes Daxenberger, Chris Stahlhut, Tristan Miller, Benjamin Schiller, Christopher Tauchmann, Steffen Eger, and Iryna Gurevych. 2018. Argumentext: Searching for arguments in heterogeneous sources. In Proceedings of the 2018 Conference of the North American Chapter of the Associatio… find
  34. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3104–3112. Curran Associates, Inc. find
  35. Stephen Edelston Toulmin. 1958. The use of argument. Cambridge University Press. find
  36. Henning Wachsmuth, Nona Naderi, Ivan Habernal, Yufang Hou, Graeme Hirst, Iryna Gurevych, and Benno Stein. 2017a. Argumentation quality assessment: Theory vs. practice. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 250–255.… find
  37. Henning Wachsmuth, Martin Potthast, Khalid Al Khatib, Yamen Ajjour, Jana Puschmann, Jiani Qu, Jonas Dorsch, Viorel Morari, Janek Bevendorff, and Benno Stein. 2017b. Building an argument search engine for the web. In Proceedings of the 4th Workshop on Argument Mining, pages 49–59. Association for Com… find
  38. Henning Wachsmuth, Manfred Stede, Roxanne El Baff, Khalid Al Khatib, Maria Skeppstedt, and Benno Stein. 2018a. Argumentation synthesis following rhetorical strategies. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3753–3765. Association for Computational Lin… find
  39. Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018b. Retrieval of the best counterargument without prior topic knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 241–251. Association for Computational Linguistics.
  40. Douglas Walton, Christopher Reed, and Fabrizio Macagno. 2008. Argumentation schemes. Cambridge University Press. find
  41. Lu Wang, Nick Beauchamp, Sarah Shugars, and Kechen Qin. 2017. Winning on the merits: The joint effects of content and style on debate outcomes. Transactions of the Association for Computational Linguistics, 5:219–232. find
  42. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2018. Learning neural templates for text generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3174–3187, Brussels, Belgium. Association for Computational Linguistics.
  43. A.3 As described in §5.2 in the main paper, we use a list of discourse markers together with topic signature words to label argumentative content sentences. The following list of discourse markers are manually selected from the Appendix B in Prasad et al. (2008). find
  44. Toshihiko Yanase, Toshinori Miyoshi, Kohsuke Yanai, Misa Sato, Makoto Iwayama, Yoshiki Niwa, Paul Reisert, and Kentaro Inui. 2015. Learning sentence ordering for opinion generation of debate. In Proceedings of the 2nd Workshop on Argumentation Mining, pages 94–103, Denver, CO. Association for Comput… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hua-2019-argument-generation-retrieval-planning-realization/