Elamur Research⌕
☰
home / papers / 2019 / The Role of Pragmatic and Discourse Context in Determining Argument Impact

The Role of Pragmatic and Discourse Context in Determining Argument Impact

Summary

This work asks whether the impact of an argumentative claim depends on the discussion that comes before it. Rhetoricians call this kairos: whether a claim is timely and fitting. NLP research had studied linguistic, audience and source features, but not this pragmatic and discourse context. Social media data mixes in noise and off-topic posts, so the authors built a cleaner resource from kialo.com. On that site, users jointly build argument trees in which every claim supports or opposes its parent, and they vote on how impactful each claim is in its place along a line of reasoning.

The authors compared several types of models:

  • feature-based SVM baselines,
  • BiLSTM and FastText models,
  • BERT classifiers that see only the target claim,
  • BERT classifiers that also see the claim's parent or preceding context, encoded either as flat text, with attention, or with a GRU.

Adding context clearly improved impact prediction. A flat encoding of the preceding claims worked best. This supports the social-science view that argument quality depends on context, which matters for argument retrieval and generation.

Key points

  • Dataset: 47,219 claims on 741 controversial topics from kialo.com, with five impact labels ranging from NO IMPACT to VERY HIGH IMPACT. A 3-class variant merges the labels into IMPACTFUL, MEDIUM IMPACT and NOT IMPACTFUL.
  • Filtering: the experiments keep only claims with at least 5 votes, which amounts to 213,277 votes out of 241,884 in total. The same claim can receive different impact labels in different contexts.
  • Baselines: the parent claim's quality is a strong simple feature, giving an SVM macro F1 of 46.61%. BERT using the claim alone significantly outperforms all baselines.
  • Best model: BERT with a flat representation of the 3 preceding claims on the argument path reaches the top macro F1 of 55.98%. Adding only the parent claim brought no significant gain.
  • Short contexts: models trained with longer context also help claims that have little context, such as a context length of 1.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

19 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 18 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1568 · arXiv 2004.03034

@inproceedings{durmus2019role,
  title = {{The Role of Pragmatic and Discourse Context in Determining Argument Impact}},
  author = {Durmus, Esin and Ladhak, Faisal and Cardie, Claire},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1568},
  eprint = {2004.03034},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d19-1568}
}

References

45 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michael Burgoon, Stephen B Jones, and Diane Stewart. 1975. Toward a message-centered theory of persuasion: Three empirical investigations of language intensity1. Human Communication Research, 1(3):240–256. find
  2. S. Chaiken. 1979. Communicator physical attractiveness and persuasion. find
  3. Shelly Chaiken. 1980. Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39:752–766. find
  4. Shelly Chaiken. 1987. The heuristic model of persuasion. In Social influence: the ontario symposium, volume 5, pages 3–39. Hillsdale, NJ: Lawrence Erlbaum. find
  5. Kyunghyun Cho, Bart van Merriënboer, Çalar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Lan… find
  6. Robert B. Cialdini. 2007. Influence: The psychology of persuasion. find
  7. Meri Coleman and T. L. Liau. 1975. A computer readability formula designed for machine scoring. Journal of Applied Psychology, 60(2):283 – 284. find
  8. Joshua Correll, Steven J. Spencer, and Mark P. Zanna. 2004. An affirmed self and an open mind: Self-affirmation and sensitivity to argument strength. find
  9. MF Davies. 1998. Dogmatism and belief formation: Output interference in the processing of supporting and contradictory cognitions. Journal of Personality and Social Psychology, 75:456–466. find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  11. James Price Dillard and Michael Pfau. 2002. The persuasion handbook: Developments in theory and practice. Sage Publications. find
  12. Amanda M Durik, M Anne Britt, Rebecca Reynolds, and Jennifer Storey. 2008. The effects of hedges in persuasive arguments: A nuanced analysis of language. Journal of Language and Social Psychology, 27(3):217–234. find
  13. Esin Durmus and Claire Cardie. 2018. Exploring the role of prior beliefs for argument persuasion. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1035–1045, New Orleans, L… find
  14. Esin Durmus and Claire Cardie. 2019. Modeling the factors of user success in online debate. In The World Wide Web Conference, WWW ’19, pages 2701–2707, New York, NY, USA. ACM. find
  15. Esin Durmus, Faisal Ladhak, and Claire Cardie. 2019. Determining relative argument specificity and stance for complex argumentative structures. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4630–4641, Florence, Italy. Association for Computational… find
  16. Rudolph Flesch. 1948. A new readability yardstick. Journal of Applied Psychology, 32(3):221 – 233. find
  17. Charles G. Lord, Lee Ross, and Mark Lepper. 1979. Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37:2098–2109. find
  18. Ivan Habernal and Iryna Gurevych. 2016. What makes a convincing argument? empirical analysis and detecting attributes of convincingness in web argumentation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1214–1223, Austin, Texas. Association for Com… find
  19. Ivan Habernal and Iryna Gurevych. 2017. Argumentation mining in user-generated web discourse. American Journal of Computational Linguistics, 43(1):125–179. find our card
  20. Curtis P. Haugtvedt and Duane T. Wegener. 1994. Message Order Effects in Persuasion: An Attitude Strength Perspective. Journal of Consumer Research, 21(1):205–218. find
  21. Christopher Hidey and Kathleen McKeown. 2018. Persuasive influence detection: The role of argument sequencing. find
  22. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 9(8):1735– 1780. find
  23. Craig R. Hullett. 2005. The impact of mood on persuasion: A meta-analysis. Communication Research, 32(4):423–442. find
  24. C. Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. find
  25. Ken Hyland. 1998. Hedging in Scientific Research Articles. John Benjamins. find our card
  26. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431, Valencia, Spain. Associatio… find
  27. Nick Joyce and Jake Harwood. 2014. Context and identification in persuasive mass communication. Journal of Media Psychology: Theories, Methods, and Applications, 26:50. find
  28. Herbert C Kelman. 1961. Processes of opinion change. Public opinion quarterly, 25(1):57–78. find
  29. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. find
  30. Stephanie Lukin, Pranav Anand, Marilyn Walker, and Steve Whittaker. 2017. Argument strength is in the eye of the beholder: Audience effects in persuasion. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 742–… find
  31. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal. Association for Computational Linguistics. find
  32. Franziska Marquart and Brigitte Naderer. 2016. Communication and Persuasion: Central and Peripheral Routes to Attitude Change, pages 231–242. Springer Fachmedien Wiesbaden, Wiesbaden. find
  33. Norman Miller, Geoffrey Maruyama, Rex J. Beaber, and Keith Valone. 1976. Speed of speech and persuasion. Journal of Personality and Social Psychology, 34(4):615–624. find
  34. Raquel Mochales and Marie-Francine Moens. 2011. Argumentation mining. Artificial Intelligence and Law, 19(1):1–22. find
  35. Joonsuk Park and Claire Cardie. 2014. Identifying appropriate support for propositions in online user comments. In Proceedings of the First Workshop on Argumentation Mining, pages 29–38, Baltimore, Maryland. Association for Computational Linguistics. find
  36. Andreas Peldszus and Manfred Stede. 2015. Joint prediction in MST-style discourse parsing for argumentation mining. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 938–948, Lisbon, Portugal. Association for Computational Linguistics. find our card
  37. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. find
  38. Geetanjali Rakshit, Kevin K. Bowden, Lena Reed, Amita Misra, and Marilyn A. Walker. 2017. Debbie, the debate bot of the future. CoRR, abs/1709.03167. — [arXiv:1709.03167](https://arxiv.org/abs/1709.03167) find
  39. Swapna Somasundaran, Josef Ruppenhofer, and Janyce Wiebe. 2007. Detecting arguing and sentiment in meetings. In Proceedings of the SIGdial Workshop on Discourse and Dialogue, volume 6. find
  40. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web, pages 613–624. International World Wide Web C… find
  41. Orit Tykocinskl, E Tory Higgins, and Shelly Chaiken. 1994. Message framing, self-discrepancies, and yielding to persuasive messages: The motivational significance of psychological situations. Personality and Social Psychology Bulletin, 20(1):107–115. find
  42. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on human language technology and empirical methods in natural language processing, pages 347–354. Association for Computational Linguistics. find
  43. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 14… find our card
  44. Justine Zhang, Ravi Kumar, Sujith Ravi, and Cristian Danescu-Niculescu-Mizil. 2016. Conversational flow in Oxford-style debates. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 136–141, San Diego… find
  45. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 20… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/durmus-2019-role-pragmatic-discourse-context-determining/