Elamur Research⌕
☰
home / papers / 2023 / Bridging Argument Quality and Deliberative Quality Annotations with Adapters

Bridging Argument Quality and Deliberative Quality Annotations with Adapters

Summary

Argument quality is judged differently by two research traditions. Argument Mining focuses mostly on properties of single arguments, while Deliberative Theory in the social sciences looks at how contributions serve the wider discussion. Each tradition has produced its own quality dimensions and annotated corpora, and it is unclear how these dimensions relate to one another.

The authors treat each dimension as a separate task. They train one adapter per dimension on a frozen transformer and then combine the adapters with adapterfusion, which learns a weighted mix of the others for a given target. The fusion weights also act as an analysis tool, showing which dimensions draw on which.

Across 6 corpora and 20 dimensions, fusion beat single-task adapters on 8 dimensions in a low-resource setup. The adapters were then transferred to a new out-of-domain task: predicting moderator interventions in an online deliberative forum. There, a fusion of quality adapters outperformed both a baseline and full fine-tuning. Because each adapter stands for a named quality dimension, the model's moderation predictions can be explained in those terms, which could make AI-assisted moderation easier to interpret and trust.

Key points

  • The study covers 20 quality dimensions: 8 from Argument Quality and 12 from Deliberative Quality, drawn from 6 datasets. These include Europolis, THF/BK, Kialo, the Grammarly Argument Quality Corpus (GAQ) and IBM-Rank-30k.
  • Adapterfusion (Pfeiffer et al., 2021) improved prediction over single-task adapters on 8 of the 20 dimensions. Most dimensions could be expressed by blending the remaining quality dimensions with learned weights.
  • For moderation prediction on data from Park et al. (2012), the best model was a fusion of all quality adapters (fusion-all). It beat both the baseline and full fine-tuning.
  • In the moderation model, the Kialo-based impact adapter received the most activation. It was followed by empathy, respect, proposal and common good, with the more affective dimensions (storytelling, narration, emotion) weighted lower.
  • Code, single-task adapters and fusion models are released at github.com/Blubberli/ArgQualityAdapters.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-eacl.187

@inproceedings{falk2023bridging,
  title = {{Bridging Argument Quality and Deliberative Quality Annotations with Adapters}},
  author = {Falk, Neele and Lapesa, Gabriella},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-eacl.187},
  url = {https://doi.org/10.18653/v1/2023.findings-eacl.187}
}

References

29 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Milad Alshomary, Timon Gurcke, Shahbaz Syed, Philipp Heinisch, Maximilian Spliethöver, Philipp Cimiano, Martin Potthast, and Henning Wachsmuth. 2021. Key point analysis via contrastive learning and extractive argument summarization. In Proceedings of the 8th Workshop on Argument Mining, pages 184– 1… find
  2. Eustasio Del Barrio, Juan A Cuesta-Albertos, and Carlos Matrán. 2018. An optimal transportation approach for assessing almost stochastic order. In The Mathematics of the Uncertain, pages 33–44. Springer. find
  3. Rotem Dror, Segev Shlomov, and Roi Reichart. 2019. Deep dominance - how to properly compare deep neural models. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 2773–2785. Associat… find
  4. Esin Durmus, Faisal Ladhak, and Claire Cardie. 2019. The role of pragmatic and discourse context in determining argument impact. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-… find our card
  5. Katharina Esau. 2022. Kommunikationsformen und Deliberationsdynamik. Nomos Verlagsgesellschaft mbH & Co. KG. find
  6. Neele Falk, Iman Jundi, Eva Maria Vecchi, and Gabriella Lapesa. 2021. Predicting moderation of deliberative arguments: Is argument quality the key? In Proceedings of the 8th Workshop on Argument Mining, pages 133–141, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  7. Michael Fromm, Max Berrendorf, Johanna Reiml, Isabelle Mayerhofer, Siddharth Bhargava, Evgeniy Faerman, and Thomas Seidl. 2022. Towards a holistic view on argument quality prediction. CoRR, abs/2205.09803. — [arXiv:2205.09803](https://arxiv.org/abs/2205.09803) find
  8. Marlène Gerber, André Bächtiger, Susumu Shikano, Simon Reber, and Samuel Rohr. 2018. Deliberative abilities and influence in a transnational deliberative poll (europolis). British Journal of Political Science, 48(4):1093–1118. find
  9. Shai Gretz, Roni Friedman, Edo Cohen-Karlik, Assaf Toledo, Dan Lahav, Ranit Aharonov, and Noam Slonim. 2020. A large-scale dataset for argument quality ranking: Construction and analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7805–7813. find
  10. Timon Gurcke, Milad Alshomary, and Henning Wachsmuth. 2021. Assessing the sufficiency of arguments through conclusion generation. In Proceedings of the 8th Workshop on Argument Mining, pages 67–77, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  11. Ivan Habernal and Iryna Gurevych. 2016. What makes a convincing argument? empirical analysis and detecting attributes of convincingness in web argumentation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1214–1223, Austin, Texas. Association for Com… find
  12. Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, and Luo Si. 2021. On the effectiveness of adapter-based tuning for pretrained language model adaptation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the… find
  13. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In ICML. find
  14. Anne Lauscher, Olga Majewska, Leonardo F. R. Ribeiro, Iryna Gurevych, Nikolai Rozanov, and Goran Glavaš. 2020a. Common sense or world knowledge? investigating adapter-based knowledge injection into pretrained transformers. In Proceedings of Deep Learning Inside Out (DeeLIO): The First Workshop on Kn… find
  15. Anne Lauscher, Lily Ng, Courtney Napoles, and Joel Tetreault. 2020b. Rhetoric, logic, and dialectic: Advancing theory-based argument quality assessment in natural language processing. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4563–4574, Barcelona, Spain… find
  16. Jialu Li, Esin Durmus, and Claire Cardie. 2020. Exploring the role of argument structure in online debate persuasion. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8905–8912, Online. Association for Computational Linguistics. find
  17. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  18. Lily Ng, Anne Lauscher, Joel Tetreault, and Courtney Napoles. 2020. Creating a domain-diverse corpus for theory-based argument quality assessment. In Proceedings of the 7th Workshop on Argument Mining, pages 117–126, Online. Association for Computational Linguistics. find
  19. Joonsuk Park, Sally Klingel, Claire Cardie, Mary Newhart, Cynthia Farina, and Joan-Josep Vallbé. 2012. Facilitative moderation for online participation in erulemaking. In Proceedings of the 13th Annual International Conference on Digital Government Research, dg.o ’12, page 173–182, New York, NY, USA… find
  20. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. AdapterFusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–503… find our card
  21. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020. AdapterHub: A framework for adapting transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,… find
  22. Henning Wachsmuth, Benno Stein, and Yamen Ajjour. 2017b. “PageRank” for argument relevance. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1117–1127, Valencia, Spain. Association for Computational Linguisti… find
  23. Henning Wachsmuth and Till Werner. 2020. Intrinsic quality assessment of arguments. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6739–6745, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  24. Thiemo Wambsganss, Christina Niklaus, Matthias Söllner, Siegfried Handschuh, and Jan Marco Leimeister. 2020. A corpus for argumentative writing support in German. In Proceedings of the 28th International Conference on Computational Linguistics, pages 856– 869, Barcelona, Spain (Online). Internationa… find
  25. Christian Stab and Iryna Gurevych. 2017. Recognizing insufficiently supported arguments in argumentative essays. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 980–990, Valencia, Spain. Association for Comp… find
  26. Reid Swanson, Brian Ecker, and Marilyn Walker. 2015. Argument mining: Extracting arguments from online dialogue. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 217–226, Prague, Czech Republic. Association for Computational Linguistics. find
  27. Dennis Ulmer, Christian Hardmeier, and Jes Frellsen. 2022. deep-significance-easy and meaningful statistical significance testing in the age of neural networks. arXiv preprint arXiv:2204.06815. find
  28. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. find
  29. Henning Wachsmuth, Nona Naderi, Yufang Hou, Yonatan Bilu, Vinodkumar Prabhakaran, Tim Alberd-ingk Thijm, Graeme Hirst, and Benno Stein. 2017a. Computational argumentation quality assessment in natural language. In Proceedings of the 15th Conference of the European Chapter of the Association for Comp… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/falk-2023-bridging-argument-quality-deliberative-quality/