Elamur Research⌕
☰
home / papers / 2026 / SKILL-IR-Discourse: A Large, Annotated Corpus of Argumentation and Domain Discourse on International Relations

SKILL-IR-Discourse: A Large, Annotated Corpus of Argumentation and Domain Discourse on International Relations

Summary

Most computational research on political language looks at debates, speeches and media. Scholarly writing in political science, where theories are actually built and argued, has received little attention. An earlier annotation scheme for argumentation and domain content in International Relations (IR) had only been tested on three articles, too few for general conclusions. This paper scales that work up into SKILL-IR-Discourse, a corpus of 190 open-access IR articles from major journals, totalling more than 1.5M tokens.

The authors used a simplified version of the Wolska et al. (2025) scheme. Its discourse layer covers argumentative and rhetorical moves, and its domain layer separates theory-based from data-based statements. Experts first annotated a small gold subset. That subset was then used to train student annotators, who labelled the rest under quality monitoring with purpose-built annotation software.

The analysis shows that IR argumentation is mostly shallow: claims often lack explicit evidence and rebuttals are uncommon. The balance of theoretical and evaluative content shifts with each article's method and the journal's orientation. The resource supports both argument mining and the study of academic writing practice.

Key points

  • Corpus size and split: 190 articles (>1,500K tokens) in total.
    • Gold-standard: 5 articles (~62K tokens), annotated by senior domain experts.
    • Silver-standard: 185 articles, annotated by trained political science students (11 annotators over the project).
  • Discourse dimension: Argumentation (Claim, Support, Attack) and Rhetorical Moves (Elaboration only).
  • Domain dimension: Theory (Definition, Theoretical Statement), Data (Speculative, Evaluative, Other Data), IR Other, and Other Domain.
  • Findings: Simple premise-conclusion structures dominate, and under half of the claims are explicitly supported. Attacks are rare, and the claim-to-support ratio differs considerably between articles.
  • Sources and availability: Articles come from open-access publications by Sage, Cambridge UP, Taylor & Francis and Oxford UP, balanced on five selection criteria. The corpus is freely available on Zenodo (record 17380680).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Treebanks

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4i2ptc5x4ese

@inproceedings{wolska2026skill,
  title = {{SKILL-IR-Discourse: A Large, Annotated Corpus of Argumentation and Domain Discourse on International Relations}},
  author = {Wolska, Magdalena and Wiegmann, Matti and Gholiagha, Sassan and Sienknecht, Mitja and Kiesel, Dora and Garcia, Irene Lopez and Riehmann, Patrick and Fröhlich, Bernd and Girgensohn, Katrin and Neyer, Jürgen and Stein, Benno},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4i2ptc5x4ese},
  url = {https://doi.org/10.63317/4i2ptc5x4ese}
}

References

33 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gavin Abercrombie and Riza Batista-Navarro. 2018. A sentiment-labelled corpus of hansard parliamentary debate speeches. In Proceedings of ParlaCLARIN. Common Language Resources and Technology Infrastructure (CLARIN). find
  2. Gavin Abercrombie and Riza Theresa Batista-Navarro. 2020. Parlvote: A corpus for sentiment analysis of political debates. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 5073–5078. find
  3. Marwa Adel Abuelwafa. 2021. Legitimation and manipulation in political speeches: a corpus-based study. Procedia Computer Science, 189:11–18. find
  4. Paul Bayley. 2004. Cross-cultural perspectives on parliamentary discourse. Cross-Cultural Perspectives on Parliamentary Discourse, pages 1–390. find
  5. Kaspar Beelen, Timothy Alberdingk Thijm, Christopher Cochrane, Kees Halvemaan, Graeme Hirst, Michael Kimmins, Sander Lijbrink, Maarten Marx, Nona Naderi, Ludovic Rheault, et al. 2017. Digitization of the canadian parliamentary debates. Canadian Journal of Political Science/Revue canadienne de scienc… find
  6. Dallas Card, Serina Chang, Chris Becker, Julia Mendelsohn, Rob Voigt, Leah Boustan, Ran Abramitzky, and Dan Jurafsky. 2022. Computational analysis of 140 years of us political speeches reveals more positive but increasingly polarized framing of immigration. Proceedings of the National Academy of Sci… find
  7. R. Charli Carpenter. 2005. “women, children and other vulnerable groups”: Gender, strategic frames and the protection of civilians as a transnational issue. International Studies Quarterly, 49(2). find
  8. Paul Chilton. 2004. Analysing political discourse: Theory and practice. Routledge. find
  9. Sara R Davis, Cody J Worsnop, and Emily M Hand. 2022. Gender bias recognition in political news articles. Machine Learning with Applications, 8:100304. find
  10. Martha Finnemore and Kathryn Sikkink. 1998. International norm dynamics and political change. International organization, 52(4). find
  11. Irena Fischer-Hwang, Dylan Grosz, Xinlan Emily Hu, Anjini Karthik, and Vivian Yang. 2020. Disarming loaded words: Addressing gender bias in political reporting. In Computation + Journalism Conference. find
  12. Pierpaolo Goffredo, Shohreh Haddadan, Vorakit Vorakitphan, Elena Cabrio, and Serena Villata. 2022. Fallacious argument classification in political debates. In Proceedings of the 31st International Joint Conference on Artificial Intelligence, pages 4143–4149. find
  13. Shohreh Haddadan, Elena Cabrio, and Serena Villata. 2019. Yes, we can! mining arguments in 50 years of us presidential campaign debates. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4684–4690. find
  14. Najeh Hajlaoui, David Kolovratnik, Jaakko Väyrynen, Ralf Steinberger, Daniel Varga, et al. 2014. DCEP – Digital Corpus of the European Parliament. In Proceedings of the 9th Language Resources and Evaluation Conference, pages 3164–3171. find
  15. Mirko Heinzel and Andrea Liese. 2021. Expert authority and support for covid-19 measures in germany and the uk: a survey experiment. West European Politics, 44(5-6). find
  16. Dániel Z Kádár and Sen Zhang. 2019. (im) politeness and alignment: A case study of public political monologues. Acta Linguistica Academica, 66(2):229–249. find
  17. Dominique Labbé and Jacques Savoy. 2021. Stylistic analysis of the french presidential speeches: Is macron really different? Digital Scholarship in the Humanities, 36(1):153–163. find
  18. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. Biometrics. find
  19. Anne Lauscher, Goran Glavaš, and Simone Paolo Ponzetto. 2018. An argument-annotated corpus of scientific publications. In Proceedings of the 5th Workshop on Argument Mining, Brussels, Belgium. ACL. find
  20. Eleonora Mancini, Federico Ruggeri, Andrea Galassi, and Paolo Torroni. 2022. Multimodal argument mining: A case study in political debates. In Proceedings of the 9th Workshop on Argument Mining, pages 158–170. find
  21. Meisam Moghadam and Niloofar Jafarpour. 2022. Pragmatic annotation of manipulation in political discourse: The case of trump-clinton presidential debate. Linguistic Forum – A Journal of Linguistics, 4(4):32–39. find
  22. Sebastian Padó, André Blessing, Nico Blokker, Erenay Dayanık, Sebastian Haunss, and Jonas Kuhn. 2019. Who sides with whom? towards computational construction of discourse networks for political debates. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages… find
  23. Robert D Putnam. 1988. Diplomacy and domestic politics: the logic of two-level games. In International organization, volume 42. CUP. find
  24. François Randour, Julien Perrez, and Min Reuchamps. 2020. Twenty years of research on political discourse: A systematic review and directions for future research. Discourse & Society, 31(4). find
  25. Maria Sidiropoulou. 2013. Representation through translation: Shared maps of pragmatic meaning and the constructionist paradigm. Journal of pragmatics, 53. find
  26. Christian Stab and Iryna Gurevych. 2014. Annotating argument components and relations in persuasive essays. In Proceedings of the 25th International Conference on Computational Linguistics: Technical papers, pages 1501–1510. find our card
  27. Ralf Steinberger, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaz Erjavec, Dan Tufis, and Dániel Varga. 2006. The JRC-Acquis: A multilingual aligned parallel corpus with 20+ languages. In Proceedings of the 5th Language Resources and Evaluation Conference. find our card
  28. Stephen E Toulmin. 2003. The uses of argument. Cambridge University Press. find
  29. Briana M Trifiro, Sejin Paik, Zhixin Fang, and Li Zhang. 2021. Politics and politeness: Analysis of incivility on twitter during the 2020 democratic presidential primary. Social Media + Society, 7(3):20563051211036939. find
  30. David Vilares and Yulan He. 2017. Detecting perspectives in political debates. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1573– 1582. find
  31. Kenneth N Waltz. 1993. The emerging structure of international politics. International security, 18(2). find
  32. Magdalena Wolska, Bernd Fröhlich, Katrin Girgensohn, Sassan Gholiagha, Dora Kiesel, Jürgen Neyer, Patrick Riehmann, Mitja Sienknecht, and Benno Stein. 2024. Argumentation in Waltz’s ’Emerging Structure of International Politics’. In 10th Conference of the International Society for the Study of Argum… find
  33. Magdalena Wolska, Sassan Gholiagha, Mitja Sienknecht, Dora Kiesel, Irene Lopez Garcia, Patrick Riehmann, Matti Wiegmann, Bernd Fröhlich, Katrin Girgensohn, Jürgen Neyer, and Benno Stein. 2025. Argumentation and Domain Discourse in Scholarly Articles on the Theory of International Relations. In 31st… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wolska-2026-skill-ir-discourse-large-annotated/