Elamur Research⌕
☰
home / papers / 2026 / Overview of the UZH Shared Task 2026 on Reconstructing the Reasoning in United Nations Resolutions

Overview of the UZH Shared Task 2026 on Reconstructing the Reasoning in United Nations Resolutions

Paper · 2026 · Anastassia Shaitarova, Yingqiang Gao, Fatma-Zohra Rezkellah et al.

Summary

Resolutions adopted by UN bodies form a highly formal, formulaic genre that argument mining has rarely studied, even though their paragraphs encode negotiated reasoning. This overview describes the shared task organized for the Workshop on Argument Mining and Reasoning in its 13th edition. It is built on resolutions and recommendations from UNESCO's IBE International Conferences on Education (1934–2008), supplied in French with English machine translations.

The task has two parts:

  1. Systems label each paragraph as preambular, operative or structural and attach tags from a 15-dimension education policy ontology.
  2. Systems detect linked paragraph pairs and classify each link as supporting, contradictive, complemental or modifying.

Only open-weight models with at most 8B parameters were permitted, and teams had to submit their chain-of-thought output. Eight teams took part. Scoring combined F1 against an LLM-built silver standard, checked by human experts, with an LLM-as-a-judge assessment.

Paragraph typing proved essentially solved. Rare tags and relation prediction remain hard: every system linked too many pairs, and the best relation labeling reached roughly 65% of human agreement. Because F1 and judge rankings diverged, accuracy and reasoning quality need separate measurement.

Key points

  • Data: The unlabeled training data comprises 2,695 resolutions from the UN-RES corpus, translated with Helsinki-NLP opus-mt-fr-en and distributed under a restricted UN license. The test set covers 45 documents (92 resolutions and recommendations), with English versions produced by gpt-4.1-mini.
  • Paragraph-type labels: Silver labels came from gpt-5.4-mini. They reached 97.0% accuracy on 363 expert-annotated paragraphs and 97.4% on a second set of 352, for 715 human-validated paragraphs in total.
  • Relation gold standard: It contains 300 expert-annotated pairs. Blind re-annotation gave intra-annotator κ = 0.540, treated as the human ceiling. The best system reached κ = 0.354, and the complemental/modifying distinction was the main source of confusion.
  • Ranking scheme: The F1 rank weights ST1 at 0.4 and ST2 at 0.6. The final rank weights F1 at 0.6 and the LLM-as-a-judge rank at 0.4.
  • Design lessons:
    • Deterministic lexical rules gave top type F1 (Argchestrators, ResolveNow).
    • Pre-filtering tags with embeddings hurt performance.
    • Judge scores do not transfer across model families: POINTERS scored 77/100 when self-evaluated with ClaudeSonnet4.6, but 26.16 from the official Gemma judge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ZurichZurich · Switzerland78 works · 12,260 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPhilosophy · 284 worksPolitical Science and International Relations · 182 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Anastassia Shaitarova, Yingqiang Gao, Fatma-Zohra Rezkellah, Reto Gubelmann and Patrick Montjouridès. 2026. Overview of the UZH Shared Task 2026 on Reconstructing the Reasoning in United Nations Resolutions. Proceedings of the 13th Workshop on Argument Mining and Reasoning.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.argmining-1.10

@inproceedings{shaitarova2026overview,
  title = {{Overview of the UZH Shared Task 2026 on Reconstructing the Reasoning in United Nations Resolutions}},
  author = {Anastassia Shaitarova, Yingqiang Gao, Fatma-Zohra Rezkellah, Reto Gubelmann and Patrick Montjouridès},
  booktitle = {Proceedings of the 13th Workshop on Argument Mining and Reasoning},
  year = {2026},
  doi = {10.18653/v1/2026.argmining-1.10},
  url = {https://doi.org/10.18653/v1/2026.argmining-1.10}
}

References

27 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Steven Bernstein. 2011. Legitimacy in Intergovernmental and Non-state Global Governance. Review of International Political Economy : RIPE, 18(1):17–51. Place: ABINGDON Publisher: Taylor & Francis Group tex.copyright: Copyright Taylor & Francis Group, LLC 2011. find
  2. Siddharth Bhargava. 2026. HybridArguer at UZH Shared Task 2026: Argument Structure Modeling in Bilingual UN Resolutions with Retrieval-Augmented and Iterative LLM Reasoning. In Proceedings of the 13th Workshop on Argument Mining and Reasoning, * https://diced.linguistik.uzh.ch/ ↗
  3. Budapest, Hungary. Association for Computational Linguistics. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language Models are Few-shot Learners. Advances in neural inform…
  4. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. Journal of Machine Learning Research, 23(120):1–39. find
  5. Yingqiang Gao, Fabian Winiger, Patrick Montjourides, Anastassia Shaitarova, Nianlong Gu, Simon Peng-Keller, and Gerold Schneider. 2025. SpiritRAG: A Q&A System for Religion and Spirituality in the United Nations Archive. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language… find
  6. Google. 2026. Gemma 4: New Open Models for Developers. Accessed: 2026-04-16. find
  7. Bogdan Grecu, Gerrit Quaremba, Elizabeth Black, Denny Vrandei, Elena Simperl, and Oana Cocarascu. 2026. Argchestrators at UZH Shared Task 2026: Efficient Argument Mining in UN Resolutions: A Sub-8B Pipeline using Agentic Debate and Heuristic Retrieval. In Proceedings of the 13th Workshop on Argument… find
  8. Vedant Gupta, Rahul Bhatia, Vaibhav Varshney, and Manjunatha Naik MC. 2026. RESOLVENOW at UZH Shared Task 2026: Rule-Based Type Classification with LLM-Driven Multi-Label Tagging for UN Resolutions. In Proceedings of the 13th Workshop on Argument Mining and Reasoning, Budapest, Hungary. Association… find
  9. Phuong Huu Vu Tran, Long Vo Minh, Son Nguyen Minh Le, and Hoang Van. 2026. LLM-INSTRUCT at UZH Shared Task 2026: Constraint-Aware Retrieval and Selective Debate for Paragraph-Level Argument Mining. In Proceedings of the 13th Workshop on Argument Mining and Reasoning, Budapest, Hungary. Association f…
  10. Rositsa V Ivanova and Reto Gubelmann. The Shift from Logic to Dialectic in Argumentation Theory: Implications for Computational Argument Quality Assessment. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4789–4802. Association for Computational Linguistics. find
  11. Siddhartha Khandelwal and Jyotsana Bhardwaj. 2026. Prompteam at UZH Shared Task 2026: RAG-Augmented Classification and Cosine-Filtered Relation Prediction for UN Resolutions. In Proceedings of the 13th Workshop on Argument Mining and Reasoning, Budapest, Hungary. Association for Computational Lingui… find
  12. Jyoti Kumari, Vinay Babu Ulli, Chandan Kumar R S, and Vaibhav Singh. 2026. TypeCoT at UZH Shared Task 2026: Reconstructing Argumentative Structure in UN Resolutions using Type-Informed Chain-of-Thought. In Proceedings of the 13th Workshop on Argument Mining and Reasoning, Budapest, Hungary. Associat…
  13. Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Kai Eckert. 2018. An Argument-Annotated Corpus of Scientific Publications. In Proceedings of the 5th Workshop on Argument Mining, pages 40–46, Brussels, Belgium. Association for Computational Linguistics. find
  14. John Lawrence and Chris Reed. 2019. Argument Mining: A Survey. Computational Linguistics, 45(4):765–818.
  15. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented Generation for Knowledge-intensive NLP Tasks. Advances in Neural Information Process…
  16. Patrick Montjouridès. 2022. Is this the Future We Want? Understanding the Legitimacy of International Education Agendas. The Example of Equity in Education. In PhD Thesis. University of Cambridge. find
  17. Margarete Sachs-Israel. 2016. The SDG 4-Education 2030 Agenda and Its Framework for Action - the Process of Its Development and First Steps in Taking It Forward. Bildung und Erziehung, 69(3):269–290. Place: Göttingen Publisher: Vandenhoeck und Ruprecht. find
  18. Giuseppina Scotto di Carlo. 2013. The Language of the UN: Vagueness in Security Council Resolutions Relating to the Second Gulf War. International Journal for the Semiotics of Law - Revue internationale de Sémiotique juridique, 26(3):693–706. find
  19. Giuseppina Scotto di Carlo. 2017. Linguistic Patterns of Modality in UN Resolutions: The Role of Shall, Should, and May in Security Council Resolutions Relating to the Second Gulf War. International Journal for the Semiotics of Law - Revue internationale de Sémiotique juridique, 30(2):223–244. find
  20. Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT–Building Open Translation Services for the World. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479–480. find
  21. UNESCO. 2012. International Standard Classification of Education (ISCED) 2011. UNESCO. find
  22. United Nations. 1983. United Nations Editorial Manual. find
  23. United Nations. 2025. United Nations Editorial Manual. Resolutions and other formal decisions of United Nations organs. find
  24. Henning Wachsmuth, Nona Naderi, Yufang Hou, Yonatan Bilu, Vinodkumar Prabhakaran, Tim Alberd-ingk Thijm, Graeme Hirst, and Benno Stein. 2017. Computational Argumentation Quality Assessment in Natural Language. In Proceedings of the 15th Conference of the European Chapter of the Association for Compu…
  25. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35:24824–24837.
  26. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jin… — Qwen3 Technical Report
  27. Sohom Sen, Avina Nakarmi, Xun Song, and Aritra Dasgupta. 2026. POINTERS at UZH Shared Task 2026: Evident: Reasoning Probes for Argumentation Mining in UN Resolutions. In Proceedings of the 13th Workshop on Argument Mining and Reasoning, Budapest, Hungary. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shaitarova-2026-overview-uzh-shared-task-2026/