Elamur Research⌕
☰
home / papers / 2026 / Edition 2.0 of the PARSEME shared task on multilingual identification and paraphrasing of multiword expressions

Edition 2.0 of the PARSEME shared task on multilingual identification and paraphrasing of multiword expressions

Summary

Multiword expressions (MWEs) remain hard for NLP because they are frequent, partly irregular and often non-compositional. Earlier PARSEME shared tasks handled only verbal MWEs. Edition 2.0 covers every syntactic category: verbal, nominal, adjectival, adverbial and functional. In subtask 1, systems label MWE tokens in sentences from the PARSEME 2.0 corpus (CoNLL-U/CUPT format, 17 languages). The test data is entirely new, which limits LLM contamination. Subtask 2 is new. Systems rewrite a sentence containing one idiom so that the idiom disappears but the meaning stays the same. Native speakers wrote the gold paraphrases for a new dataset in 14 languages. Evaluation runs on Codabench and combines F-scores, BERT-score, manual judgments and diversity metrics. Ten systems entered identification and five entered paraphrasing, baselines included. Pre-trained encoders with BIO tagging remain strong, but long-known identification problems are still unsolved. Diversity goes along with better identification but trades off against paraphrase quality. The resources offer a multilingual benchmark for MWE processing in the LLM era.

Key points

  • Resources: the PARSEME 2.0 corpus, annotated for all MWE categories in 17 languages, and a new idiom-paraphrase dataset in 14 languages covering verbal, nominal and adjectival idioms (VID, NID, AdjID).
  • Identification metrics: MWE-based and token-based F1, plus phenomenon-specific scores for discontinuous, single-token, variant and unseen MWEs. Category labels are not scored.
  • Diversity in identification: MWE-based F1 correlates with the hybrid variety-balance measure at Pearson 0.72 and Spearman 0.76. Variety alone correlates at 0.81, while balance correlates negatively (-0.39 Pearson, -0.46 Spearman).
  • Paraphrasing shows a performance-diversity trade-off. Star-Paraphraser-Cosine gets high BERT-scores because its output stays close to the input, yet it is not best in manual evaluation. MISP for Romanian is lexically rich but receives low manual scores.
  • Limitations: the data is unbalanced across languages. Georgian is very large but sparsely annotated, and Ancient Greek has no training or development data. No statistical significance tests are reported. The LLM baseline is open-weight but not fully open.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.mwe-1.33

@inproceedings{scholivet2026edition,
  title = {{Edition 2.0 of the PARSEME shared task on multilingual identification and paraphrasing of multiword expressions}},
  author = {Scholivet, Manon and Savary, Agata and Ramisch, Carlos and Bilinski, Eric and Nakamura, Takuya and Mitrofan, Maria and Pais, Vasile},
  booktitle = {Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.mwe-1.33},
  url = {https://doi.org/10.18653/v1/2026.mwe-1.33}
}

References

60 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Iñigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015. SemEval-2015 task 2: Semantic textual similarity, English, Spanish and pilot on interpretability. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 252–263, Denver, Colorado. Association for Computational Linguistics. find
  2. Dogukan Arslan, Rodrigo Wilkens, Wei He, Dilara Torunoğlu-Selamet, Thomas Pickard, Aline Villavicencio, Adriana S. Pagano, and Gülşen Eryiğit. 2026. MWE-2026 Shared Task 2: AdMIRe 2 - Advancing Multimodal Idiomaticity Representation. In Proceedings of the 22nd International Workshop on Multiword Expressions (MWE-2026). find
  3. Ron Artstein and Massimo Poesio. 2008. Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. find
  4. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of natural language processing, volume 2, pages 267–292. CRC Press, Boca Raton, USA. find
  5. Petra Barančíková and Václava Kettnerová. 2018. Paraphrases of verbal multiword expressions: The case of Czech light verbs and idioms. In Stella Markantonatou, Carlos Ramisch, Agata Savary, and Veronika Vincze, editors, Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop, pages 35–59. Language Science Press., Berlin. ↗
  6. Elif Bayraktar, Vedat Doğancan, Muhammed A. Gümüş, and Nusret Ali Kızılaslan. 2026. Semantic Stars at PARSEME 2.0 Subtask 2:Alternative Approaches for MWE Paraphrasing. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  7. Anna Bogdanova and Ileana Bucur. 2026. PMI MWE Scorer at PARSEME 2.0 Subtask 1: identifying multiword expressions using pointwise mutual information and universal dependencies. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  8. Cristina Butnariu, Su Nam Kim, Preslav Nakov, Diarmuid Ó Séaghdha, Stan Szpakowicz, and Tony Veale. 2009. SemEval-2010 task 9: The interpretation of noun compounds using paraphrasing verbs and prepositions. In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions (SEW-2009), pages 100–105, Boulder, Colorado. Association for Computational Linguistics. find
  9. Marie Candito, Mathieu Constant, Carlos Ramisch, Agata Savary, Bruno Guillaume, Yannick Parmentier, and Silvio Cordeiro. 2021. A french corpus annotated for multiword expressions and named entities. Journal of Language Modelling, 8(2). ↗
  10. Debora Ciminari and Alberto Barrón-Cedeño. 2026. MISP at PARSEME 2.0 Subtask 2: A Cross-lingual Approach to Multiword Expression Paraphrasing. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  11. Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke van der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017. Multiword Expression Processing: A Survey. Computational Linguistics, 43(4):837–892. ↗ our card
  12. Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, and Arman Cohan. 2024. Unveiling the spectrum of data contamination in language model: A survey from detection to remediation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 16078–16092, Bangkok, Thailand. Association for Computational Linguistics. find
  13. Ahmet Erdem and Oguzhan Karaarslan. 2026. Cross Lingual BERT at PARSEME 2.0 Subtask 1: Can Cross-Lingual Interactions Improve MWE Identification? In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  14. Louis Estève, Marie-Catherine de Marneffe, Nurit Melnik, Agata Savary, and Olha Kanishcheva. 2025. A survey of diversity quantification in natural language processing: The why, what, where and how. Preprint, arXiv:2507.20858. ↗
  15. Gaston Gross. 1988. Degré de figement des noms composés. Langages, 90:57–72. ↗
  16. Maurice Gross. 1986. Lexicon-grammar: The representation of compound words. In Proceedings of the 11th Coference on Computational Linguistics, COL-ING ’86, pages 1–6. Association for Computational Linguistics. find
  17. Maurice Gross and Jean Senellart. 1998. Nouvelles bases statistiques pour les mots du français. In Proceedings of JADT’98, Nice 1998, pages 335–349. find
  18. Michael Hanna and Ondřej Bojar. 2021. A fine-grained analysis of BERTScore. In Proceedings of the Sixth Conference on Machine Translation, pages 507–517, Online. Association for Computational Linguistics. ↗ our card
  19. Wei He, Tiago Kramer Vieira, Marcos Garcia, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2025. Investigating idiomaticity in word representations. Computational Linguistics, 51:505–555. ↗
  20. Iris Hendrickx, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Stan Szpakowicz, and Tony Veale. 2013. SemEval-2013 task 4: Free paraphrases of noun compounds. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 138– 143, Atlanta, Georgia, USA. Association for Computational Linguistics. find
  21. Veronika Vincze. 2021. PMWE conventions for examples containing multiword expressions. Technical report, Phraseology and Multiword Expressions – book series at Language Science Press. find
  22. Anna Hülsing, Noah-Manuel Michael, Daniel Ignacio Mora Melanchthon, and Andrea Horbach. 2026. IPN at PARSEME 2.0 Subtask 1: MWE Identification via Related Languages and Attempts at Harnessing Thinking Mode. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  23. Igor Mel’čuk. 2010. La phraséologie en langue, en dictionnaire et en TALN. In Actes de la 17ème Conférence sur le Traitement Automatique des Langues Naturelles 2010, Montréal, Canada. find
  24. Daphne Ippolito, Reno Kriz, João Sedoc, Maria Kustikova, and Chris Callison-Burch. 2019. Comparison of diverse decoding methods from conditional language models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3752–3762, Florence, Italy. Association for Computational Linguistics. find
  25. Irina Moise and Sergiu Nisioi. 2026. MorphoFiltered-Gemini at PARSEME 2.0 Subtask 1: Tackling LLM Overgeneration via Universal POS-based Constraints. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  26. Florian Mueller. 2025. First copyright ruling against OpenAI worldwide: music rights collecting society wins German injunction over song lyrics —to be appealed now. Accessed on 01.01.2026. find
  27. Maria Pershina, Yifan He, and Ralph Grishman. 2015. Idiom paraphrases: Seventh heaven vs cloud nine. In Proceedings of the First Workshop on Linking Computational Models of Lexical, Sentential and Discourse-level Semantics, pages 76–82, Lisbon, Portugal. Association for Computational Linguistics. find
  28. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Computational Linguistics. find our card
  29. Yunus Karatepe, Mert Sülük, Begüm Özbay, and Zeynep Tuğçe Kırımlı. 2026. Sahara Tokenizers at PARSEME 2.0 Subtask 1: Combining Contextual Embeddings with Structural Decoding for Multi-Word Expression Detection. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  30. Wuwei Lan, Siyu Qiu, Hua He, and Wei Xu. 2017. A continuously growing dataset of sentential paraphrases. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1224–1234, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  31. Adam Lion-Bouton, Yagmur Ozturk, Agata Savary, and Jean-Yves Antoine. 2022. Evaluating diversity of multiword expressions in annotated text. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3285–3295, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. find
  32. Thomas Pickard, Aline Villavicencio, Maggie Mi, Wei He, Dylan Phelps, and Marco Idiart. 2025. SemEval-2025 task 1: AdMIRe - advancing multimodal idiomaticity representation. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2597–2609, Vienna, Austria. Association for Computational Linguistics. ↗ our card
  33. Jipeng Qiang, Yang Li, Chaowei Zhang, Yun Li, Yi Zhu, Yunhao Yuan, and Xindong Wu. 2023. Chinese idiom paraphrasing. Transactions of the Association for Computational Linguistics, 11:740–754. ↗ our card
  34. Pedro Ramaciotti Morales, Robin Lamarche-Perrin, Raphaël Fournier-S’Niehotta, Rémy Poulain, Lionel Tabourier, and Fabien Tarissan. 2021. Measuring diversity in heterogeneous information networks. Theoretical Computer Science, 859:80–115. Publisher: Elsevier. ↗
  35. Changsheng Liu and Rebecca Hwa. 2016. Phrasal substitution of idiomatic expressions. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 363–373, San Diego, California. Association for Computational Linguistics. find
  36. Carlos Ramisch, Silvio Ricardo Cordeiro, Agata Savary, Veronika Vincze, Verginica Barbu Mititelu, Archna Bhatia, Maja Buljan, Marie Candito, Polona Gantar, Voula Giouli, Tunga Güngör, Abdelati Hawwari, Uxoa Iñurrieta, Jolanta Kovalevskaitė, Simon Krek, Timm Lichte, Chaya Liebeskind, Johanna Monti, Carla Parra Escartín, and 6 others. 2018. Edition 1.1 of the PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pages 222–240, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card
  37. Stella Markantonatou, Carlos Ramisch, Victoria Rosén, Mike Rosner, Manfred Sailer, Agata Savary, and Carlos Ramisch, Agata Savary, Bruno Guillaume, Jakub Waszczuk, Marie Candito, Ashwini Vaidya, Verginica Barbu Mititelu, Archna Bhatia, Uxoa Iñurrieta, Voula Giouli, Tunga Güngör, Menghan Jiang, Timm Lichte, Chaya Liebeskind, Johanna Monti, Renata Ramisch, Sara Stymne, Abigail Walsh, and Hongzhi Xu. 2020. Edition 1.2 of the PARSEME shared task on semi-supervised identification of verbal multiword expressions. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 107– 118, online. Association for Computational Linguistics. find our card
  38. Carlos Ramisch, Abigail Walsh, Thomas Blanchard, and Shiva Taslimipoor. 2023. A survey of MWE identification experiments: The devil is in the details. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 106–120, Dubrovnik, Croatia. Association for Computational Linguistics. find
  39. Rares-Alexandru Roscan and Sergiu Nisioi. 2026. Archaeology at PARSEME 2.0 Subtasks 1 and 2: Parsing is for Encoders, Paraphrasing is for LLMs. In Proceedings of the 22nd Workshop on Multiword Expressions (MWE 2026), Rabat, Morroco. Association for Computational Linguistics. find
  40. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for nlp. In Computational Linguistics and Intelligent Text Processing, pages 1–15, Berlin, Heidelberg. Springer Berlin Heidelberg. find our card
  41. Lukas Santing, Ryan Sijstermans, Giacomo Anerdi, Pedro Jeuris, Marijn ten Thij, and Riza Batista-Navarro. 2022. Food for thought: How can we exploit contextual embeddings in the translation of idiomatic expressions? In Proceedings of the 3rd Workshop on Figurative Language Processing (FLP), pages 100–110, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find
  42. Agata Savary, Cherifa Ben Khelil, Carlos Ramisch, Voula Giouli, Verginica Barbu Mititelu, Najet Hadj Mohamed, Cvetana Krstev, Chaya Liebeskind, Hongzhi Xu, Sara Stymne, Tunga Güngör, Thomas Pickard, Bruno Guillaume, Eduard Bejček, Archna Bhatia, Marie Candito, Polona Gantar, Uxoa Iñurrieta, Albert Gatt, and 9 others. 2023. PARSEME corpus release 1.3. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 24–35, Dubrovnik, Croatia. Association for Computational Linguistics. find
  43. Agata Savary, Marie Candito, Verginica Barbu Mititelu, Eduard Bejček, Fabienne Cap, Slavomír Čéplö, Silvio Ricardo Cordeiro, Gülşen Eryiğit, Voula Giouli, Maarten van Gompel, Yaakov HaCohen-Kerner, Jolanta Kovalevskaitė, Simon Krek, Chaya Liebeskind, Johanna Monti, Carla Parra Escartín, Lonneke van der Plas, Behrang QasemiZadeh, Carlos Ramisch, and 3 others. 2018. PARSEME multilingual corpus of verbal multiword expressions. In Stella Markantonatou, Carlos Ramisch, Agata Savary, and Veronika Vincze, editors, Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop, pages 87–147. Language Science Press., Berlin. find
  44. Agata Savary, Simon Petitjean, Timm Lichte, Laura Kallemeyer, and Jakub Waszczuk. 2020. Object-oriented lexical encoding of multiword expressions: Short and sweet. Lexique, 27:87–120. ↗
  45. Agata Savary, Carlos Ramisch, Silvio Cordeiro, Federico Sangati, Veronika Vincze, Behrang QasemiZadeh, Marie Candito, Fabienne Cap, Voula Giouli, Ivelina Stoyanova, and Antoine Doucet. 2017. The PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 31–47, Valencia, Spain. Association for Computational Linguistics. find our card
  46. Agata Savary, Manon Scholivet, Carlos Ramisch, Takuya Nakamura, Eric Bilinski, Sara Stymne, Voula Giouli, Stella Markantonatou, Vasile Păiș, Maria Mitrofan, Louis Estève, Bruno Guillaume, Verginica Barbu Mititelu, Jaka Čibej, Roberto A. Díaz Hernández, Victoria Fendel, Polona Gantar, Olha Kanishcheva, Cvetana Krstev, and 9 others. 2026. PARSEME 2.0 multilingual corpus of multiword expressions. In Submitted to LREC 2026, under review. Agata Savary, Daniel Zeman, Verginica Barbu Mititelu, Anabela Barreiro, Olesea Caftanatov, Marie-Catherine de Marneffe, Kaja Dobrovoljc, Gülşen Eryiğit, Voula Giouli, Bruno Guillaume, Stella Markantonatou, Nurit Melnik, Joakim Nivre, Atul Kr. find our card
  47. Ojha, Carlos Ramisch, Abigail Walsh, Beata Wójtowicz, and Alina Wróblewska. 2024. UniDive: A COST action on universality, diversity and idiosyncrasy in language technology. In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024, pages 372–382, Torino, Italia. ELRA and ICCL. find
  48. Nathan Schneider, Dirk Hovy, Anders Johannsen, and Marine Carpuat. 2016. SemEval-2016 task 10: Detecting minimal semantic units and their meanings (DiMSUM). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 546–559, San Diego, California. Association for Computational Linguistics. find our card
  49. Vered Shwartz and Ido Dagan. 2019. Still a pain in the neck: Evaluating text representations on lexical composition. Transactions of the Association for Computational Linguistics, 7:403–419. ↗ our card
  50. Benjamin Smith and J. Bastow Wilson. 1996. A Consumer’s Guide to Evenness Indices. Oikos, 76(1):70– 82. Number: 1 Publisher: [Nordic Society Oikos, Wiley]. ↗
  51. Andy Stirling. 2007. A general framework for analysing diversity in science, technology and society. Journal of The Royal Society Interface, 4(15):707–719. Number: 15 Publisher: Royal Society. ↗
  52. Tianxiang Sun, Junliang He, Xipeng Qiu, and Xuanjing Huang. 2022. BERTScore is unfair: On social bias in language model-based metrics for text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3726– 3739, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  53. Minghuan Tan and Jing Jiang. 2021. Does BERT understand idioms? a probing-based empirical study of BERT encodings of idioms. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1397– 1407, Held Online. INCOMA Ltd. find
  54. Shiva Taslimipoor, Sara Bahaadini, and Ekaterina Kochmar. 2020. MTLB-STRUCT @parseme 2020: Capturing unseen multiword expressions using multi-task learning and pre-trained masked language models. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 142– 148, online. Association for Computational Linguistics. ↗
  55. Harish Tayyar Madabushi, Edward Gow-Smith, Marcos Garcia, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2022. SemEval-2022 task 2: Multilingual idiomaticity detection and sentence embedding. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 107–121, Seattle, United States. Association for Computational Linguistics. ↗ our card
  56. Takashi Wada, Yuji Matsumoto, Timothy Baldwin, and Jey Han Lau. 2023. Unsupervised paraphrasing of multiword expressions. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4732–4746, Toronto, Canada. Association for Computational Linguistics. find
  57. Wei Xu, Chris Callison-Burch, and Bill Dolan. 2015. SemEval-2015 task 1: Paraphrase and semantic similarity in Twitter (PIT). In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 1–11, Denver, Colorado. Association for Computational Linguistics. find our card
  58. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. ↗
  59. Jianing Zhou and Suma Bhat. 2021. Paraphrase generation: A survey of the state of the art. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5075–5086, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  60. Jianing Zhou, Hongyu Gong, and Suma Bhat. 2021. PIE: A parallel idiomatic expression corpus for idiomatic sentence generation and paraphrasing. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 33–48, Online. Association for Computational Linguistics. A Detailed shared task results find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/scholivet-2026-edition-2-0-parseme-shared/