Elamur Research⌕
☰
home / papers / 2025 / MultiCoPIE: A Multilingual Corpus of Potentially Idiomatic Expressions for Cross-lingual PIE Disambiguation

MultiCoPIE: A Multilingual Corpus of Potentially Idiomatic Expressions for Cross-lingual PIE Disambiguation

Summary

Potentially idiomatic expressions (PIEs) can be read literally or figuratively depending on context. Annotated PIE data outside English is scarce, which limits work on PIE disambiguation, the task of deciding which reading a given occurrence has. The authors manually built MultiCoPIE, a corpus of PIE occurrences in Catalan, Italian and Russian. For each idiom it records semantic compositionality, the part of speech of its head, and its English counterpart.

They fine-tuned XLM-RoBERTa on English PIE data and tested it on the three new languages. It was compared with a majority baseline and with XLM-RoBERTa fine-tuned monolingually on MultiCoPIE (xlm-r-multicopie), with and without surrounding context. The English-trained model beat the majority baseline in every language and outperformed the monolingual model for Italian and Catalan, but scored slightly lower on Russian.

Accuracy rose sharply for idioms whose English equivalent, similar in form and meaning, appeared in training. Such lexical overlap can therefore make cross-lingual results look better than they are. Even so, clear transfer remained once these idioms were set aside, which helps researchers evaluate idiom handling in lower-resource languages more honestly.

Key points

  • Resource: MultiCoPIE covers Catalan, Italian and Russian and is publicly released at github.com/at-uliana/multicopie. Idioms were drawn from dictionaries and lexicographic sources, including Il Nuovo De Mauro, Hoepli and the Russian Wiktionary, and verbal idioms form the majority.
  • Setup: XLM-RoBERTa was fine-tuned on English data across 10 runs and evaluated cross-lingually. Idioms were split into "shared and seen" versus "not shared or not seen" groups, and the difference was tested with one-way ANOVA.
  • Lexical overlap: Accuracy and F1 were significantly higher for shared idioms in all three languages. In the non-shared group, the model still beat the majority baseline everywhere and xlm-r-multicopie for Italian and Catalan.
  • Pre-training data: The authors partly attribute Russian's stronger monolingual baseline to its much larger XLM-R pre-training data: 23,408M tokens, compared with 4,983M for Italian and 1,752M for Catalan.
  • Limitations:
    • The data is skewed toward idiomatic instances.
    • Each language had only one annotator.
    • Only Indo-European languages were tested.
    • Removing context gave mixed effects on the metrics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

4 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.mwe-1.8

@inproceedings{sentsova2025multicopie,
  title = {{MultiCoPIE: A Multilingual Corpus of Potentially Idiomatic Expressions for Cross-lingual PIE Disambiguation}},
  author = {Sentsova, Uliana and Ciminari, Debora and Genabith, Josef van and España-Bonet, Cristina},
  booktitle = {Proceedings of the 21st Workshop on Multiword Expressions (MWE 2025)},
  year = {2025},
  doi = {10.18653/v1/2025.mwe-1.8},
  url = {https://doi.org/10.18653/v1/2025.mwe-1.8}
}

References

59 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Paul Cook, Afsaneh Fazly, and Suzanne Stevenson. 2008. The vnc-tokens dataset. find
  2. Julien Abadji, Pedro Javier Ortiz Suárez, Laurent Romary, and Benoît Sagot. 2021. Ungoliant: An optimized pipeline for the generation of a very large-scale multilingual web corpus. In Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-9) 2021. Limerick, 12 July 2021 (… find
  3. Verna Dankers, Christopher Lucas, and Ivan Titov. 2022. Can transformer be too compositional? analysing idiom processing in neural machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3608–3626, Dublin, Ireland… find our card
  4. Lasha Abzianidze, Johannes Bjerva, Kilian Evang, Hessel Haagsma, Rik van Noord, Pierre Ludmann, Duc-Duy Nguyen, and Johan Bos. 2017. The Parallel Meaning Bank: Towards a multilingual corpus of translations annotated with compositional meaning representations. In Proceedings of the 15th Conference of… find our card
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  6. Rafael Ehren, Timm Lichte, Laura Kallmeyer, and Jakub Waszczuk. 2020. Supervised disambiguation of German verbal idioms with a BiLSTM architecture. In Proceedings of the Second Workshop on Figurative Language Processing, pages 211–220, Online. Association for Computational Linguistics. find
  7. Tosin Adewumi, Roshanak Vadoodi, Aparajita Tripathy, Konstantina Nikolaido, Foteini Liwicki, and Marcus Liwicki. 2022. Potential idiomatic expression (PIE)-English: Corpus for classes of idioms. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 689–696, Marseille,… find
  8. Samin Fakharian and Paul Cook. 2021. Contextualized embeddings encode monolingual and cross-lingual knowledge of idiomaticity. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 23–32, Online. Association for Computational Linguistics. find
  9. Ruchit Agrawal, Vighnesh Chenthil Kumar, Vignesh-waran Muralidharan, and Dipti Sharma. 2018. No more beating about the bush : A step towards idiom handling for Indian language NLP. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Jap… find
  10. Afsaneh Fazly, Paul Cook, and Suzanne Stevenson. 2009. Unsupervised type and token identification of idiomatic expressions. Computational Linguistics, 35(1):61–103. find our card
  11. Katsiaryna Aharodnik, Anna Feldman, and Jing Peng. 2018. Designing a russian idiom-annotated corpus. In International Conference on Language Resources and Evaluation. find
  12. Fabienne Fritzinger, Marion Weller, and Ulrich Heid. 2010. A survey of idiomatic preposition-noun-verb triples on token level. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA). find
  13. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Handbook of Natural Language Processing. find
  14. BNC Consortium. 2007. The british national corpus, XML edition. Distributed by Oxford University Computing Services on behalf of the BNC Consortium. URL: http://www.natcorp.ox.ac.uk/. ↗
  15. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021a. Assessing the representations of idiomaticity in vector models with a noun compound dataset labeled at type and token levels. In Proceedings of the 59th Annual Meeting of the Association for Computati… find
  16. Zheng Chu, Ziqing Yang, Yiming Cui, Zhigang Chen, and Ming Liu. 2022. HIT at SemEval-2022 task 2: Pre-trained language model for idioms detection. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 221– 227, Seattle, United States. Association for Computat… find
  17. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021b. Probing for idiomaticity in vector space models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3551–3564, Online. A… find our card
  18. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. find
  19. Lynn Grant. 2004. Criteria for re-defining idioms: Are we barking up the wrong tree? Applied Linguistics -APPL LINGUIST, 25:38–61. source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedin… find
  20. Hessel Haagsma, Johan Bos, and Malvina Nissim. 2020. MAGPIE: A large corpus of potentially idiomatic expressions. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 279–287, Marseille, France. European Language Resources Association. find our card
  21. Ioannis Korkontzelos, Torsten Zesch, Fabio Massimo Zanzotto, and Chris Biemann. 2013. SemEval-2013 task 5: Evaluating phrasal semantics. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEv… find
  22. Najet Hadj Mohamed, Agata Savary, Cherifa Ben Khelil, Jean-Yves Antoine, Iskandar Keskes, and Lamia Hadrich-Belguith. 2024. Lexicons gain the upper hand in Arabic MWE identification. In Proceedings of the Joint Workshop on Multiword Expressions and Universal Dependencies (MWE-UD) @ LREC-COLING 2024,… find
  23. Murathan Kurfalı and Robert Östling. 2020. Disambiguation of potentially idiomatic expressions with contextual embeddings. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 85–94, online. Association for Computational Linguistics. find
  24. Reyhaneh Hashempour and Aline Villavicencio. 2020. Leveraging contextual embeddings and idiom principle for detecting idiomaticity in potentially idiomatic expressions. In Proceedings of the Workshop on the Cognitive Aspects of the Lexicon, pages 72–80, Online. Association for Computational Linguist… find
  25. Murathan Kurfalı, Robert Östling, Johan Sjons, and Mats Wirén. 2020. A multi-word expression dataset for Swedish. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4402–4409, Marseille, France. European Language Resources Association. find
  26. Chikara Hashimoto and Daisuke Kawahara. 2008. Construction of an idiom corpus and its application to idiom identification based on WSD incorporating idiom-specific features. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 992–1001, Honolulu, Hawaii. A… find
  27. Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020. From zero to hero: On the limitations of zero-shot language transfer with multilingual Transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4483–4499, Online. Asso… find our card
  28. Bradley Hauer, Seeratpal Jaura, Talgat Omarov, and Grzegorz Kondrak. 2022. UAlberta at SemEval 2022 task 2: Leveraging glosses and translations for multilingual idiomaticity detection. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 145–150, Seattle, Un… find
  29. Li Liu and Francois Lareau. 2024. Assessing BERT’s sensitivity to idiomaticity. In Proceedings of the Joint Workshop on Multiword Expressions and Universal Dependencies (MWE-UD) @ LREC-COLING 2024, pages 14–23, Torino, Italia. ELRA and ICCL. find
  30. Andrea Horbach, Andrea Hensler, Sabine Krome, Jakob Prange, Werner Scholze-Stubenrecht, Diana Steffen, Stefan Thater, Christian Wellner, and Manfred Pinkal. 2016. A corpus of literal and idiomatic uses of German infinitive-verb compounds. In Proceedings of the Tenth International Conference on Langu… find
  31. Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020. Camembert: a tasty french language model. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Co… find our card
  32. Oren Melamud, Jacob Goldberger, and Ido Dagan. 2016. context2vec: Learning generic context embedding with bidirectional LSTM. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pages 51–61, Berlin, Germany. Association for Computational Linguistics. find our card
  33. Agne Knietaite, Adam Allsebrook, Anton Minkov, Adam Tomaszewski, Norbert Slinko, Richard Johnson, Thomas Pickard, Dylan Phelps, and Aline Villavicencio. 2024. Is less more? quality, quantity and context in idiom processing with natural language models. Preprint, arXiv:2405.08497. find
  34. Navnita Nandakumar, Timothy Baldwin, and Bahar Salehi. 2019. How well do embedding models capture non-compositionality? a view from multiword expressions. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, pages 27–34, Minneapolis, USA. Association for Computation… find
  35. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. find
  36. Moses: Open Vasudevan Nedumpozhimana and John Kelleher. 2021. Finding BERT’s idiomatic key. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 57–62, Online. Association for Computational Linguistics. expressions at length and in depth: Extended papers from the MWE 2017 w… find
  37. Agata Savary, Silvio Cordeiro, Timm Lichte, Carlos Ramisch, Uxoa Iñurrieta, and Voula Giouli. 2019. Literal occurrences of multiword expressions: Rare birds that cause a stir. 112:5–54. find
  38. Geoffrey Nunberg, Ivan A. Sag, and Thomas Wasow. 1994. Idioms. Language, 70(3):491–538. find
  39. Agata Savary, Manfred Sailer, Yannick Parmentier, Michael Rosner, Victoria Rosén, Adam Przepiórkowski, Cvetana Krstev, Veronika Vincze, Beata Wójtowicz, Gyri Smørdal Losnegaard, Carla Parra Escartín, Jakub Waszczuk, Mathieu Constant, Petya Osenova, and Federico Sangati. 2015. PARSEME – PARSing and M… find
  40. Pedro Javier Ortiz Suárez, Benoît Sagot, and Laurent Romary. 2019. Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. In Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-7) 2019. Cardiff, 22nd July 2019, pages 9–16, Mannhei… find
  41. Carlos Ramisch. 2023. Multiword expressions in computational linguistics. Habilitation à diriger des recherches, Aix Marseille Université (AMU). find
  42. Prateek Saxena and Soma Paul. 2020. Epie dataset: A corpus for possible idiomatic expressions. In Text, Speech, and Dialogue: 23rd International Conference, TSD 2020, Brno, Czech Republic, September 8–11, 2020, Proceedings, page 87–94, Berlin, Heidelberg. Springer-Verlag. find
  43. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann A. Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for nlp. In Conference on Intelligent Text Processing and Computational Linguistics. find our card
  44. Vered Shwartz and Ido Dagan. 2019. Still a pain in the neck: Evaluating text representations on lexical composition. Transactions of the Association for Computational Linguistics, 7:403–419. find our card
  45. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. Preprint, arXiv:1910.01108. find
  46. J. Sinclair. 1991. Corpus, Concordance, Collocation. Describing English language. Oxford University Press. find
  47. Mahtab Sarlak, Yalda Yarandi, and Mehrnoush Shamsfard. 2023. Predicting compositionality of verbal multiword expressions in Persian. In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 14–23, Dubrovnik, Croatia. Association for Computational Linguistics. find
  48. Caroline Sporleder, Linlin Li, Philip Gorinski, and Xaver Koch. 2010. Idioms in context: The IDIX corpus. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA). find
  49. Agata Savary, Cherifa Ben Khelil, Carlos Ramisch, Voula Giouli, Verginica Barbu Mititelu, Najet Hadj Mohamed, Cvetana Krstev, Chaya Liebeskind, Hongzhi Xu, Sara Stymne, Tunga Güngör, Thomas Pickard, Bruno Guillaume, Eduard Bejček, Archna Bhatia, Marie Candito, Polona Gantar, Uxoa Iñurrieta, Albert… find
  50. Minghuan Tan and Jing Jiang. 2021. Does BERT understand idioms? a probing-based empirical study of BERT encodings of idioms. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1397–1407, Held Online. INCOMA Ltd. find
  51. Harish Tayyar Madabushi, Edward Gow-Smith, Marcos Garcia, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2022. SemEval-2022 task 2: Multilingual idiomaticity detection and sentence embedding. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 107… find our card
  52. Agata Savary, Marie Candito, Verginica Barbu Mititelu, Eduard Bejček, Fabienne Cap, Slavomír Čéplö, Silvio Ricardo Cordeiro, Gülşen Cebiroğlu Eryiğit, Voula Giouli, Maarten van Gompel, Yaakov Hacohen-Kerner, Jolanta Kovalevskaitė, Simon Krek, Chaya Liebeskind, Johanna Monti, Carla Parra Escart… find our card
  53. Simone Tedeschi, Federico Martelli, and Roberto Navigli. 2022. ID10M: Idiom identification in 10 languages. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2715–2726, Seattle, United States. Association for Computational Linguistics. find our card
  54. Simone Tedeschi and Roberto Navigli. 2022. NER4ID at SemEval-2022 task 2: Named entity recognition for idiomaticity detection. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 204–210, Seattle, United States. Association for Computational Linguistics. find
  55. Ye Tian, Isobel James, and Hye Son. 2023. How are idioms processed inside transformer language models? In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 174–179, Toronto, Canada. Association for Computational Linguistics. find our card
  56. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  57. Atsuki Yamaguchi, Gaku Morio, Hiroaki Ozaki, and Yasuhiro Sogawa. 2022. Hitachi at SemEval-2022 task 2: On the effectiveness of span-based classification approaches for multilingual idiomaticity detection. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages… find
  58. Lang Yu and Allyson Ettinger. 2020. Assessing phrasal representation and composition in transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4896–4907, Online. Association for Computational Linguistics. find
  59. Ziheng Zeng and Suma Bhat. 2021. Idiomatic expression identification using semantic compatibility. Transactions of the Association for Computational Linguistics, 9:1546–1562. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/sentsova-2025-multicopie-multilingual-corpus-potentially-idiomatic/