Elamur Research⌕
☰
home / papers / 2022 / ID10M: Idiom Identification in 10 Languages

ID10M: Idiom Identification in 10 Languages

Summary

Idioms are expressions whose figurative sense cannot be fully reconstructed from their parts. They are a persistent obstacle for language understanding, and most earlier identification work covered only English and a small set of expressions. ID10M addresses this gap across ten languages. The authors treat idiom identification as sequence labeling, similar to NER, so the system tags idiomatic spans directly in raw text and needs no pre-selected candidate phrases. Training data is built automatically: multi-word expressions from Wiktionary are matched against WikiMatrix sentences, using lemmatization to capture inflected variants, and each match inherits Wiktionary's idiomatic or literal label. A dual-encoder filter then corrects noisy labels. It calls an expression idiomatic when its embedding is dissimilar enough from its context's embedding. The authors also manually curated a test benchmark in four languages. The resulting tagger handles seen idioms almost perfectly and identifies most unseen ones. This gives a multilingual foundation for downstream tasks such as machine translation, WSD and QA.

Key points

  • The silver training data spans ten languages: the Romance languages Spanish, Portuguese, Italian and French, the Germanic languages German, English and Dutch, plus Polish, Japanese and Chinese. It is drawn from Wiktionary (via the Wiktextract library) and WikiMatrix, a parallel corpus mined from Wikipedia that spans 83 languages.
  • Automatic validation uses two bert-base-multilingual-cased encoders, one for the expression and one for its context. A potentially idiomatic expression is labeled idiomatic when the cosine similarity between the two falls below a manually tuned threshold δ.
  • Training on the validated data improves results by 4.3 F1 points on average compared with the raw silver data.
  • Average F1 is 91.5 on seen idioms and 59.6 on unseen ones, with a best macro F1 of 85.4.
  • The most frequent error is tagging a literal occurrence as idiomatic, which stems from skewed per-expression label distributions in training. Data and code are released at github.com/Babelscape/ID10M.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

39 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 31 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-naacl.208

@inproceedings{tedeschi2022id10m,
  title = {{ID10M: Idiom Identification in 10 Languages}},
  author = {Tedeschi, Simone and Martelli, Federico and Navigli, Roberto},
  booktitle = {Findings of the Association for Computational Linguistics: NAACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-naacl.208},
  url = {https://doi.org/10.18653/v1/2022.findings-naacl.208}
}

References

52 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Judit Ács, Ákos Kádár, and Andras Kornai. 2021. Subword pooling makes a difference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2284–2295, Online. Association for Computational Linguistics. find
  2. Paul Cook, Afsaneh Fazly, and Suzanne Stevenson. 2007. Pulling their weight: Exploiting syntactic forms for the automatic identification of idiomatic expressions in context. In Proceedings of the Workshop on A Broader Perspective on Multiword Expressions, pages 41–48, Prague, Czech Republic. Associa… find our card
  3. Dimitra Anastasiou. 2010. Idiom treatment experiments in machine translation. Cambridge Scholars Publishing. find
  4. Paul Cook, Afsaneh Fazly, and Suzanne Stevenson. 2008. The vnc-tokens dataset. In Proceedings of the LREC Workshop Towards a Shared Task for Multiword Expressions (MWE 2008), pages 19–22. find
  5. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. Handbook of natural language processing, 2:267–292. find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1). find
  7. Michele Bevilacqua, Tommaso Pasini, Alessandro Raganato, and Roberto Navigli. 2021. Recent trends in word sense disambiguation: A survey. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4330–4338. International Joint Conferences on Artificia… find
  8. Terra Blevins and Luke Zettlemoyer. 2020. Moving down the long tail of word sense disambiguation with gloss informed bi-encoders. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1006–1017, Online. Association for Computational Linguistics. find
  9. Mona Diab and Pravin Bhutada. 2009. Verb noun construction mwe token classification. In Proceedings of the Workshop on Multiword Expressions: Identification, Interpretation, Disambiguation and Applications (MWE 2009), pages 17–22. find
  10. Rafael Ehren. 2017. Literal or idiomatic? identifying the reading of single occurrences of German multiword expressions using word embeddings. In Proceedings of the Student Research Workshop at the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 103– 1… find
  11. Samin Fakharian. 2021. Contextualized embeddings encode knowledge of English verb-noun combination idiomaticity. Ph.D. thesis, University of New Brunswick. find
  12. Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Taylor Berg-Kirkpatrick. 2021. Investigating robustness of dialog models to popular figurative language constructs. find
  13. Afsaneh Fazly and Suzanne Stevenson. 2006. Automatically constructing a lexicon of verb phrase idiomatic combinations. In 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, Italy. Association for Computational Linguistics. find our card
  14. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. find
  15. Adriano Ferraresi, Eros Zanchetta, Marco Baroni, and Silvia Bernardini. 2008. Introducing and evaluating ukwac, a very large web-derived corpus of english. In Proceedings of the 4th Web as Corpus Workshop (WAC-4) Can we beat Google, pages 47–54. find
  16. Mahak Gambhir and Vishal Gupta. 2017. Recent automatic text summarization techniques: a survey. Artificial Intelligence Review, 47(1):1–66. find
  17. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021. Probing for idiomaticity in vector space models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3551–3564, Online. As… find our card
  18. Waseem Gharbieh, Virendra Bhavsar, and Paul Cook. 2016. A word embedding approach to identifying verb-noun idiomatic combinations. In Proceedings of the 12th Workshop on Multiword Expressions, pages 112–118, Berlin, Germany. Association for Computational Linguistics. find
  19. Raymond W Gibbs Jr. 1992. What do idioms really mean? Journal of Memory and language, 31(4):485–506. find
  20. Hongyu Gong, Suma Bhat, and Pramod Viswanath. 2017. Geometry of compositionality. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31. find
  21. David Graff and Cieri Christopher. 2003. English gigaword ldc2003t05. web download. In Philadelphia: Linguistic Data Consortium, 2003. find
  22. Reyhaneh Hashempour and Aline Villavicencio. 2020. Leveraging contextual embeddings and idiom principle for detecting idiomaticity in potentially idiomatic expressions. In Proceedings of the Workshop on the Cognitive Aspects of the Lexicon, pages 72–80. find
  23. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. find our card
  24. Nancy Ide and Catherine Macleod. 2001. The american national corpus: A standardized resource of american english. In Proceedings of corpus linguistics, volume 3, pages 1–7. Citeseer. find
  25. Ioannis Korkontzelos, Torsten Zesch, Fabio Massimo Zanzotto, and Chris Biemann. 2013. SemEval-2013 task 5: Evaluating phrasal semantics. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEv… find
  26. Dilin Liu. 2017. Idioms: Description, comprehension, acquisition, and pedagogy. Routledge. find
  27. Harish Tayyar Madabushi, Edward Gow-Smith, Carolina Scarton, and Aline Villavicencio. 2021. Astitchinlanguagemodels: Dataset and methods for the exploration of idiomaticity in pre-trained language models. find our card
  28. Amit Mishra and Sanjay Kumar Jain. 2016. A survey on question answering systems with classification. Journal of King Saud University-Computer and Information Sciences, 28(3):345–361. find
  29. David Mueller, Nicholas Andrews, and Mark Dredze. 2020. Sources of transfer in multilingual named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8093–8104, Online. Association for Computational Linguistics. find
  30. Grace Muzny and Luke Zettlemoyer. 2013. Automatic idiom identification in Wiktionary. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1417–1421, Seattle, Washington, USA. Association for Computational Linguistics. find
  31. Vasudevan Nedumpozhimana and John Kelleher. 2021. Finding bert’s idiomatic key. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 57–62. find
  32. Geoffrey Nunberg, Ivan A Sag, and Thomas Wasow. 1994. Idioms. Language, 70(3):491–538. find
  33. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32. find our card
  34. Jing Peng, Anna Feldman, and Hamza Jazmati. 2015. Classifying idiomatic and literal expressions using vector space representations. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 507–511, Hissar, Bulgaria. INCOMA Ltd. Shoumen, BULGARIA. find
  35. Carlos Ramisch, Aline Villavicencio, Leonardo Moura, and Marco Idiart. 2008. Picking them up and figuring them out: Verb-particle constructions, noise and idiomaticity. In CoNLL 2008: Proceedings of the Twelfth Conference on Computational Natural Language Learning, pages 49–56, Manchester, England. find
  36. Coling 2008 Organizing Committee. find
  37. Ivan A Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for nlp. In International conference on intelligent text processing and computational linguistics, pages 1–15. Springer. find our card
  38. Manfred Sailer and Stella Markantonatou. 2018. Multiword Expressions: Insights from a multi-lingual perspective. Language Science Press. find
  39. Nathan Schneider, Dirk Hovy, Anders Johannsen, and Marine Carpuat. 2016. SemEval-2016 task 10: Detecting minimal semantic units and their meanings (DiMSUM). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 546–559, San Diego, California. Association for… find our card
  40. Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2021. Wiki-Matrix: Mining 135M parallel sentences in 1620 language pairs from Wikipedia. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages… find
  41. Marco Silvio Giuseppe Senaldi, Yuri Bizzoni, and Alessandro Lenci. 2019. What do neural networks actually learn, when they learn to identify idioms? Proceedings of the Society for Computation in Linguistics, 2(1):310–313. find
  42. Ozge Sevgili, Artem Shelmanov, Mikhail Arkhipov, Alexander Panchenko, and Chris Biemann. 2020. Neural entity linking: A survey of models based on deep learning. arXiv preprint arXiv:2006.00575. find
  43. Ekaterina Shutova, Lin Sun, and Anna Korhonen. 2010. Metaphor identification using verb and noun clustering. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1002–1010, Beijing, China. find
  44. Coling 2010 Organizing Committee. find
  45. Caroline Sporleder and Linlin Li. 2009. Unsupervised recognition of literal and non-literal use of idiomatic expressions. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), pages 754–762, Athens, Greece. Association for Computational Linguistics. find our card
  46. Caroline Sporleder, Linlin Li, Philip Gorinski, and Xaver Koch. 2010. Idioms in context: The IDIX corpus. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA). find
  47. Laura Street, Nathan Michalov, Rachel Silverstein, Michael Reynolds, Lurdes Ruela, Felicia Flowers, Angela Talucci, Priscilla Pereira, Gabriella Mor-gon, Samantha Siegel, Marci Barousse, Antequa Anderson, Tashom Carroll, and Anna Feldman. 2010. Like finding a needle in a haystack: Annotating the Ame… find
  48. Simone Tedeschi, Simone Conia, Francesco Cecconi, and Roberto Navigli. 2021. Named Entity Recognition for Entity Linking: What works and what’s next. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2584–2596, Punta Cana, Dominican Republic. Association for Computation… find
  49. Simone Tedeschi and Roberto Navigli. 2022. NER4ID at SemEval-2022 Task 2: Named Entity Recognition for Idiomaticity Detection. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics. find
  50. Rakesh Verma and Vasanthi Vuppuluri. 2015. A new approach for idiom identification using meanings and the web. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 681–687, Hissar, Bulgaria. IN-COMA Ltd. Shoumen, BULGARIA. find
  51. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  52. Vikas Yadav and Steven Bethard. 2018. A survey on recent advances in named entity recognition from deep learning models. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2145–2158, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/tedeschi-2022-id10m-idiom-identification-10-languages/