Elamur Research⌕
☰
home / papers / 2023 / MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation

MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation

Summary

The paper argues that multiword expression (MWE) identification and word sense disambiguation (WSD) belong together. A sense cannot be assigned correctly to a word inside an unrecognized idiom, and many MWEs are themselves ambiguous or can be used literally. To propose MWE candidates, the authors pair a WordNet-derived lexicon with a sequence of hand-crafted rules modeled on the approach of Kulkarni and Finlayson (2011). A trainable gloss-aware model then keeps or discards each candidate by checking whether any sense definition fits the context, and the same model also performs ordinary WSD. Building on BERT Bi-encoders for WSD, they introduce a Poly-encoder variant called distinct codes attention (DCA), which uses separate learned codes for target tokens and surrounding context. The system mainly gains precision. It sets a new state of the art on DiMSUM and is competitive on PARSEME 1.1 English, while losing relatively little WSD quality. This shows that glosses, which had not previously been used for MWE identification, can support both tasks within one model.

Key points

  • On DiMSUM, MWE identification improves on the previous best by up to 1.9 F1; results on PARSEME 1.1 English are competitive.
  • In joint use, WSD F1 is only about 6% below an equivalent single-task model.
  • The DCA Poly-encoder clearly beats the standard Poly-encoder on both datasets and outperforms the Bi-encoder on PARSEME.
  • The error analysis covered 400 cases from SemCor-trained and fine-tuned models:
    • Over 80% of false positives had a fitting lexicon sense but were not annotated as MWEs, which points to inconsistent MWE definitions between WordNet and the datasets.
    • Over 85% of false negatives were MWEs missing from the lexicon.
  • Recall is limited by lexicon coverage, and the method requires glosses, which makes it hard to apply to low-resource languages. Code, data and models are available at github.com/Mindful/MWEasWSD.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Word sense disambiguation

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-emnlp.14

@inproceedings{tanner2023mwe,
  title = {{MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation}},
  author = {Tanner, Joshua and Hoffman, Jacob},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.14},
  url = {https://doi.org/10.18653/v1/2023.findings-emnlp.14}
}

References

40 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Handbook of Natural Language Processing. find
  2. Edoardo Barba, Tommaso Pasini, and Roberto Navigli. 2021a. ESC: Redesigning WSD with extractive sense comprehension. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4661–4672, Online. Association… find
  3. Edoardo Barba, Luigi Procopio, and Roberto Navigli. 2021b. ConSeC: Word sense disambiguation as continuous sense comprehension. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1492–1503, Online and Punta Cana, Dominican Republic. Association for Compu… find
  4. Michele Bevilacqua, Marco Maru, and Roberto Navigli. 2020. Generationary or “how we went beyond word sense inventories and learned to gloss”. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7207–7221, Online. Association for Computational Ling… find our card
  5. Vasiliki Foufi, Luka Nerima, and Éric Wehrli. 2017. Parsing and MWE detection: Fips at the PARSEME shared task. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 54– 59, Valencia, Spain. Association for Computational Linguistics. find
  6. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In International Conference on Learning Representations. find
  7. Kamil Kanclerz and Maciej Piasecki. 2022. Deep neural representations for multiword expressions detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 444–453, Dublin, Ireland. Association for Computational Linguistics. find
  8. Minju Kim, Chaehyeong Kim, Yong Ho Song, Seung-won Hwang, and Jinyoung Yeo. 2022. BotsTalk: Machine-sourced framework for automatic curation of large-scale multi-skill dialogue datasets. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5149– 5170, Abu… find
  9. Angelika Kirilin, Felix Krauss, and Yannick Versley. 2016. ICL-HD at SemEval-2016 task 10: Improving the detection of minimal semantic units and their meanings with an ontology and word embeddings. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 937– 94… find
  10. Harsh Kohli. 2021. Training bi-encoders for word sense disambiguation. CoRR, abs/2105.10146. — [arXiv:2105.10146](https://arxiv.org/abs/2105.10146) find
  11. Nidhi Kulkarni and Mark Finlayson. 2011. jMWE: A Java toolkit for detecting multi-word expressions. In Proceedings of the Workshop on Multiword Expressions: from Parsing and Generation to the Real World, pages 122–124, Portland, Oregon, USA. Association for Computational Linguistics. find
  12. Murathan Kurfalı and Robert Östling. 2020. Disambiguation of potentially idiomatic expressions with contextual embeddings. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 85–94, online. Association for Computational Linguistics. find
  13. Zichao Li, Prakhar Sharma, Xing Han Lu, Jackie Cheung, and Siva Reddy. 2022. Using interactive feedback to improve the accuracy and explainability of question answering systems post-deployment. In Findings of the Association for Computational Linguistics: ACL 2022, pages 926–937, Dublin, Ireland. As… find
  14. Nelson F. Liu, Daniel Hershcovich, Michael Kranzlein, and Nathan Schneider. 2021. Lexical semantic recognition. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), pages 49–56, Online. Association for Computational Linguistics. find
  15. Marco Maru, Simone Conia, Michele Bevilacqua, and Roberto Navigli. 2022. Nibbling at the hard core of Word Sense Disambiguation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4724–4737, Dublin, Ireland. Association for Compu… find
  16. Marek Maziarz, Stan Szpakowicz, and Maciej Piasecki. 2015. A procedural definition of multi-word lexical units. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 427–435, Hissar, Bulgaria. INCOMA Ltd. Shoumen, BULGARIA. find
  17. George A. Miller. 1995. Wordnet: A lexical database for english. Communications of the ACM, 38(1):39– 41. find our card
  18. George A. Miller, Claudia Leacock, Randee Tengi, and Ross T. Bunker. 1993. A semantic concordance. In Human Language Technology: Proceedings of a Workshop Held at Plainsboro, New Jersey, March 21-24, 1993. find
  19. Ines Montani and Matthew Honnibal. Prodigy: A modern and scriptable annotation tool for creating training data for machine learning models. find
  20. Andrea Moro and Roberto Navigli. 2015. SemEval-2015 task 13: Multilingual all-words sense disambiguation and entity linking. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 288–297, Denver, Colorado. Association for Computational Linguistics. find
  21. Roberto Navigli, David Jurgens, and Daniele Vannella. 2013. SemEval-2013 task 12: Multilingual word sense disambiguation. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages… find
  22. Martha Palmer, Christiane Fellbaum, Scott Cotton, Lauren Delfs, and Hoa Trang Dang. 2001. English tasks: All-words and verb lexical sample. In Proceedings of SENSEVAL-2 Second International Workshop on Evaluating Word Sense Disambiguation Systems, pages 21–24, Toulouse, France. Association for Compu… find
  23. Caroline Pasquer, Agata Savary, Carlos Ramisch, and Jean-Yves Antoine. 2020. Verbal multiword expression identification: Do we need a sledgehammer to crack a nut? In Proceedings of the 28th International Conference on Computational Linguistics, pages 3333–3345, Barcelona, Spain (Online). Internation… find
  24. Sameer Pradhan, Edward Loper, Dmitriy Dligach, and Martha Palmer. 2007. SemEval-2007 task-17: English lexical sample, SRL and all words. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 87–92, Prague, Czech Republic. Association for Computational Ling… find
  25. Manuel F. Pulido. 2022. Why are multiword units hard to acquire for late l2 learners? insights from cognitive science on adult learning, processing, and retrieval. Linguistics Vanguard, 8:237 – 247. find
  26. Alessandro Raganato, Jose Camacho-Collados, and Roberto Navigli. 2017. Word sense disambiguation: A unified evaluation framework and empirical comparison. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 99–1… find our card
  27. Carlos Ramisch, Silvio Ricardo Cordeiro, Agata Savary, Veronika Vincze, Verginica Barbu Mititelu, Archna Bhatia, Maja Buljan, Marie Candito, Polona Gantar, Voula Giouli, Tunga Güngör, Abdelati Hawwari, Uxoa Iñurrieta, Jolanta Kovalevskaitė, Simon Krek, Timm Lichte, Chaya Liebeskind, Johanna Monti,… find our card
  28. Carlos Ramisch, Aline Villavicencio, and Christian Boitet. 2010. mwetoolkit: a framework for multiword expression identification. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA). find
  29. Omid Rohanian, Shiva Taslimipoor, Samaneh Kouchaki, Le An Ha, and Ruslan Mitkov. 2019. Bridging the gap: Attending to discontinuity in identification of multiword expressions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human… find
  30. Ivan Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for nlp. pages 1–15. find our card
  31. Agata Savary, Silvio Cordeiro, and Carlos Ramisch. 2019. Without lexicons, multiword expression identification will never fly: A position statement. In Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019), pages 79– 91, Florence, Italy. Association for Computational L… find
  32. Nathan Schneider, Dirk Hovy, Anders Johannsen, and Marine Carpuat. 2016. SemEval-2016 task 10: Detecting minimal semantic units and their meanings (DiMSUM). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 546–559, San Diego, California. Association for… find our card
  33. Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Jakob Prange, Austin Blodgett, Sarah R. Moeller, Aviram Stern, Adi Bitan, and Omri Abend. 2018. Comprehensive supersense disambiguation of English prepositions and possessives. In Proceedings of the 56th Annual Meeting of the Association for Computati… find
  34. Benjamin Snyder and Martha Palmer. 2004. The English all-words task. In Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, pages 41–43, Barcelona, Spain. Association for Computational Linguistics. find
  35. Yang Song, Xin Cai Ong, Hwee Tou Ng, and Qian Lin. 2021. Improved word sense disambiguation with enhanced sense representations. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4311–4320, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  36. Shiva Taslimipoor, Omid Rohanian, and Le An Ha. 2019. Cross-lingual transfer learning and multitask learning for capturing multiword expressions. In Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019), pages 155– 161, Florence, Italy. Association for Computational Li… find
  37. Jake Williams. 2017. Boundary-based MWE segmentation with text partitioning. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 1–10, Copenhagen, Denmark. Association for Computational Linguistics. find
  38. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  39. Andrea Zaninello and Alexandra Birch. 2020. Multiword expression aware neural machine translation. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3816–3825, Marseille, France. European Language Resources Association. find
  40. Guobiao Zhang, Wenpeng Lu, Xueping Peng, Shoujin Wang, Baoshuo Kan, and Rui Yu. 2022. Word sense disambiguation with knowledge-enhanced and local self-attention-based extractive sense comprehension. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4061– 4070, G… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/tanner-2023-mwe-as-wsd-solving-multiword/