Elamur Research⌕
☰
home / papers / 2018 / Edition 1.1 of the PARSEME Shared Task on Automatic Identification of Verbal Multiword Expressions

Edition 1.1 of the PARSEME Shared Task on Automatic Identification of Verbal Multiword Expressions

Summary

Verbal multiword expressions (VMWEs) are idiomatic combinations headed by a verb, such as light verb constructions or verb-particle constructions. They are hard for NLP systems because their lexicalized parts can be split apart, inflected, or seldom seen in training data. This paper describes the 2018 round (edition 1.1) of the PARSEME shared task, where systems must locate VMWEs in running text.

After the 2017 campaign, contributors raised about 80 guideline issues, and the organizers used them to revise the multilingual annotation methodology. The category inventory was redesigned, and annotators now apply one unified decision tree instead of a two-stage identify-then-classify procedure. Annotated corpora were extended or newly built for 20 languages, and the evaluation gained phenomenon-specific measures.

Seventeen systems from 12 teams in 9 countries participated, well above the previous edition. Systems struggled most with VMWEs never observed in training. The authors argue that solving this will likely require large lexicons or semi/unsupervised methods. The resulting corpora give a consistent cross-lingual benchmark for VMWE research.

Key points

  • The revised typology has two universal categories: LVC (split into LVC.full and LVC.cause) and VID. It also has quasi-universal categories (IRV; VPC.full and VPC.semi; MVC), the Italian-specific LS.ICV, and an optional experimental IAV category.
  • Participation grew from 7 systems in 2017 to 17 in 2018, and 11 submissions covered 19 languages. The 1.0 edition had covered 18 languages.
  • Average F1-MWE gaps across systems:
    • about 50 points (45–56) for seen vs. unseen-in-train VMWEs
    • 27 points for multi-token vs. single-token VMWEs
    • 20 points for identical vs. variant-of-train VMWEs
    • 17 points for continuous vs. discontinuous VMWEs
  • Guideline issues, data and system outputs are public on GitLab (parseme/sharedtask-guidelines, parseme/sharedtask-data). The authors plan further work on the MVC guidelines and the IAV category, and want future editions to cover non-verbal MWEs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

75 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 67 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ramisch2018edition,
  title = {{Edition 1.1 of the PARSEME Shared Task on Automatic Identification of Verbal Multiword Expressions}},
  author = {Ramisch, Carlos and Cordeiro, Silvio Ricardo and Savary, Agata and Vincze, Veronika and Mititelu, Verginica Barbu and Bhatia, Archna and Buljan, Maja and Candito, Marie and Gantar, Polona and Giouli, Voula and Güngör, Tunga and Hawwari, Abdelati and Iñurrieta, Uxoa and Kovalevskaitė, Jolanta and Krek, Simon and Lichte, Timm and Liebeskind, Chaya and Monti, Johanna and Escartín, Carla Parra and QasemiZadeh, Behrang and Ramisch, Renata and Schneider, Nathan and Stoyanova, Ivelina and Vaidya, Ashwini and Walsh, Abigail},
  booktitle = {Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018)},
  year = {2018},
  url = {https://aclanthology.org/W18-4925/}
}

References

40 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hazem Al Saied, Matthieu Constant, and Marie Candito. 2017. The ATILF-LLF system for Parseme shared task: a transition-based verbal multiword expression tagger. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 127–132, Valencia, Spain, April. Association for Computation… find
  2. Maria Jesús Aranzabe, Aitziber Atutxa, Kepa Bengoetxea, Arantza Diaz de Ilarraza, Iakes Goenaga, Koldo Gojenola, and Larraitz Uria. 2015. Automatic conversion of the Basque dependency treebank to Universal Dependencies. In Markus Dickinsons, Erhard Hinrichs, Agnieszka Patejuk, and Adam Przepiórkowsk… find
  3. Špela Arhar Holdt, Vojko Gorjanc, and Simon Krek. 2007. FidaPLUS corpus of Slovenian: the new generation of the Slovenian reference corpus: its design and tools. In Proceedings of the Corpus Linguistics Conference, CL2007, Birmingham. find
  4. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of Natural Language Processing, Second Edition, pages 267–292. CRC Press, Taylor and Francis Group, Boca Raton, FL. ISBN 978-1420085921. find
  5. Riyaz Ahmad Bhat, Rajesh Bhatt, Annahita Farudi, Prescott Klassen, Bhuvana Narasimhan, Martha Palmer, Owen Rambow, Dipti Misra Sharma, Ashwini Vaidya, Sri Ramagurumurthy Vishnu, and Fei Xia, 2015. The Hindi/Urdu Treebank Project. Springer Press. find
  6. Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi,… find our card
  7. Tiberiu Boroş, Sonia Pipa, Verginica Barbu Mititelu, and Dan Tufiş. 2017. A data-driven approach to verbal multiword expression detection. PARSEME Shared Task system description paper. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 121–126, Valencia, Spain, April. A… find
  8. Marie Candito and Matthieu Constant. 2014. Strategies for contiguous multiword expression analysis and dependency parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 743–753, Baltimore, Maryland, June. Association for Com… find
  9. Marie Candito and Djamé Seddah. 2012. Le corpus sequoia : annotation syntaxique et exploitation pour l’adaptation d’analyseur par pont lexical. In Proceedings of TALN 2012 (in French), Grenoble, France, June. find
  10. Matthieu Constant and Joakim Nivre. 2016. A transition-based system for joint lexical and syntactic analysis. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 161–171, Berlin, Germany, August. Association for Computational Ling… find
  11. Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke van der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017. Multiword expression processing: A survey. Computational Linguistics, 43(4):837– 892. find
  12. Dóra Csendes, János Csirik, Tibor Gyimóthy, and András Kocsor. 2005. The Szeged TreeBank. In Václav Matousek, Pavel Mautner, and Tomás Pavelka, editors, Proceedings of the 8th International Conference on Text, Speech and Dialogue, TSD 2005, Lecture Notes in Computer Science, pages 123–132, Berlin /… find
  13. Jenny Rose Finkel and Christopher D. Manning. 2009. Joint parsing and named entity recognition. In HLT-NAACL, pages 326–334. The Association for Computational Linguistics. find
  14. Spence Green, Marie-Catherine de Marneffe, John Bauer, and Christopher D. Manning. 2011. Multiword expression identification with tree substitution grammars: A parsing tour de force with French. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 725–735,… find
  15. Spence Green, Marie-Catherine de Marneffe, and Christopher D. Manning. 2013. Parsing models for identifying multiword expressions. Computational Linguistics, 39(1):195–227. find
  16. Natalia Klyueva, Antoine Doucet, and Milan Straka. 2017. Neural networks for multi-word expression detection. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 60–65, Valencia, Spain, April. Association for Computational Linguistics. find
  17. Svetla Koeva, Ivelina Stoyanova, Svetlozara Leseva, Rositsa Dekova, Tsvetana Dimitrova, and Ekaterina Tarpomanova. 2012. The Bulgarian National Corpus: Theory and practice in corpus design. Journal of Language Modelling, 0(1):65–110. find
  18. Simon Krek, Kaja Dobrovoljc, Tomaž Erjavec, Sara Može, Nina Ledinek, Nanika Holz, Katja Zupan, Polona Gantar, and Taja Kuzman. 2017. Training corpus ssj500k 2.0. Slovenian language resource repository CLARIN.SI, http://hdl.handle.net/11356/1165. ↗
  19. Joseph Le Roux, Antoine Rozenknop, and Matthieu Constant. 2014. Syntactic parsing and compound recognition via dual decomposition: Application to French. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1875–1885, Dublin, Ireland,… find
  20. Verena Lyding, Egon Stemle, Claudia Borghetti, Marco Brunello, Sara Castagnoli, Felice Dell’Orletta, Henrik Dittmann, Alessandro Lenci, and Vito Pirrelli. 2014. The PAISÀ Corpus of Italian Web Texts. In Proceedings of the 9th Web as Corpus Workshop (WaC-9), pages 36–43, Gothenburg, Sweden, April. As… find
  21. Alfredo Maldonado, Lifeng Han, Erwan Moreau, Ashjan Alsulaimani, Koel Dutta Chowdhury, Carl Vogel, and Qun Liu. 2017. Detection of verbal multi-word expressions via conditional random fields with syntactic dependency features and semantic re-ranking. In Proceedings of the 13th Workshop on Multiword… find
  22. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló, and Jungmee Lee. 2013. Universal dependency annotation for multilingual parsing. In Proceedings of the 51… find
  23. Alexis Nasr, Carlos Ramisch, José Deulofeu, and André Valli. 2015. Joint dependency parsing and multiword expression tokenization. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Vol… find
  24. Luka Nerima, Vasiliki Foufi, and Eric Wehrli. 2017. Parsing and MWE detection: Fips at the PARSEME shared task. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 54–59, Valencia, Spain, April. Association for Computational Linguistics. find
  25. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: a multilingual treebank collection. In Nicoletta Calzolari, Khali… find our card
  26. Maciej Ogrodniczuk, Katarzyna Głowińska, Mateusz Kopeć, Agata Savary, and Magdalena Zawisławska. 2015. Coreference in Polish: Annotation, Resolution and Evaluation. Walter De Gruyter. find
  27. Adam Przepiórkowski, Mirosław Bańko, Rafał L. Górski, Barbara Lewandowska-Tomaszczyk, Marek Łaziński, and Piotr P˛ezik. 2011. National Corpus of Polish. In Zygmunt Vetulani, editor, Proceedings of the 5th Language & Technology Conference: Human Language Technologies as a Challenge for Computer Sci… find
  28. Behrang QasemiZadeh and Saeed Rahimi. 2006. Persian in MULTEXT-East framework. In Tapio Salakoski, Filip Ginter, Sampo Pyysalo, and Tapio Pahikkala, editors, Advances in Natural Language Processing, pages 541–551, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  29. Victoria Rosén, Gyri Smørdal Losnegaard, Koenraad De Smedt, Eduard Bejček, Agata Savary, Adam Przepiórkowski, Petya Osenova, and Verginica Barbu Mitetelu. 2015. A survey of multiword expressions in treebanks. In Proceedings of the 14th International Workshop on Treebanks & Linguistic Theories confe… find
  30. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword Expressions: A Pain in the Neck for NLP. In Proceedings of the 3rd International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2002), pages 1–15, Mexico City, Mexico. find our card
  31. Agata Savary, Manfred Sailer, Yannick Parmentier, Michael Rosner, Victoria Rosén, Adam Przepiórkowski, Cvetana Krstev, Veronika Vincze, Beata Wójtowicz, Gyri Smørdal Losnegaard, Carla Parra Escartín, Jakub Waszczuk, Matthieu Constant, Petya Osenova, and Federico Sangati. 2015. PARSEME – PARSing and… find
  32. Agata Savary, Carlos Ramisch, Silvio Cordeiro, Federico Sangati, Veronika Vincze, Behrang QasemiZadeh, Marie Candito, Fabienne Cap, Voula Giouli, Ivelina Stoyanova, and Antoine Doucet. 2017. The PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the 13… find our card
  33. Agata Savary, Marie Candito, Verginica Barbu Mititelu, Eduard Bejček, Fabienne Cap, Slavomír Čéplö, Silvio Ricardo Cordeiro, Gülşen Eryiğit, Voula Giouli, Maarten van Gompel, Yaakov HaCohen-Kerner, Jolanta Kovalevskaitė, Simon Krek, Chaya Liebes kind, Johanna Monti, Carla Parra Escartín, Lonnek… find
  34. Nathan Schneider, Dirk Hovy, Anders Johannsen, and Marine Carpuat. 2016. SemEval-2016 Task 10: Detecting Minimal Semantic Units and their Meanings (DiMSUM). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 546–559, San Diego, California, June. Associatio… find our card
  35. Katalin Ilona Simkó, Viktória Kovács, and Veronika Vincze. 2017. USzeged: Identifying verbal multiword expressions with POS tagging and parsing techniques. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 48–53, Valencia, Spain, April. Association for Computational Ling… find
  36. Mariona Taulé, Aina Peris, and Horacio Rodríguez. 2016. Iarg-AnCora: Spanish corpus annotated with implicit arguments. Language Resources and Evaluation, 50(3):549–584, Sep. find
  37. Veronika Vincze, János Zsibrita, and István Nagy T. 2013. Dependency parsing for identifying Hungarian light verb constructions. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 207–215, Nagoya, Japan, October. Asian Federation of Natural Language Proc… find
  38. Jakub Waszczuk, Agata Savary, and Yannick Parmentier. 2016. Promoting multiword expressions in A* TAG parsing. In Nicoletta Calzolari, Yuji Matsumoto, and Rashmi Prasad, editors, COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers,… find
  39. Eric Wehrli, Violeta Seretan, and Luka Nerima. 2010. Sentence analysis and collocation identification. In Proceedings of the Workshop on Multiword Expressions: from Theory to Applications (MWE 2010), pages 27–35, Beijing, China, August. Association for Computational Linguistics. find
  40. Eric Wehrli. 2014. The relevance of collocations for parsing. In Proceedings of the 10th Workshop on Multiword Expressions (MWE), pages 26–32, Gothenburg, Sweden, April. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ramisch-2018-edition-1-1-parseme-shared/