Elamur Research⌕
☰
home / papers / 2023 / AlphaMWE-Arabic: Arabic Edition of Multilingual Parallel Corpora with Multiword Expression Annotations

AlphaMWE-Arabic: Arabic Edition of Multilingual Parallel Corpora with Multiword Expression Annotations

Paper · 2023 · Najet Hadj Mohamed, Malak Rassem, Lifeng Han et al.

Summary

Parallel corpora that mark multiword expressions (MWEs) in both languages are rare, and none existed for English–Arabic. MWEs are idiomatic, ambiguous and non-compositional, so they are hard for machine translation. The authors extend the multilingual AlphaMWE resource with an Arabic edition. Following AlphaMWE, they start from about 750 English sentences of the PARSEME corpus that carry verbal MWE (VMWE) tags.

  • Modern Standard Arabic (MSA): they machine-translated the sentences, then had two native speakers post-edit and annotate them, each checking the other's work.
  • Tunisian and Egyptian Arabic: current MT handles these dialects too poorly, so they were translated by hand from scratch.

They compared Google MT and Systran Translate qualitatively. They scored Google output with the human-in-the-loop metric HOPE, after adding error categories, one of them MWE-specific. Only about a third of the evaluated sentences came out correct, and MWE-related problems carried a large share of the penalty weight. The resource supports MWE-aware translation research and Arabic lexicography work.

Key points

  • Source data: the PARSEME English corpus (Walsh et al., 2018) has 832 manually annotated VMWEs across 7,437 sentences. AlphaMWE takes about 750 VMWE-bearing sentences from it, split into five files of roughly 150 sentences each (aa–ae).
  • Three Arabic corpora: MSA (reported as 2,700 tokens), Tunisian and Egyptian Arabic. Two annotators (one Egyptian, one Tunisian) cross-validated the MSA post-edits and spent about 15+ hours each.
  • Google vs Systran: when Systran was wrong, its errors were severe, such as inventing unrelated context. Google's wrong outputs stayed partly sensible. Systran was better at translating entities such as "copyright".
  • HOPE results on file 'aa' (150 segments): 35% of segments correct, 44% with minor errors, 21% with major errors. The average was 3.03 penalty points per segment (455 points in total).
  • Error breakdown: Impact (IMP) was the largest category at 25% of penalty weight. The new "MWE Missed Chance" (MMC) took 17%, followed by Mistranslation and Style at 15% each.

Interesting ideas

  • I1. Add an explicit "missed MWE" category to the error taxonomy, so literal word-by-word renderings of idioms get counted separately instead of hiding among general mistranslations.
  • I2. Score each segment with penalty points weighted by severity, then sort segments into correct/minor/major buckets. This gives a more transparent picture than one averaged score.
  • I3. Use machine output as a draft where it is good enough, but switch to human translation from scratch where the tool is too weak to help. The paper did this for the dialects.
  • I4. Have a second annotator check the first annotator's edit, and plan to measure agreement separately on MWE and non-MWE material.
  • I5. Treat verbal MWEs, whose canonical form is headed by a verb (e.g. "take ... for granted"), as one predicate unit rather than a verb plus arguments.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): when generating or translating from the meaning graph, flag cases where an idiom node was spelled out compositionally, and report this as its own error class.
  • I2 (learn): evaluation could weight parse or frame errors by how badly they distort the meaning, and report the share of sentences in each severity bucket alongside LAS.
  • I3 (learn): set a quality threshold for when teacher or witness-parser output is worth correcting versus relabeling manually, per genre or construction.
  • I4 (learn): when agreeing silver data across witness parsers, compute agreement separately on idiom and phrasal-verb spans to see where consensus is unreliable.
  • I5 (frames): detect VMWEs before VerbNet role assignment, so "kick the bucket" produces a single dying event rather than a kicking frame with a bucket as patient.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of StuttgartStuttgart · Germany91 works · 46,376 citations🎓University of ManchesterManchester · United Kingdom87 works · 44,938 citations🎓University of SfaxSfax · Tunisia14 works · 231 citations🎓Université de ToursTours · France6 works · 311 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Semitic languages · 2,908 worksMultilingual & cross-lingual NLP · 828 worksIdioms & multiword expressions · 217 works

Cite

Cite as Najet Hadj Mohamed, Malak Rassem, Lifeng Han and Goran Nenadic. 2023. AlphaMWE-Arabic: Arabic Edition of Multilingual Parallel Corpora with Multiword Expression Annotations. Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mohamed2023alphamwe,
  title = {{AlphaMWE-Arabic: Arabic Edition of Multilingual Parallel Corpora with Multiword Expression Annotations}},
  author = {Najet Hadj Mohamed, Malak Rassem, Lifeng Han and Goran Nenadic},
  booktitle = {Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing},
  year = {2023},
  doi = {10.26615/978-954-452-092-2\_050},
  url = {https://doi.org/10.26615/978-954-452-092-2_050}
}

References

30 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of Natural Language Processing, Second Edition, pages 267–292. CRC Press, Taylor and Francis Group, Boca Raton, FL. ISBN 978-1420085921. find
  2. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. A multidialectal parallel corpus of arabic. In LREC, pages 1240–1245.
  3. Rahma Boujelbane, Mariem Ellouze Khemekhem, and Lamia Belguith Hadrich. 2013. Mapping rules for building a Tunisian dialect lexicon and generating corpora. In Proceedings of the Sixth International Joint Conference on Natural Language Processing. find
  4. Parthena Charalampidou and Serge Gladkoff. 2022. A case of application of a new human mt quality evaluation metric in the emt classroom. In New Trends in Translation and Technology (NeTTT) Conference, pages 161 – 165. find
  5. Yu Chen and Andreas Eisele. 2012. Multiun v2: Un documents with multilingual alignments. In LREC, pages 2500–2504. find
  6. Serge Gladkoff and Lifeng Han. 2022. HOPE: A task-oriented and human-centric evaluation framework using professional post-editing towards more effective MT evaluation. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 13–21, Marseille, France. European Language Resources Association. find
  7. Serge Gladkoff, Lifeng Han, and Goran Nenadic. 2023. Student’s t-distribution: On measuring the inter-rater reliability when the observations are scarce. ↗
  8. Yvette Graham, Barry Haddow, and Philipp Koehn. 2020. Statistical power and translationese in machine translation evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 72–81, Online. Association for Computational Linguistics.
  9. Najet Hadj Mohamed, Cherifa Ben Khelil, Agata Savary, Iskandar Keskes, Jean-Yves Antoine, and Lamia Hadrich-Belguith. 2022. Annotating verbal multiword expressions in Arabic: Assessing the validity of a multilingual annotation procedure. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1839–1848, Marseille, France. European Language Resources Association.
  10. Lifeng Han. 2022a. Hilmeme: A human-in-the-loop machine translation evaluation metric looking into multi-word expressions. ↗
  11. Lifeng Han. 2022b. An investigation into multiword expressions in machine translation. Ph.D. thesis, Dublin City University. find
  12. Lifeng Han, Gareth Jones, and Alan Smeaton. 2020a. AlphaMWE: Construction of multilingual parallel corpora with MWE annotations. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 44–57, online. Association for Computational Linguistics. find
  13. Lifeng Han, Gareth Jones, and Alan Smeaton. 2020b. MultiMWE: Building a multi-lingual multi-word expression (MWE) parallel corpora. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2970–2979, Marseille, France. European Language Resources Association. ↗
  14. Lifeng Han, Gareth Jones, Alan Smeaton, and Paolo Bolzoni. 2021. Chinese character decomposition for neural MT with multi-word expressions. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 336–344, Reykjavik, Iceland (Online). Linköping University Electronic Press, Sweden. find
  15. Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5:339–351. ↗
  16. Aleksandar Kovačević, Azad Dehghan, Michele Filannino, John A Keane, and Goran Nenadic. 2013. Combining rules and machine learning for extraction of temporal expressions and events from clinical narratives. Journal of the American Medical Informatics Association, 20(5):859–866. ↗
  17. Samuel Läubli, Rico Sennrich, and Martin Volk. 2018. Has machine translation achieved human parity? a case for document-level evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4791–4796, Brussels, Belgium. Association for Computational Linguistics.
  18. Mohamed Maamouri, Seth Kulick, and Ann Bies. 2006. Diacritization: A challenge to arabic treebank annotation and parsing. In Proceedings of the International Conference on the Challenge of Arabic for NLP/MT, pages 35–47. find
  19. Alfredo Maldonado, Lifeng Han, Erwan Moreau, Ashjan Alsulaimani, Koel Dutta Chowdhury, Carl Vogel, and Qun Liu. 2017. Detection of verbal multi-word expressions via conditional random fields with syntactic dependency features and semantic re-ranking. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 114–120, Valencia, Spain. Association for Computational Linguistics. find
  20. Stella Markantonatou, Carlos Ramisch, Agata Savary, and Veronika Vincze, editors. 2017. Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017). Association for Computational Linguistics, Valencia, Spain. find
  21. KHALED MILAD. 2022. Comparative evaluation of translation memory (tm) and machine translation (mt) systems in translation between arabic and english. In New Trends in Translation and Technology (NeTTT) Conference, pages 142–151. ↗
  22. Erwan Moreau, Ashjan Alsulaimani, Alfredo Maldonado, Lifeng Han, Carl Vogel, and Koel Dutta Chowdhury. 2018. Semantic reranking of CRF label sequences for verbal multiword expression identification. In Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop, pages 177 – 207. Language Science Press. find
  23. John Ortega, Atul Kr. Ojha, Katharina Kann, and Chao-Hong Liu, editors. 2021. Proceedings of the 4th Workshop on Technologies for MT of Low Resource Languages (LoResMT2021). Association for Machine Translation in the Americas, Virtual. find
  24. Carlos Ramisch, Silvio Ricardo Cordeiro, Agata Savary, Veronika Vincze, Verginica Barbu Mititelu, Archna Bhatia, Maja Buljan, Marie Candito, Polona Gantar, Voula Giouli, Tunga Güngör, Abdelati Hawwari, Uxoa Iñurrieta, Jolanta Kovalevskaitė, Simon Krek, Timm Lichte, Chaya Liebeskind, Johanna Monti, Carla Parra Escartín, Behrang QasemiZadeh, Renata Ramisch, Nathan Schneider, Ivelina Stoyanova, Ashwini Vaidya, and Abigail Walsh. 2018. Edition 1.1 of the PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pages 222–240, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  25. Carlos Ramisch, Agata Savary, Bruno Guillaume, Jakub Waszczuk, Marie Candito, Ashwini Vaidya, Verginica Barbu Mititelu, Archna Bhatia, Uxoa Iñurrieta, Voula Giouli, Tunga Güngör, Menghan Jiang, Timm Lichte, Chaya Liebeskind, Johanna Monti, Renata Ramisch, Sara Stymne, Abigail Walsh, and Hongzhi Xu. 2020. Edition 1.2 of the PARSEME shared task on semi-supervised identification of verbal multiword expressions. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 107–118, online. Association for Computational Linguistics.
  26. Ivan A Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for nlp. In Computational Linguistics and Intelligent Text Processing: Third International Conference, CICLing 2002 Mexico City, Mexico, February 17–23, 2002 Proceedings 3, pages 1–15. Springer.
  27. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Conference on Neural Information Processing System, pages 6000–6010.
  28. Salloum Wael and Habash Nizar. 2012. Elissa: A Dialectal to Standard Arabic Machine Translation System. . In In: Proc. 24th International. Conference on Computational Linguistics, COLING.
  29. Abigail Walsh, Claire Bonial, Kristina Geeraert, John Philip McCrae, Nathan Schneider, and Clarissa Somers. 2018. Constructing an annotated corpus of verbal mwes for english. In Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pages 193–200. find
  30. Michał Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen. 2016. The united nations parallel corpus v1. 0. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3530–3534.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mohamed-2023-alphamwe-arabic-arabic-edition-multilingual/