Elamur Research⌕
☰
home / papers / 2023 / Exploring the Impact of Transliteration on NLP Performance: Treating Maltese as an Arabic Dialect

Exploring the Impact of Transliteration on NLP Performance: Treating Maltese as an Arabic Dialect

Paper · 2023 · Kurt Micallef, Fadhl Eryani, Nizar Habash et al.

Summary

Maltese is a low-resource Semitic language. Its grammar is largely Arabic, but it is written in Latin script and has absorbed many Italian and English words. Script mismatch is known to limit how well multilingual models transfer to related languages. The authors therefore test whether rewriting Maltese in Arabic script helps downstream NLP.

They build several Maltese-to-Arabic transliteration pipelines, each with two stages, mapping and ranking:

  • The simplest uses a deterministic one-to-one character map.
  • Richer variants add manually annotated word mappings for closed-class tokens.
  • Others produce several candidate spellings per word and rank them with n-gram models of Tunisian Arabic, the Arabic variety closest to Maltese.

The pipelines are evaluated on POS tagging, dependency parsing (MUDT) and sentiment analysis. The models compared are CAMeLBERT, mBERT and a monolingual Maltese BERT.

The richer pipelines consistently beat the simple map. On transliterated text, CAMeLBERT generally outperforms mBERT and is competitive with mBERT fine-tuned on raw Maltese. A blind human readability study agrees with these rankings. The work shows that cheap script conversion can open a related high-resource language's models to a low-resource one.

Key points

  • Tasks: POS tagging, dependency parsing and sentiment analysis. Data includes the Maltese UD treebank MUDT (Čéplö, 2018). Experiments use Large and Small training setups.

  • Models compared:

    • CAMeLBERT (Arabic) on transliterated text.
    • mBERT on raw and on transliterated text.
    • Monolingual Maltese BERT (Micallef et al., 2022).

  • Best pipeline: candidates ranked by a Tunisian Arabic word-level model, combined with the complete set of closed-class word mappings. Non-deterministic mapping with target-language ranking helps most in ambiguous cases. Closed-class word mappings help especially when training data is scarce.

  • Readability study: one native Tunisian Arabic speaker blind-ranked 4 systems on 50 MUDT sentences (1 = most readable).

    SystemAverage rank
    Word Model + Full1.1
    Deterministic + Full2.3
    Word Model + None2.5
    Deterministic + None4.0

  • Linguistic background:

    • Maltese has about 150 surface variants of the definite article, all mapping to Arabic Al.
    • Mutual intelligibility with Tunisian and Libyan Arabic is roughly 30–40%.
    • Italian-origin words and univerbations (e.g. waranofsinhar) remain hard to transliterate.

Interesting ideas

  • I1. Instead of committing to one output per character, generate every plausible rewrite and let a language model of the target variety pick the most natural one; this resolves ambiguity cheaply without parallel data.
  • I2. A small hand-annotated table for closed-class words (articles, prepositions, conjunctions and their fused forms) gives outsized gains, mostly in low-data settings.
  • I3. The pivot resource is chosen by measured linguistic closeness (Tunisian Arabic) rather than by size or prestige (MSA).
  • I4. The human evaluation hides system identities, shuffles output order, gives a reference translation, and then checks that the human rankings match the automatic downstream scores.
  • I5. Reporting results under both Large and Small training regimes shows where a trick genuinely matters, since some gains appear mainly with little data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Nonstandard or ambiguous spellings could be expanded into candidate forms and ranked by a word n-gram model before tagging, so the chosen normalization stays traceable.
  • I2 (tree): When converting between UD dialects or treebank conventions, a curated function-word mapping table could handle the highest-frequency cases explicitly before general rules apply.
  • I3 (learn): When picking which treebank or witness parser seeds silver data for a new genre, choose by measured similarity to the target text, not just corpus size.
  • I4 (back): Outputs of the text → graph → text compressor could be blind-ranked the same way, and agreement with automatic metrics would validate those metrics.
  • I5 (learn): Evaluating repair rules or teacher tiers at several training-set sizes would reveal which components are only crutches for data scarcity.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations🎓University of TübingenTübingen · Germany75 works · 41,081 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations🎓University of MaltaMsida · Malta29 works · 894 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Kurt Micallef, Fadhl Eryani, Nizar Habash, Houda Bouamor and Claudia Borg. 2023. Exploring the Impact of Transliteration on NLP Performance: Treating Maltese as an Arabic Dialect. Proceedings of the Workshop on Computation and Written Language (CAWL 2023).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.cawl-1.4

@inproceedings{micallef2023exploring,
  title = {{Exploring the Impact of Transliteration on NLP Performance: Treating Maltese as an Arabic Dialect}},
  author = {Kurt Micallef, Fadhl Eryani, Nizar Habash, Houda Bouamor and Claudia Borg},
  booktitle = {Proceedings of the Workshop on Computation and Written Language (CAWL 2023)},
  year = {2023},
  doi = {10.18653/v1/2023.cawl-1.4},
  url = {https://doi.org/10.18653/v1/2023.cawl-1.4}
}

References

33 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Albert Gatt and Slavomír Čéplö. 2013. Digital Corpora and Other Electronic Resources for Maltese. In Proceedings of the International Conference on Corpus Linguistics, pages 96–97. UCREL, Lancaster, UK. find
  2. Joseph Aquilina. 1987. Maltese-English Dictionary Vol. I, A-L. Midsea Books, Valletta, Malta. find
  3. Joseph Aquilina. 1990. Maltese-English Dictionary Vol. II, M-Z. Midsea Books, Valletta, Malta. find
  4. Kyle Gorman. 2016. Pynini: A Python library for weighted finite-state grammar compilation. In Proceedings of the SIGFSM Workshop on Statistical NLP and Weighted Automata, pages 75–80, Berlin, Germany. Association for Computational Linguistics. find
  5. Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2022. Hierarchical aggregation of dialectal data for Arabic dialect identification. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4586–4596, Marseille, France. European Language Resources Association.
  6. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al-Shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh, and Hind Saddiki. 2018. Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  7. Albert Borg and Marie Azzopardi-Alexander. 1997. Maltese: Descriptive Grammars. Routledge, London and New York. find
  8. Mark Borg, Keith Bugeja, Colin Vella, Gordon Mangion, and Carmel Gafà. 2014. Preparation of a free-running text corpus for Maltese concatenative speech synthesis. Perspectives on Maltese Linguistics, Studia Typologica, 14:297–318. Berlin:Akademie Verlag. ↗
  9. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods, pages 15–22. Springer, Netherlands. ↗
  10. Slavomír Čéplö. 2018. Constituent order in Maltese: A quantitative analysis. Ph.D. thesis, Charles University, Prague. find
  11. Slavomír Čéplö, Ján Bátora, Adam Benkato, Jiří Milička, Christophe Pereira, and Petr Zemánek. 2016. Mutual Intelligibility of Spoken Maltese, Libyan Arabic, and Tunisian Arabic Functionally Tested: A Pilot Study. Folia Linguistica, 50(2):583–628. ↗
  12. Nizar Y Habash. 2010. Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
  13. Injy Hamed, Ngoc Thang Vu, and Slim Abdennadher. 2020. ArzEn: A speech corpus for code-switched Egyptian Arabic-English. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4237–4246, Marseille, France. European Language Resources Association. find
  14. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. ↗
  15. Kenneth Heafield. 2011. KenLM: Faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187–197, Edinburgh, Scotland. Association for Computational Linguistics.
  16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  17. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92– 104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  18. Ray Fabri, Michael Gasser, Nizar Habash, George Kiraz, and Shuly Wintner. 2014. Linguistic Introduction: The Orthography, Morphology and Syntax of Semitic Languages, pages 3–41. Springer Berlin Heidelberg, Berlin, Heidelberg. find
  19. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing Universal Dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–2795, Hong Kong, China. Association for Computational Linguistics. ↗
  20. Charles A. Ferguson. 1959. Diglossia. Word, 15(2):325– 340. find
  21. Antonio Martínez-García, Toni Badia, and Jeremy Barnes. 2021. Evaluating morphological typology David M. Eberhard, Gary F. Simons, and Charles D. Fennig. 2022. 25 edition. SIL International. [link]. in zero-shot cross-lingual transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3136–3153, Online. Association for Computational Linguistics. find
  22. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. ↗
  23. Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, and Claudia Borg. 2022. Pre-training data quality and quantity for a low-resource language: New corpus and BERT models for Maltese. In Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing, pages 90–101, Hybrid. Association for Computational Linguistics.
  24. Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833–844, Hong Kong, China. Association for Computational Linguistics. ↗
  25. Benjamin Muller, Antonios Anastasopoulos, Benoît Sagot, and Djamé Seddah. 2021. When being unseen from mBERT is just the beginning: Handling new languages with multilingual language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 448–462, Online. Association for Computational Linguistics.
  26. Shijie Wu and Mark Dredze. 2020. Are all languages created equal in multilingual BERT? In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120–130, Online. Association for Computational Linguistics.
  27. Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers. 2017. Universal Dependencies. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts, Valencia, Spain. Association for Computational Linguistics. find
  28. Inès Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Belguith, and Nizar Habash. 2014. A conventional orthography for Tunisian Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2355–2361, Reykjavik, Iceland. European Language Resources Association (ELRA).
  29. Mike Rosner and Claudia Borg. 2022. Report on the Maltese Language. Language Technology Support of Europe’s Languages in 2020/2021. Maria Giagkou, Stelios Piperidis, Georg Rehm, Jane Dunne (Series Editors). Available online at https://european-language-equality.eu/deliverables/. ↗
  30. Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. How good is your tokenizer? on the monolingual performance of multilingual language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3118–3135, Online. Association for Computational Linguistics.
  31. Edmund F. Sutcliffe. 1936. A grammar of the Maltese language, with chrestomathy and vocabulary. Oxford University Press: Humphrey Milford, London. find
  32. Dima Taji, Nizar Habash, and Daniel Zeman. 2017. Universal dependencies for Arabic. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), Valencia, Spain.
  33. Mary Ann Walter. 2006. Pharyngealization effects in Maltese Arabic. In Perspectives on Arabic Linguistics: Papers from the annual symposium on Arabic linguistics. Volume XVI:, Cambridge, March 2002, volume 266, pages 161–178. John Benjamins Publishing. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/micallef-2023-exploring-impact-transliteration-nlp-performance/