Elamur Research⌕
☰
home / papers / 2025 / Data Augmentation for Maltese NLP using Transliterated and Machine Translated Arabic Data

Data Augmentation for Maltese NLP using Transliterated and Machine Translated Arabic Data

Paper · 2025 · Kurt Micallef, Nizar Habash and Claudia Borg

Summary

Maltese is a low-resource Semitic language written in Latin script. Its spelling and vocabulary have drifted far from Arabic, so Arabic NLP data cannot be fed to Maltese models directly. The authors test whether Arabic labelled data can still help Maltese models when it is first brought closer to Maltese. They compare five forms of the Arabic training data:

  • the original Arabic script,
  • two generic romanisations (Buckwalter, Uroman),
  • two new Arabic-to-Maltese transliterators they release (CharTx, character rules; MorphTx, rules guided by a morphological disambiguator),
  • Google Translate output into Maltese.

Each model is fine-tuned on Arabic first and then on a small Maltese set. The tasks are NER and sentiment analysis, and the models are BERTu, mBERT and mBERTu.

The main result is that the best way to present the Arabic data depends on how much Maltese the model already knows:

  • BERTu (Maltese-only) gains most from Maltese-looking input, either transliterated or machine-translated.
  • mBERT and mBERTu (multilingual) already link to raw Arabic and do best when several data versions are chained in sequential fine-tuning stages.

This matters for any low-resource language with a high-resource relative written in another script.

Key points

  • Data sizes: NER used ANERCorp (3,973 Arabic training sentences) with MAPA for Maltese, downsampled to 155 sentences. Sentiment used 15,305 Arabic sentences (neutral removed) and 595 Maltese sentences. Validation and test sets were always Maltese.
  • MorphTx design:
    • It relies on CAMeL Tools with the Egyptian CALIMA model, chosen because dialectal Arabic is closer to Maltese than MSA; it falls back to MSA when Egyptian analysis fails.
    • Morpheme-level rules override character rules (e.g., the article becomes "il-").
    • It applies Maltese conventions such as sun-letter assimilation and the fi + il- → fil- contraction.
  • BERTu results:
    • NER: from 58.3 (Maltese only) to 61.3 with MorphTx, and to 67.3 with MT.
    • Sentiment: from 82.5 to 85.7 with CharTx.
    • Raw Arabic script actually hurt it on NER (55.7).
  • Multilingual results with staged fine-tuning on NER:
    • mBERTu: 64.9 → 70.2 with original then transliterated data.
    • mBERT: 50.1 → 61.8 with the full original + transliteration + MT chain.
  • Tokeniser fertility: Raw Arabic is split into almost 4 subwords per word by BERTu's tokeniser. The new transliterators and MT lower this. For mBERT and mBERTu the trend is reversed.

Interesting ideas

  • I1. Rules on predicted morphemes: Transliteration rules are applied to morphemes from a contextual analyser, with affix-specific rules overriding letter rules, so the output follows target-language conventions rather than raw characters.
  • I2. Fertility as a cheap diagnostic: Measuring how finely a tokeniser fragments each input variant predicts which augmentation a given model can actually use, before any training.
  • I3. Staged fine-tuning across data versions: Fine-tuning on raw, then transliterated, then translated versions of the same labelled data in sequence beat any single version for multilingual models.
  • I4. Baseline-model-dependent conclusions: The same augmentation helped a monolingual model and hurt a multilingual one, so a data trick should always be judged per model.
  • I5. Pruning target-alien outputs: The transliterator deliberately never produces letters that mostly occur in loanwords and picks the most frequent mapping for ambiguous ones. This favours consistent output over theoretical coverage.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A morphology layer could drive normalisation of spelling variants or archaic forms through analysed affixes instead of string patterns, which keeps each rewrite traceable to a morpheme.
  • I2 (learn): Before adding converted or foreign silver data to gparse training, check subword fertility under its tokeniser to filter out sources that will mostly look like noise.
  • I3 (tree): Treebanks converted from other UD dialects or annotation schemes could be used in ordered stages, from far to near, ending on native English UD, rather than mixed into one pool.
  • I4 (learn): Evaluate each new silver-data source separately on the perceptron parser and on gparse, with a native-data-only run as the negative control.
  • I5 (back): Generation and the text → graph → text compressor could likewise restrict surface choices to the most frequent native realisations, giving deterministic and checkable output.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates124 works · 4,709 citations🎓University of MaltaMsida · Malta32 works · 895 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Kurt Micallef, Nizar Habash and Claudia Borg. 2025. Data Augmentation for Maltese NLP using Transliterated and Machine Translated Arabic Data. Findings of the Association for Computational Linguistics: EMNLP 2025.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.findings-emnlp.1177

@inproceedings{micallef2025data,
  title = {{Data Augmentation for Maltese NLP using Transliterated and Machine Translated Arabic Data}},
  author = {Kurt Micallef, Nizar Habash and Claudia Borg},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2025},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.1177},
  url = {https://doi.org/10.18653/v1/2025.findings-emnlp.1177}
}

References

26 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Judit Ács. 2019. Exploring BERT’s vocabulary. Judit Ács’s blog (Accessed 2025-05-15). find
  2. Ramy Baly, Alaa Khaddaj, Hazem Hajj, Wassim El-Hajj, and Khaled Shaban. 2018. Arsentd-lev: A multi-topic corpus for target-based sentiment analysis in arabic levantine tweets. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources Association (ELRA). ↗
  3. Yassine Benajiba, Paolo Rosso, and José Miguel Benedí Ruiz. 2007. ANERsys: An arabic named entity recognition system based on maximum entropy. In Computational Linguistics and Intelligent Text Processing, pages 143–153, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  4. Tim Buckwalter. 2002. Buckwalter Arabic morphological analyzer version 1.0. Linguistic Data Consortium (LDC) catalog number LDC2002L49, ISBN 1-58563-257-0. ↗
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  6. Fadhl Eryani and Nizar Habash. 2021. Automatic Romanization of Arabic bibliographic records. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 213–218, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  7. Lucie Gianola, Ēriks Ajausks, Victoria Arranz, Chomicha Bendahman, Laurent Bié, Claudia Borg, Aleix Cerdà, Khalid Choukri, Montse Cuadros, Ona de Gibert, Hans Degroote, Elena Edelman, Thierry Etchegoyhen, Ángela Franco Torres, Mercedes García Hernandez, Aitor García Pablos, Albert Gatt, Cyril Grouin, Manuel Herranz, and 10 others. 2020. Automatic removal of identifying information in official eu languages for public administrations: The MAPA project. In Proceedings of the 33rd International Conference on Legal Knowledge and Information Systems (JURIX’20), pages 223–226. IOS Press. find
  8. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al-Shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh, and Hind Saddiki. 2018. Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  9. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for Egyptian Arabic. In Proceedings of the Twelfth Meeting of the Special Interest Group on Computational Morphology and Phonology, pages 1–9, Montréal, Canada. Association for Computational Linguistics.
  10. Nizar Y Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. ↗
  11. Ulf Hermjakob, Jonathan May, and Kevin Knight. 2018. Out-of-the-box universal Romanization tool uroman. In Proceedings of ACL 2018, System Demonstrations, pages 13–18, Melbourne, Australia. Association for Computational Linguistics. find
  12. Go Inoue, Salam Khalifa, and Nizar Habash. 2022. Morphosyntactic tagging with pre-trained language models for Arabic and its dialects. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1708–1719, Dublin, Ireland. Association for Computational Linguistics.
  13. Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020. From zero to hero: On the limitations of zero-shot language transfer with multilingual Transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4483–4499, Online. Association for Computational Linguistics.
  14. Yihong Liu, Chunlan Ma, Haotian Ye, and Hinrich Schütze. 2025. TransMI: A framework to create strong baselines from multilingual pretrained language models for transliterated data. In Proceedings of the 31st International Conference on Computational Linguistics, pages 469–495, Abu Dhabi, UAE. Association for Computational Linguistics. ↗
  15. Antonio Martínez-García, Toni Badia, and Jeremy Barnes. 2021. Evaluating morphological typology in zero-shot cross-lingual transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3136–3153, Online. Association for Computational Linguistics. find
  16. Kurt Micallef, Fadhl Eryani, Nizar Habash, Houda Bouamor, and Claudia Borg. 2023. Exploring the impact of transliteration on NLP performance: Treating Maltese as an Arabic dialect. In Proceedings of the Workshop on Computation and Written Language (CAWL 2023), pages 22–32, Toronto, Canada. Association for Computational Linguistics.
  17. Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, and Claudia Borg. 2022. Pre-training data quality and quantity for a low-resource language: New corpus and BERT models for Maltese. In Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing, pages 90–101, Hybrid. Association for Computational Linguistics.
  18. Kurt Micallef, Nizar Habash, Claudia Borg, Fadhl Eryani, and Houda Bouamor. 2024. Cross-lingual transfer from related languages: Treating low-resource Maltese as multilingual code-switching. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1014–1025, St. Julian’s, Malta. Association for Computational Linguistics.
  19. Benjamin Muller, Antonios Anastasopoulos, Benoît Sagot, and Djamé Seddah. 2021. When being unseen from mBERT is just the beginning: Handling new languages with multilingual language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 448–462, Online. Association for Computational Linguistics.
  20. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  21. Fabian David Schmidt, Ivan Vulić, and Goran Glavaš. 2022. Don‘t stop fine-tuning: On training regimes for few-shot cross-lingual transfer with multilingual language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10725–10742, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  22. Genta Winata, Shijie Wu, Mayank Kulkarni, Thamar Solorio, and Daniel Preotiuc-Pietro. 2022. Cross-lingual few-shot learning on unseen languages. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 777–791, Online only. Association for Computational Linguistics. find
  23. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  24. Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833–844, Hong Kong, China. Association for Computational Linguistics. ↗
  25. Shijie Wu and Mark Dredze. 2020. Are all languages created equal in multilingual BERT? In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120–130, Online. Association for Computational Linguistics.
  26. Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, and Hinrich Schütze. 2021. A closer look at few-shot crosslingual transfer: The choice of shots matters. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5751–5767, Online. Association for Computational Linguistics. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/micallef-2025-data-augmentation-maltese-nlp-transliterated/