Elamur Research⌕
☰
home / papers / 2025 / Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models

Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models

Paper · 2025 · Abubakr Mohamed and Hamdy Mubarak

Summary

Written Modern Standard Arabic usually omits diacritics, the marks for short vowels and grammatical endings. As a result, one consonant skeleton can stand for several different words. The authors improve automatic restoration of these marks in three ways: better training data, better evaluation, and better models.

  • Training data: they audit a licensed 4.7M-word MSA corpus by counting how often each word appears with each vowel pattern. Patterns that disagree with a strongly dominant form go to a linguist for review.
  • Evaluation: they add a multi-reference version of the standard WikiNews-2014 test set, which accepts several valid readings. They also release a new expert-checked benchmark, WikiNews-2024.
  • Models: they compare several architectures and settle on a BiLSTM over a variable-length input. In this input, words whose vowelling barely depends on context arrive already diacritized, so the model only decides the ambiguous parts.

The model sets a new state of the art. A variant keeps any diacritics the user has already typed. Automatically diacritized clean text helps when training data is small. Together, these give the field cleaner data, fairer scoring and practical tools.

Key points

  • The training corpus has about 129K sentences and 4.7M words from the commercial provider RDI, and cannot be redistributed. The audit looked only at stems seen at least 50 times. Any form covering at least 95% of a stem's occurrences was treated as dominant.
  • The audit found two systematic problems: inconsistent marking of the alif-lam article (sun vs. moon letters) and missing diacritics on word-initial alif. Both were fixed by rule or after linguist review.
  • The BiLSTM model reaches 3.12% WER on WikiNews-2014 and 2.70% on WikiNews-2024. The best result previously reported on WikiNews was 4.49% WER (Mubarak et al., 2019a).
  • Two expert linguists reviewed WikiNews-2024. The error analysis groups 94% of errors into a few types: POS errors, noun–adjective agreement, conjunction parallelism, named entities, subject/object confusion, and two kinds of reference problems (HUM-MRef, HUM-Error).
  • The authors release a five-million-word Wikipedia corpus diacritized by their best model, together with their code and benchmarks.

Interesting ideas

  • I1. Check a corpus for internal consistency by listing every analysis each frequent stem receives and flagging rare variants that clash with a near-unanimous form, while ignoring the context-dependent ending. This finds silent annotation noise cheaply and works on any annotated dataset.
  • I2. Let the input mix whole tokens and single characters: words with a fixed answer come in already resolved, and only the uncertain pieces are left for the model. This shortens sequences and removes needless guessing.
  • I3. Treat any labels the user has already supplied as fixed constraints and predict only the rest. Partial human input is then never overwritten.
  • I4. Score against several references when more than one answer is legitimate. Single-reference scoring wrongly penalizes valid alternatives.
  • I5. In error analysis, separate gold-standard mistakes and missing alternatives (HUM-Error, HUM-MRef) from true system errors, then fix the benchmark itself.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Run the same audit on silver treebanks, comparing lemma, UPOS and feature choices per word form, so that suspicious minority annotations get reviewed before training.
  • I2 (tokens): Have the tagger and lemmatizer pre-fill words whose analysis is almost always the same and send only ambiguous items to the statistical model, recording which decisions were lexicon-based for explainability.
  • I3 (tree): Let gparse and the perceptron parser accept pre-specified arcs or labels from a user or an earlier stage and decode the remaining tree around them.
  • I4 (answer): Store sets of acceptable answers for school tests and fable QA, so that paraphrased or equally correct responses are not counted as failures.
  • I5 (learn): When evaluating parsers against treebanks, tag disagreements caused by gold errors or UD dialect variation separately, so that repair rules are not learned from bad references.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hamad bin Khalifa UniversityDoha · Qatar144 works · 14,175 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Abubakr Mohamed and Hamdy Mubarak. 2025. Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.846

@inproceedings{mohamed2025advancing,
  title = {{Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models}},
  author = {Abubakr Mohamed and Hamdy Mubarak},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.846},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.846}
}

References

22 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gheith A Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18:183–197. ↗
  2. Hanan Aldarmaki and Ahmad Ghannam. 2023. Diacritic recognition performance in arabic asr. arXiv preprint arXiv:2302.14022. ↗
  3. Sawsan Alqahtani, Ajay Mishra, and Mona Diab. 2020. A multitask learning approach for diacritic restoration. arXiv preprint arXiv:2006.04016. ↗
  4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  5. Yonatan Belinkov and James Glass. 2015. Arabic diacritization with recurrent neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281– 2285.
  6. Kareem Darwish, Hamdy Mubarak, and Ahmed Abdelali. 2017. Arabic diacritization: Stats, rules, and hacks. In Proceedings of the third Arabic natural language processing workshop, pages 9–17.
  7. Muhammad Morsy Elmallah, Mahmoud Reda, Kareem Darwish, Abdelrahman El-Sheikh, Ashraf Hatim El-neima, Murtadha Aljubran, Nouf Alsaeed, Reem Mohammed, and Mohamed Al-Badrashiny. 2024. Arabic diacritization using morphologically informed character-level model. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1446–1454.
  8. Moustafa Elshafei, Husni Al-Muhtaseb, and Mansour Alghamdi. 2006. Statistical methods for automatic diacritization of arabic text. In The Saudi 18th National Computer Conference. Riyadh, volume 18, pages 301–306. find
  9. Ya’akov Gal. 2002. An hmm approach to vowel restoration in arabic and hebrew. In Proceedings of the ACL-02 workshop on Computational approaches to semitic languages.
  10. Nawar Halabi. 2016. Modern standard Arabic phonetics for speech synthesis. Ph.D. thesis, University of Southampton. find
  11. Mohamed Maamouri et al. 2010. Arabic Treebank: Part 3 v3.2. Linguistic Data Consortium, Philadelphia, PA. LDC2010T08, Web Download. find
  12. Hamdy Mubarak, Ahmed Abdelali, Kareem Darwish, Mohamed Eldesouki, Younes Samih, and Hassan Sajjad. 2019a. A system for diacritizing four varieties of arabic. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 217–222.
  13. Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih, and Kareem Darwish. 2019b. Highly effective arabic diacritization using sequence to sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2390–2395.
  14. Rani Nelken and Stuart M. Shieber. 2005. Arabic diacritization using weighted finite-state transducers. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 79–86, Ann Arbor, Michigan. Association for Computational Linguistics.
  15. Arfath Pasha, Mohamed Al-Badrashiny, Mona T Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic. In Lrec, volume 14, pages 1094–1101.
  16. Mohsen AA Rashwan, Ahmad A Al Sallab, Hazem M Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. IEEE/ACM Transactions on Audio, Speech, And Language Processing, 23(3):505–516. ↗
  17. Ahmed Said, Mohamed El-Sharqwi, Achraf Chalabi, and Eslam Kamal. 2013. A hybrid approach for arabic diacritization. In International Conference on Application of Natural Language to Information Systems, pages 53–64. Springer. find
  18. Brian Thompson and Ali Alshehri. 2022. Improving Arabic diacritization by learning to diacritize and translate. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 11–21, Dublin, Ireland (in-person and online). Association for Computational Linguistics.
  19. Catalin Ungurean, Dragos Burileanu, Vladimir Popescu, Cristian Negrescu, and Aurelian Dervis. 2008. Automatic diacritic restoration for a tts-based e-mail reader application. UPB Scientific Bulletin, Series C, 70(4):3–12. find
  20. Dimitra Vergyri and Katrin Kirchhoff. 2004. Automatic diacritization of arabic for acoustic modeling in speech recognition. In Proceedings of the workshop on computational approaches to Arabic script-based languages, pages 66–73. ↗
  21. Nasser Zalmout and Nizar Habash. 2019. Adversarial multitask learning for joint multi-feature and multi-dialect morphological modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1775–1786, Florence, Italy. Association for Computational Linguistics.
  22. Imed Zitouni, Jeffrey Sorensen, and Ruhi Sarikaya. 2006. Maximum entropy based restoration of arabic diacritics. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 577–584.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mohamed-2025-advancing-arabic-diacritization-improved-datasets/