Elamur Research⌕
☰
home / papers / 2025 / The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectness

The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectness

Paper · 2025 · Sanad Sha'ban and Nizar Habash

Summary

Arabic dialect NLP usually assigns texts discrete dialect labels. Newer work such as ALDi treats dialectness as a continuous value, but that value only measures how far text drifts from MSA. The authors add a second, complementary axis: the Arabic Generality Score (AGS). It rates, on a 0–1 scale, how widely a word is shared across dialects and MSA. Widespread dialectal words like "šwy∼" ('a little') can therefore score as general even though they are non-standard.

To build training labels, they annotate the parallel MADAR corpus with word-level AGS. The pipeline aligns dialect word pairs, compares them with an edit distance that is aware of etymology and phonology (using CODA spelling and CAPHI phonology), and smooths the resulting similarities. A CAMeL-BERT regression model then learns to predict AGS for words in context. Sentence scores come from the least general words.

On the multi-dialect MDID test set, the fine-tuned models achieve lower RMSE than a lexicon lookup and than state-of-the-art dialect-ID systems. Together, ALDi and AGS give a two-dimensional, interpretable map of Arabic dialect variation.

Key points

  • Data: MADAR is 26-way parallel, with 2,000 BTEC sentences in MSA plus 25 city dialects (MADAR-26). Five dialects (CAI, BEI, DOH, TUN, RAB) have 10,000 extra sentences each (MADAR-6). MADAR-CODA normalizes 2,000 of the MADAR-6 sentences.
  • The MADAR Lexicon covers 1,045 concepts across 25 dialects, with 47,466 dialectal entries carrying CODA and CAPHI annotations. The CAPHI table maps phonemes to CODA script.
  • Gold sentence-level AGS in MDID is the share of dialects (out of 11) in which a sentence is judged valid. A predicted sentence score is the geometric mean of its k=2 least-general words.
  • Baselines:
    • lookup in the MADAR-26 lexicon, with 0.5 assigned to unknown words;
    • B2BERT (0.5963 macro-F1 on MDID);
    • the NADI2024-baseline (0.4697 macro-F1).
    • The dialect-ID outputs were converted into AGS by averaging over predicted labels.
  • CAMeL-BERT trained on MADAR-26 achieved the best RMSE. The version trained on MADAR-6 came close, which suggests that a few geographically spread dialects already carry most of the generality signal.

Interesting ideas

  • I1. Instead of making one scale more precise, the authors add an independent axis (breadth of use vs. distance from the standard), so items can be told apart that a single score would merge.
  • I2. Spelling variants are first normalized to a conventional orthography, and only then compared with a phonology-aware distance, so surface spelling noise is not counted as lexical difference.
  • I3. A sentence's score is driven by its few least general words via a geometric mean, rather than by averaging all words, because one rare item is enough to localize a text.
  • I4. Existing multi-label validity judgments are turned into a continuous target (valid labels / total labels), and competing classifiers are pushed through the same conversion so that all systems are compared on one metric.
  • I5. A generality score per word enables swapping localized expressions for broadly understood alternatives in controlled generation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): For UD dialect conversion, constructions could be scored both by how far they deviate from standard UD and by how many treebanks share them, helping decide which conversions to apply by default.
  • I2 (tokens): Variant forms (British/American spelling, informal phonetic spellings) could be mapped to one canonical lemma before any comparison, keeping lemma statistics clean.
  • I3 (answer): A reader could flag a passage as hard or regional based on its two or three rarest lemmas, rather than average word frequency, when judging what a student needs explained.
  • I4 (learn): Evaluation could reuse existing categorical annotations as graded targets, and project rival systems' outputs into the same measure, giving fair baselines without new labeling.
  • I5 (back): When generating text from the meaning graph, Mova could prefer high-generality lexical choices for general audiences and keep regional words only on request.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Sanad Sha'ban and Nizar Habash. 2025. The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectness. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.1524

@inproceedings{shaban2025arabic,
  title = {{The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectness}},
  author = {Sanad Sha'ban and Nizar Habash},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1524},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1524}
}

References

28 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, El Moatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The fourth nuanced Arabic dialect identification shared task. In Proceedings of ArabicNLP 2023, pages 600– 613, Singapore (Hybrid). Association for Computational Linguistics.
  2. Muhammad Abdul-Mageed, Amr Keleg, AbdelRahim Elmadany, Chiyu Zhang, Injy Hamed, Walid Magdy, Houda Bouamor, and Nizar Habash. 2024. NADI 2024: The fifth nuanced Arabic dialect identification shared task. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 709–728, Bangkok, Thailand. Association for Computational Linguistics.
  3. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The first nuanced Arabic dialect identification shared task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  4. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021. NADI 2021: The second nuanced Arabic dialect identification shared task. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  5. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022. NADI 2022: The third nuanced Arabic dialect identification shared task. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 85–97, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  6. Bashar Alhafni, Sarah Al-Towaity, Ziyad Fawzy, Fatema Nassar, Fadhl Eryani, Houda Bouamor, and Nizar Habash. 2024. Exploiting dialect identification in automatic dialectal text normalization. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 42–54, Bangkok, Thailand. Association for Computational Linguistics.
  7. El-Said M. Badawi and Martin Hinds. 1986. A Dictionary of Egyptian Arabic. Librairie du Liban, Beirut. find
  8. Said M. Badawi. 1973. Mustawayāt al-çarabiyya al-muāşira fı̄ Mis.r. Dār al-Maçārif bi-Mis.r, Cairo. find
  9. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  10. Zi-Yi Dou and Graham Neubig. 2021. Word alignment by fine-tuning embeddings on parallel corpora. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2112–2128, Online. Association for Computational Linguistics. ↗
  11. Fadhl Eryani, Nizar Habash, Houda Bouamor, and Salam Khalifa. 2020. A spelling correction corpus for multiple Arabic dialects. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4130–4138, Marseille, France. European Language Resources Association.
  12. Ramy Eskander, Nizar Habash, Owen Rambow, and Nadi Tomeh. 2013. Processing spontaneous orthography. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 585–595, Atlanta, Georgia. find
  13. Charles A. Ferguson. 1959. Diglossia. Word, 15(2):325– 340. find
  14. Nizar Habash, Salam Khalifa, Fadhl Eryani, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh, and Hind Saddiki. 2018. Unified Guidelines and Resources for Arabic Dialect Orthography. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan.
  15. Nizar Habash, Owen Rambow, Mona Diab, and Reem Faraj. 2008. Guidelines for annotation of arabic dialectness. Proceedings of the LREC Workshop on HLT & NLP within the Arabic world. find
  16. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods, pages 15–22. Springer, Netherlands. ↗
  17. Injy Hamed, Ngoc Thang Vu, and Slim Abdennadher. 2020. ArzEn: A speech corpus for code-switched Egyptian Arabic-English. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4237–4246, Marseille, France. European Language Resources Association. find
  18. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92– 104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  19. Ana Iriarte Diez, Claudia Laaber, Nina Kampen, and Monserrat Fernández. 2023. What is white arabic? new labels in a changing arab world. Revista Española de Lingüística, 53:229–266. find
  20. Amr Keleg, Sharon Goldwater, and Walid Magdy. 2023. ALDi: Quantifying the Arabic level of dialectness of text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10597–10611, Singapore. Association for Computational Linguistics.
  21. Amr Keleg, Sharon Goldwater, and Walid Magdy. 2025. Revisiting common assumptions about Arabic dialects in NLP. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3309–3327, Vienna, Austria. Association for Computational Linguistics.
  22. Amr Keleg and Walid Magdy. 2023. Arabic dialect identification under scrutiny: Limitations of single-label classification. In Proceedings of ArabicNLP 2023, pages 385–398, Singapore (Hybrid). Association for Computational Linguistics.
  23. Katrin Kirchhoff, Owen Rambow, Nizar Habash, and Mona Diab. 2007. Semi-automatic error analysis for large-scale statistical machine translation. In Proceedings of Machine Translation Summit XI: Papers, Copenhagen, Denmark.
  24. Helene Olsen, Samia Touileb, and Erik Velldal. 2023. Arabic dialect identification: An in-depth error analysis on the MADAR parallel corpus. In Proceedings of ArabicNLP 2023, pages 370–384, Singapore (Hybrid). Association for Computational Linguistics.
  25. Hassan Sajjad, Ahmed Abdelali, Nadir Durrani, and Fahim Dalvi. 2020. AraBench: Benchmarking dialectal Arabic-English machine translation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5094–5107, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  26. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained Arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332– 1344, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  27. Omar F. Zaidan and Chris Callison-Burch. 2011. The Arabic online commentary dataset: an annotated dataset of informal Arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 37–41, Portland, Oregon, USA. Association for Computational Linguistics.
  28. Omar F. Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ban-2025-arabic-generality-score-another-dimension/