Elamur Research⌕
☰
home / papers / 2026 / A Corpus-Based Investigation of Contemporary Arabic Dialects Using the SADA Corpus

A Corpus-Based Investigation of Contemporary Arabic Dialects Using the SADA Corpus

Paper · 2026 · Ghada Alfattni

Summary

Spoken Arabic varies widely across regions, but large multi-dialect speech corpora that allow systematic comparison across linguistic levels are still scarce. Most dialect benchmarks are built from written social-media text. This study does not build a new resource. It treats the SADA corpus, a large spoken Arabic dataset with speaker metadata, as a basis for comparing dialect groups with each other and with MSA. Segments are split into per-dialect subcorpora using the SpeakerDialect label, and the text is normalised and POS-tagged. The analysis then uses Sketch Engine-style methods:

  • frequency profiles
  • log-likelihood keywords
  • collocations and word sketches
  • trigram phraseology

Dialects are compared through Jaccard and cosine similarity, hierarchical clustering and heatmaps. The dialects turn out to share a large common core. They differ consistently in frequent discourse markers, evaluative expressions and recurring multiword formulae, and they group regionally. The work matters because it offers interpretable evidence of these differences for dialectology and for building dialect-aware NLP and speech systems.

Key points

  • Data: the SADA corpus (Alotaibi et al., 2024), filtered by SpeakerDialect. Segments marked multi-speaker or with an unknown dialect were dropped.
  • Table 1 lists 48 speakers and 168,687 segments across Egyptian, Hijazi, Iraqi, Janubi, Khaliji, Levantine, Maghrebi, MSA and Najdi. The subsets are very unbalanced: Najdi has 94,611 segments, Hijazi 36,170 and Khaliji 30,320, while Iraqi has only 2 and Maghrebi 41.
  • Preprocessing was done in Python with pandas, pyarabic and regex. Steps:
    • unify alif/hamza variants
    • strip diacritics
    • tokenise
    • POS-tag
  • Comparison combines set overlap (Jaccard) with frequency-vector similarity (cosine), followed by hierarchical clustering shown as heatmaps.
  • The main differences between dialects lie in discourse-pragmatic material (markers, evaluative language, formulae) rather than in the shared core vocabulary. Planned future work covers fuller morphosyntactic analysis, embeddings and richer use of sociolinguistic metadata.

Interesting ideas

  • I1. Remove data whose provenance label is mixed or unknown before any comparison, so that differences cannot be explained by contamination.
  • I2. Normalise orthographic variants before counting, because inconsistent spelling quietly distorts frequency and distributional measures.
  • I3. Use log-likelihood keyword comparison of each variety against a reference register to show what makes a subcorpus distinctive.
  • I4. Measure similarity in two complementary ways, a set overlap and a frequency-weighted cosine, and then cluster, so that the shared vocabulary and the usage profile are viewed separately.
  • I5. Treat recurrent trigrams and frequent discourse markers as the main carriers of variation in spoken register, beyond single content words.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver data or treebank sentences of uncertain source or register could be held out of cross-corpus statistics, keeping dialect or genre comparisons clean.
  • I2 (tokens): Typographic variants such as curly versus straight apostrophes and quotes, and spelling variants, could be unified before lemma and tag frequency statistics are computed in crate en.
  • I3 (tree): The same test on lemma–deprel pairs could reveal systematic annotation differences between UD treebanks, which would suggest targets for UD dialect conversion.
  • I4 (learn): Training and evaluation corpora could be compared this way to choose sensible negative controls and to explain domain-shift drops.
  • I5 (frames): A small inventory of multiword discourse and evaluative formulae could feed speech-act and irony cues in crate prag instead of relying on single tokens.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Umm al-Qura UniversityMakkah al Mukarramah · Saudi Arabia21 works · 274 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Ghada Alfattni. 2026. A Corpus-Based Investigation of Contemporary Arabic Dialects Using the SADA Corpus. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.abjadnlp-1.35

@inproceedings{alfattni2026corpus,
  title = {{A Corpus-Based Investigation of Contemporary Arabic Dialects Using the SADA Corpus}},
  author = {Ghada Alfattni},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.35},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.35}
}

References

26 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The first nuanced Arabic dialect identification shared task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  2. Agence France-Presse and Linguistic Data Consortium. 2007. Arabic gigaword third edition. find
  3. A. Ahmed, N. Ali, M. Alzubaidi, W. Zaghouani, A. Abd-Alrazaq, and M. Househ. 2022. Freely available arabic corpora: A scoping review. Computer Methods and Programs in Biomedicine Update, 2:100049. ↗
  4. Noura Al-Shenaifi and 1 others. 2024. Advancing ai-driven linguistic analysis: Developing and evaluating large-scale arabic dialectal corpora. Mathematics, 12(19):3120. find
  5. Yousef A. Alotaibi, Sameer Alsharhan, Faisal Alotaibi, Ahmed Alshehri, Mohammad Alghamdi, Mohammed Alqahtani, and Abdulrahman Almuhareb. 2024. Sada: A large-scale multi-dialect arabic speech corpus for linguistic and computational applications. In Proceedings of the 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11716–11720. IEEE. find
  6. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  7. G. Badaro, R. Baly, H. Hajj, N. Habash, and W. El-Hajj. 2014. A large scale arabic sentiment lexicon for arabic opinion mining. Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP).
  8. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken arabic dialect identification using phonotactic modeling.
  9. H. Bouamor, S. Hassan, and N. Habash. 2019. The madar shared task on arabic fine-grained dialect identification. Proceedings of the Fourth Arabic Natural Language Processing Workshop.
  10. Kareem Darwish. 2024. A panoramic survey of natural language processing in the arab world. Communications of the ACM. find
  11. K. Dukes, E. Atwell, and N. Habash. 2011. Supervised collaboration for syntactic annotation of quranic arabic. Language Resources and Evaluation, 47:33–62. ↗
  12. Ahmed Ebeid, Younes Samih, and Kareem Darwish. 2023. A comprehensive study of arabic text normalization techniques for nlp applications. In Proceedings of the 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7), pages 45–55, Dubrovnik, Croatia. Association for Computational Linguistics. find
  13. Mohamed I. Eldesouki, Younes Samih, Ahmed Abde-lalí, Mohammed Attia, Hamdy Mubarak, Kareem Darwish, and Laura Kallmeyer. 2017. Arabic multi-dialect segmentation: Bi-lstm-crf vs. svm. ↗
  14. Ibrahim Abu Farha and Walid Magdy. 2022. The effect of arabic dialect familiarity on data annotation.
  15. Charles A. Ferguson. 1959. Diglossia. Word, 15(2):325– 340. find
  16. Salima Harrat, Karima Meftouh, Mourad Abbas, Salma Jamoussi, Motaz Saad, and Kamel Smaıli. 2015. Cross-dialectal arabic processing. find
  17. Mohammad Ali Humayun, Hayati Yassin, and Pg Emeroylariffion Abas. 2023. Dialect classification using acoustic and linguistic features in arabic speech. Iaes International Journal of Artificial Intelligence (Ij-Ai). ↗
  18. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2016. Curras: An annotated corpus for the palestinian arabic dialect. Language Resources and Evaluation. ↗
  19. Salam Khalifa, Nizar Habash, Dana Abdulrahim, and Sara Hassan. 2016. A large scale corpus of gulf arabic.
  20. Adam Kilgarriff, Vít Baisa, Jan Bušta, Miloš Jakubíček, Vojtěch Kovář, Jan Michelfeit, Pavel Rychlý, and Vít Suchomel. 2014. The sketch engine: Ten years on. Lexicography, 1(1):7–36. find
  21. Shervin Malmasi and Marcos Zampieri. 2017. Arabic dialect identification using ivectors and asr transcripts.
  22. Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. 2021. QASR: QCRI aljazeera speech resource a large scale annotated Arabic speech corpus. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2274–2285, Online. Association for Computational Linguistics. ↗
  23. Omar Obeid, Go Inoue, and Muhammad Abdul-Mageed. 2023. A benchmark suite for arabic nlp: Dialectal, classical, and modern standard evaluation. Natural Language Engineering, 29(5):765–789. find
  24. Ossama Obeid, Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2019. ADIDA: Automatic dialect identification for Arabic. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 6–11, Minneapolis, Minnesota. Association for Computational Linguistics.
  25. Kees Versteegh. 2014. The Arabic Language. Edinburgh University Press. ↗
  26. Marcos Zampieri, Liling Tan, Nikola Ljubešić, and Jörg Tiedemann. 2014. A report on the DSL shared task 2014. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 58–67, Dublin, Ireland. Association for Computational Linguistics and Dublin City University. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alfattni-2026-corpus-based-investigation-contemporary-arabic/