Elamur Research⌕
☰
home / papers / 2025 / Arabic Dialect Classification using RNNs, Transformers, and Large Language Models: A Comparative Analysis

Arabic Dialect Classification using RNNs, Transformers, and Large Language Models: A Comparative Analysis

Summary

The paper tackles automatic identification of Arabic dialects in noisy social-media text, where informality, code-switching and closeness between neighbouring varieties make the task hard. The authors use the QADI corpus of roughly 440,000 tweets covering 18 country-level dialects. They build one shared pipeline: text cleaning based on the QADI authors' script, splits that keep the original class proportions, tokenization, training, and evaluation. Three families of models are compared within it:

  • recurrent networks (LSTM, GRU) on AraVec 3.0 embeddings with self-attention;
  • fine-tuned Arabic and multilingual Transformers;
  • large language models queried through prompting.

The best system is MARBERTv2, a Transformer pre-trained on Arabic social media, with about 65% accuracy and 64% F1. The recurrent baselines reach only 52%. The authors stress that careful normalization of tweets matters a great deal. They also note that LLMs are limited by tokenizers and context size that are poorly suited to Arabic. The work is a useful reference point for dialect-aware chatbots, social media monitoring and Arabic accessibility tools.

Key points

  • Data: QADI, about 440k tweets, 18 dialects; Mauritania, Somalia, Djibouti and Comoros are absent for lack of data.
  • RNN models: two stacked LSTM or GRU layers on 100-dimensional AraVec 3.0 (Word2Vec CBOW) vectors, plus Keras self-attention, dropout and early stopping.
    • Both reach 52% accuracy.
    • GRU trains faster and uses less memory.
  • Transformers fine-tuned:
    • Models: AraBERT Base Twitter v02, E5-Multilingual-Large, MARBERTv2, CAMeLBERT-DA.
    • Training setup: AdamW, weight decay 0.01, learning rate 2e-5, warmup ratio 0.1, gradient accumulation, fp16/bf16.
    • MARBERTv2's learning rate was lowered to 1e-5 after it began overfitting after epoch 2.
  • Best result: MARBERTv2 with about 65% accuracy and 64% F1.
    • Its report shows macro F1 ≈ 0.62 against weighted F1 ≈ 0.64.
    • For comparison, a prior AraBERT result on QADI was 60.6% F1.
  • LLMs (Gemma 3, Qwen2.5) were tested through prompt engineering. Tokenizer and context limits were named the main bottleneck.

Interesting ideas

  • I1. Before modelling, noisy text is cleaned in fixed steps: URLs and user mentions become generic placeholders, letter variants are unified, diacritics are stripped, and broken word boundaries are repaired. This removes variation that carries no signal for the task.
  • I2. All model families run through one modular pipeline with identical preprocessing and splits, so differences in scores reflect the model and not the setup.
  • I3. Splits keep the original label proportions, and results are reported as both macro and weighted averages; the gap between the two exposes weak minority classes.
  • I4. Overfitting is handled with a lower learning rate once it appears early, and the best checkpoint is restored by validation loss rather than keeping the last epoch.
  • I5. A GRU matched the LSTM's accuracy while being cheaper, which shows the lighter recurrent cell can be enough when resources are tight.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A similar pre-tokenizer layer could map URLs, handles and emoji to typed placeholder tokens while keeping their original spans, so the tagger sees clean input and every substitution stays traceable.
  • I2 (learn): Witness parsers that vote on silver data could be benchmarked through one shared harness with frozen splits, making their agreement and disagreement directly comparable.
  • I3 (learn): Evaluations of gparse or the perceptron parser could report macro-averaged LAS per relation or per genre next to the overall figure, so that rare constructions which fail are not hidden.
  • I4 (learn): gparse training could adopt checkpoint restoration by validation loss as the default and log when the learning rate was adjusted, keeping training runs explainable.
  • I5 (tree): When a fast CPU fallback for gparse is needed, a GRU-based encoder is worth testing before heavier options.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Language models & pretraining Semitic languages

Cite

Cite as Omar A.Essameldin, Ali O.Elbeih, Wael H.Gomaa and Wael F.Elsersy. 2025. Arabic Dialect Classification using RNNs, Transformers, and Large Language Models: A Comparative Analysis. arXiv:2506.19753.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2506.19753 · arXiv 2506.19753

@misc{aessameldin2025arabic,
  title = {{Arabic Dialect Classification using RNNs, Transformers, and Large Language Models: A Comparative Analysis}},
  author = {Omar A.Essameldin, Ali O.Elbeih, Wael H.Gomaa and Wael F.Elsersy},
  year = {2025},
  doi = {10.48550/arxiv.2506.19753},
  eprint = {2506.19753},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2506.19753}
}

References

20 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Abdul-Mageed, A. Elmadany, and E. M. B. Nagoudi, “Marbert: Large-scale arabic language model pretraining for arabic nlp,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021, pp. 6740–6760. [Online]. Available: https://aclanthology.org/2021.emnlp-main.540/ ↗
  2. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423/ ↗ our card
  3. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019. [Online]. Available: https://arxiv.org/abs/1907.11692 ↗
  4. W. Antoun, F. Baly, and H. Hajj, “Arabert: Transformer-based model for arabic language understanding,” in LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, p. 9. our card
  5. W. Luo, F. Liu, Z. Liu, and D. Litman, “Automatic summarization of student course feedback,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, K. Knight, A. Nenkova, and O. Rambow, Eds. San Diego, California: Association for Computational Linguistics, Jun. 2016, pp. 80–85. [Online]. Available: https://aclanthology.org/N16-1010/ ↗
  6. O. F. Zaidan and C. Callison-Burch, “Arabic dialect identification,” Computational Linguistics, vol. 40, no. 1, pp. 171–202, 2014. our card
  7. A. Abdelali, H. Mubarak, Y. Samih, S. Hassan, and K. Darwish, “Arabic dialect identification in the wild,” arXiv preprint arXiv:2005.06557, 2020. ↗ our card
  8. A. Mohammed, Z. Jiangbin, and A. Murtadha, “A three-stage neural model for arabic dialect identification,” Computer Speech & Language, vol. 80, p. 101488, 2023. find
  9. H. Bouamor, N. Habash, M. Salameh, W. Zaghouani, O. Rambow, D. Abdulrahim, O. Obeid, S. Khalifa, F. Eryani, A. Erdmann et al., “The madar arabic dialect corpus and lexicon,” in Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018), 2018. find
  10. H. Bouamor, N. Habash, H. Sajjad, H. Mubarak, A. Abdelali, K. Darwish, and Y. Samih, “Overview of the nadi shared task: Nuanced arabic dialect identification,” in Proceedings of the Fifth Arabic Natural Language Processing Workshop, 2020, pp. 1–13. find
  11. W. M. Yafooz, “Enhancing arabic dialect detection on social media: A hybrid model with an attention mechanism,” Information, vol. 15, no. 6, p. 316, 2024. find
  12. H. Nayel, A. Hassan, M. Sobhi, and A. El-Sawy, “Machine learning-based approach for Arabic dialect identification,” in Proceedings of the Sixth Arabic Natural Language Processing Workshop, N. Habash, H. Bouamor, H. Hajj, W. Magdy, W. Zaghouani, F. Bougares, N. Tomeh, I. Abu Farha, and S. Touileb, Eds. Kyiv, Ukraine (Virtual): Association for Computational Linguistics, Apr. 2021, pp. 287–290. [Online]. Available: https://aclanthology.org/2021.wanlp-1.34/ ↗
  13. A. Elnagar, S. M. Yagi, A. B. Nassif, I. Shahin, and S. A. Salloum, “Systematic literature review of dialectal arabic: identification and detection,” IEEE Access, vol. 9, pp. 31 010–31 042, 2021. find
  14. H. Saleh, A. AlMohimeed, R. Hassan, M. M. Ibrahim, S. H. Alsamhi, M. R. Hassan, and S. Mostafa, “Advancing arabic dialect detection with hybrid stacked transformer models,” Frontiers in Human Neuroscience, 2025. find
  15. M. Soliman, K. Eisa, and S. El-Beltagy, “Aravec: A set of arabic word embedding models for use in arabic nlp,” in Proceedings of the Third Arabic Natural Language Processing Workshop (WANLP), 2017, pp. 9–15. [Online]. Available: https://aclanthology.org/W17-1301/ ↗
  16. L. Wang, N. Yang, X. Huang, L. Yang, R. Majumder, and F. Wei, “Multilingual e5 text embeddings: A technical report,” arXiv preprint arXiv:2402.05672, 2024. ↗ our card
  17. M. Abdul-Mageed, A. Elmadany, and E. M. B. Nagoudi, “ARBERT & MARBERT: Deep bidirectional transformers for Arabic,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics, Aug. 2021, pp. 7088–7105. [Online]. Available: https://aclanthology.org/2021.acl-long.551 ↗ our card
  18. G. Inoue, B. Alhafni, N. Baimukan, H. Bouamor, and N. Habash, “The interplay of variant, size, and task type in Arabic pre-trained language models,” in Proceedings of the Sixth Arabic Natural Language Processing Workshop. Kyiv, Ukraine (Online): Association for Computational Linguistics, Apr. 2021. our card
  19. G. Team, “Gemma 3,” 2025. [Online]. Available: https://goo.gle/Gemma3Report ↗
  20. Q. Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: https://qwenlm.github.io/blog/qwen2.5/ ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/a-essameldin-2025-arabic-dialect-classification-using-rnns/