Elamur Research⌕
☰
home / papers / 2025 / Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation

Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation

Paper · 2025 · Hassan Barmandah

Summary

Open Arabic LLMs are trained mostly on Modern Standard Arabic (MSA), so they tend to answer in a formal pan-Arabic register. Saudi varieties such as Hijazi and Najdi end up poorly served. The author builds a private, synthetic Saudi Dialect Instruction dataset of 5,466 instruction–response pairs, split evenly between Hijazi and Najdi. With it, ALLaM-7B-Instruct-preview is adapted through LoRA. Two training regimes are compared: one marks each instruction with an explicit dialect tag, and the other leaves the tag out so the model must infer the dialect. Evaluation uses a held-out test set and three kinds of measurement: an external dialect classifier (MARBERTv2, with GLF as the proxy for Saudi), chrF++ and BERTScore for fidelity, and diversity measures. The tagged variant gives the strongest control. Its Saudi-classified share rises to about 84%, and outputs judged as MSA drop to about 6%. Both LoRA variants beat several larger or generic instruction models. The data and weights stay private, but code and a datasheet are published so others can check the evaluation.

Key points

  • Data: 5,466 synthetic instruction–response pairs, exactly 50/50 Hijazi/Najdi. Entries with unknown or ambiguous dialect labels were dropped. Five everyday topics make up about 55% of the data: shopping, work, education, health and technology.
  • Dialect-Token model: Saudi rate goes from 47.97% to 84.21%, and MSA leakage falls from 32.63% to 6.21%. chrF++ improves by 3.53 and BERTScore by 0.059.
  • Judge: MARBERTv2, an external classifier for written Arabic varieties that assigns each text to one of five classes (MAGHREB/LEV/MSA/GLF/EGY), with GLF standing in for Saudi usage.
  • Baselines beaten on dialect control and fidelity: Falcon-7B-Instruct, Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, AceGPT-v2-8B-Chat and JAIS-13B-Chat. These baselines often repeat the metadata tags in their output.
  • Release: training, evaluation and inference code plus a datasheet with schema and aggregate statistics. No dataset or adapters are released. Stated limitations are synthetic, single-turn data and small scale.

Interesting ideas

  • I1. Scoring outputs with a separate, independently trained classifier gives a clear "leakage" number: the share of outputs that fall into the unwanted class (here MSA).
  • I2. The paper treats repeating control tags in the output as a failure mode in its own right, separate from fluency.
  • I3. Strict label balance, with ambiguous items removed, keeps the label from being confounded with topic, so scores cannot simply reflect skewed representation.
  • I4. When data cannot be shared, publishing a detailed datasheet together with full evaluation code still lets others rerun the scoring pipeline independently.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • Putting a control tag such as <DIALECT=NAJDI> in front of the input, and ablating it against a tag-free run, isolates how much explicit conditioning contributes to variety control. — (back): when generating text from the meaning graph, a register or dialect tag could be passed explicitly. A tagged-vs-untagged comparison would then show whether the generator really follows it.
  • I1 (learn): evaluation could report how often outputs drift into a forbidden category. One example is the share of converted trees that still carry the source treebank's conventions, used as a negative control.
  • I2 (back): the text → graph → text compressor could get a check that no internal frame labels or role names appear in the regenerated surface text.
  • I3 (learn): silver data agreed by witness parsers could be balanced across genres or constructions before training. This way, gains do not just mirror the mix of the corpus.
  • I4 (learn): a datasheet and the evaluation harness could be shipped for any licensed or private treebank used, so results stay checkable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Umm al-Qura UniversityMakkah al Mukarramah · Saudi Arabia21 works · 274 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Hassan Barmandah. 2025. Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation. arXiv:2508.13525.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2508.13525 · arXiv 2508.13525

@misc{barmandah2025saudi,
  title = {{Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation}},
  author = {Hassan Barmandah},
  year = {2025},
  doi = {10.48550/arxiv.2508.13525},
  eprint = {2508.13525},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2508.13525}
}

References

26 references, 15 of them with a card of ours. ↗ leads to the work.

  1. Meta AI, “Llama 3 model card,” 2024. [Online]. Available: https://github.com/meta-llama/llama3/blob/main/MODEL CARD.md ↗
  2. M. S. Bari, Y. Alnumay, N. A. Alzahrani, N. M. Alotaibi, H. A. Alyahya, S. AlRashed, F. A. Mirza, S. Z. Alsubaie, H. A. Alahmed, G. Alabduljabbar, R. Alkhathran, Y. Almushayqih, R. Alnajim, S. Alsubaihi, M. Al Mansour, S. A. Hassan, M. Alrubaian, A. Alammari, Z. Alawami, A. Al-Thubaity, A. Abdelali, J. Kuriakose, A. Abujabal, N. Al-Twairesh, A. Alowisheq, and H. Khan, “ALLam: Large language models for arabic and english,” in ICLR, 2025. [Online]. Available: https://openreview.net/forum?id=MscdsFVZrN ↗
  3. N. Sengupta, S. K. Sahu, B. Jia, S. Katipomu, H. Li, F. Koto, O. M. Afzal, S. Kamboj, O. Pandit, R. Pal, L. Pradhan, Z. M. Mujahid, M. Baali, A. F. Aji, Z. Liu, A. Hock, A. Feldman, J. Lee, A. Jackson, P. Nakov, T. Baldwin, and E. Xing, “Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models,” 2023. ↗
  4. W. Antoun, F. Baly, and H. Hajj, “AraGPT2: Pre-trained transformer for Arabic language generation,” in Proceedings of the Sixth Arabic Natural Language Processing Workshop. Kyiv, Ukraine (Virtual): Association for Computational Linguistics, Apr. 2021, pp. 196–207. [Online]. Available: https://www.aclweb.org/anthology/2021.wanlp-1.21 ↗
  5. A. El Filali, M. ALOUI, T. Husaain, A. Alzubaidi, B. E. A. Boussaha, R. Cojocaru, C. Fourrier, N. Habib, and H. Hacid, “Open arabic llm leaderboard 2,” https://huggingface.co/spaces/OALL/Open-Arabic-LLM-Leaderboard, 2025. ↗
  6. M. Abdul-Mageed, A. Elmadany, and E. M. B. Nagoudi, “Arbert & marbert: Deep bidirectional transformers for arabic,” in ACL, 2021. [Online]. Available: https://aclanthology.org/2021.acl-long.551/ ↗
  7. I. Amin, “Marbertv2 arabic written dialect classifier,” 2025. [Online]. Available: https://huggingface.co/IbrahimAmin/marbertv2-arabic-written-dialect-classifier ↗
  8. E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, E. Goffinet, D. Heslow, J. Launay, Q. Malartic, B. Noune, B. Pannier, and G. Penedo, “Falcon-40B: An open large language model with state-of-the-art performance,” 2023. [Online]. Available: https://huggingface.co/tiiuae/falcon-7b-instruct ↗
  9. Falcon-LLM Team, “Falcon-arabic: A breakthrough in arabic language models,” May 2025. [Online]. Available: https://falcon-lm.github.io/ blog/falcon-arabic ↗
  10. Qwen Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: https://qwenlm.github.io/blog/qwen2.5/ ↗
  11. J. Liang, Z. Cai, J. Zhu, H. Huang, K. Zong, B. An, M. Alharthi, J. He, L. Zhang, H. Li, B. Wang, and J. Xu, “Alignment at pre-training! towards native alignment for arabic llms,” in NeurIPS, 2024. [Online]. Available: https://openreview.net/forum?id=woRFmNJiLp ↗
  12. Asharq Al-Awsat, “Humane launches allam, the first arabic artificial intelligence model from saudi arabia, august 2025,” 2025. [Online]. Available: https://aawsat.com/%D8%A7%D9%84%D8%A7%D9%82% D8%AA%D8%B5%D8%A7%D8%AF/5174385-%D9%87%D9%8A% D9%88%D9%85%D8%A7%D9%8A%D9%86-%D8%A5%D8%B7% D9%84%D8%A7%D9%82-%D8%B9%D9%84%D9%91%D8%A7% D9%85-%D8%A3%D9%88%D9%84-%D9%86%D9%85%D9%88% D8%B0%D8%AC- %D8%B0%D9%83%D8%A7%D8%A1- %D8% A7%D8%B5%D8%B7%D9%86%D8%A7%D8%B9%D9%8A-%D8% B9%D8%B1%D8%A8%D9%8A-%D9%85%D9%86-%D8%A7%D9% 84%D8%B3%D8%B9%D9%88%D8%AF%D9%8A%D8%A9-%D8% A3%D9%88%D8%A7%D8%AE%D8%B1-%D8%A3%D8%BA%D8% B3%D8%B7%D8%B3 ↗
  13. F. Qarah, “Saudibert: A large language model pretrained on saudi dialect corpora,” arXiv preprint arXiv:2405.06239, 2024. ↗
  14. M. Abdul-Mageed, A. Elmadany, and E. M. B. Nagoudi, “Arbert & marbert: Deep bidirectional transformers for arabic,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). misc: Association for Computational Linguistics, Aug. 2021, pp. 7088–7105. [Online]. Available: https://aclanthology.org/2021.acl-long.551 ↗
  15. H. Chouikhi, M. Aloui, C. Ben Hammou, G. Chaabane, H. Kchaou, and C. Dhaouadi, “Gemmar: Enhancing llms through arabic instruction-tuning,” arXiv preprint arXiv:2407.02147, 2024. [Online]. Available: https://arxiv.org/abs/2407.02147 ↗
  16. ——, “Llamar & gemmar: Enhancing llms through arabic instruction-tuning,” 2024.
  17. H. Abu-Rayyash and N. Alanazi, “Saudial: The saudi arabic dialects game localization dataset,” 2025. [Online]. Available: https://data.mendeley.com/datasets/mzdwkb2t6d/2 ↗
  18. S. S. Alahmari, “SADSLyC: A corpus for saudi Arabian multi-dialect identification through song lyrics,” in Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4), S. Ezzini, H. Alami, I. Berrada, A. Benlahbib, A. El Mahdaouy, S. Lamsiyah, H. Derrouz, A. Haddad, M. Jarrar, M. El-Haj, R. Mitkov, and P. Rayson, Eds. Abu Dhabi, UAE: Association for Computational Linguistics, Jan. 2025, pp. 38–43. [Online]. Available: https://aclanthology.org/2025.wacl-1.4/ ↗
  19. National Center for Artificial Intelligence (SDAIA) and Saudi Broadcasting Authority, “Sada: Saudi audio dataset for arabic,” 2022, 667 hours of Saudi Arabic audio with transcripts across 57 TV programs. [Online]. Available: https://www.kaggle.com/datasets/ sdaiancai/sada2022 ↗
  20. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017, pp. 5998–6008. [Online]. Available: https://arxiv.org/abs/1706.03762 ↗
  21. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, L. Wang, and W. Wang, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021. [Online]. Available: https://doi.org/10.48550/arXiv.2106.09685 ↗
  22. A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” in International Conference on Learning Representations (ICLR), 2020. [Online]. Available: https://arxiv.org/abs/1904.09751 ↗
  23. M. Popović, “chrf++: words helping character n-grams,” in Proceedings of the Second Conference on Machine Translation (WMT). Copenhagen, Denmark: Association for Computational Linguistics, Sep. 2017, pp. 612–618. [Online]. Available: https://aclanthology.org/W17-4770/ ↗
  24. T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” in International Conference on Learning Representations (ICLR), 2020. [Online]. Available: https://arxiv.org/abs/1904.09675 ↗
  25. J. Li, M. Galley, C. Brockett, J. Gao, and B. Dolan, “A diversity-promoting objective function for neural conversation models,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). San Diego, California: Association for Computational Linguistics, 2016, pp. 110–119. [Online]. Available: https://aclanthology.org/N16-1014/ ↗
  26. Y. Zhu, S. Lu, L. Zheng, J. Guo, W. Zhang, J. Wang, and Y. Yu, “Texygen: A benchmarking platform for text generation models,” arXiv preprint arXiv:1802.01886, 2018. [Online]. Available: https://arxiv.org/abs/1802.01886 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/barmandah-2025-saudi-dialect-allam-lora-fine/