Elamur Research⌕
☰
home / papers / 2025 / Large Language Models and Arabic Content: A Review

Large Language Models and Arabic Content: A Review

Summary

Arabic is spoken by roughly 422 million people across 27 countries, yet it has few tools, datasets and benchmarks, and most large language model research is in English. This review surveys how pre-trained and large language models have been applied to Arabic.

It first explains why the language is hard for machines:

  • root-and-pattern morphology with derivation, inflection and agglutination;
  • a script with no letter case, where short vowels are usually omitted and many written words become lexically ambiguous;
  • a split between Classical Arabic, MSA and about 30 major spoken dialects.

It then covers encoder models such as AraBERT, MARBERT and QARIB, newer generative Arabic models, and the use of fine-tuning and prompting. It also catalogues common Arabic datasets, grouped by purpose.

The authors observe three trends: fast growth of Arabic-adapted models, steady work on the resource gap, and rising attention to dialects. Their open directions include dialect-aware training, Arabic-specific metrics, bias audits and human-assisting workflows. The paper is useful mainly as a compact map of the field.

Key points

  • AraBERT was adapted from BERT and pre-trained on about 70M sentences from open corpora and news sites, using MLM and NSP objectives. Words are first split into prefixes, stems and suffixes with Farasa segmentation, which reduced lexical redundancy. It led on SA, NER and QA but covered few dialects.
  • Arabic was included in Google's 2019 mBERT, which handled general tasks but was weaker on dialects than Arabic-only models.
  • Dataset catalogue:
    • MADAR: 25 dialects.
    • PADIC: dialect classification and MT.
    • QALB: grammar error correction.
    • Tashkeela: diacritization, 75 million words.
    • Also OpenITI, Arabic Wikipedia, ATD, OSACT4 (offensive-tweet detection) and the AraT5 data.
  • Paper counts at ACL and EMNLP for 2019–2024 show Arabic LM work growing but far behind English. Recent Arabic LLMs include ArabianLLM, AraGPT, ALLaM and Arabic-LLaMA.
  • The literature was gathered from a Google Scholar seed query, then expanded by following references and citing papers in both directions.

Interesting ideas

  • I1. Splitting words into affixes and stems before subword tokenization (as AraBERT does with Farasa) keeps the vocabulary meaningful and reduces redundant forms — For Mova (tokens): split English clitics and inflections ('s, n't, -ed, -ing) explicitly before lemmatization, so every token boundary has a stated linguistic reason.
  • I2. Undiacritized Arabic shows that dropped surface marks create ambiguity that a separate restoration task, with its own corpus like Tashkeela, can resolve — For Mova (tokens): treat English POS/lemma ambiguity (e.g. "record" as noun or verb) as an explicit restoration step whose chosen reading is logged for explanation.
  • I3. Treating MSA and dialects as related varieties, bridged by transfer learning rather than separate models, lets one model generalize across language variants — For Mova (tree): view treebanks in different UD dialects or annotation styles as varieties and learn conversion mappings, instead of training parsers per source.
  • I4. Grouping datasets by their real-world role (classification, dialect, structure-specific) shows at a glance which capabilities have training and test coverage — For Mova (learn): tag every Mova evaluation set by the stage and skill it probes, to expose untested stages and choose negative controls.
  • I5. The call for metrics that capture language-specific nuances, rather than generic scores, fits systems whose errors matter in particular structures — For Mova (learn): add targeted metrics for things like attachment of affix-derived forms or frame role accuracy, alongside LAS.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

5 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2505.08004 · arXiv 2505.08004

@misc{rhel2025large,
  title = {{Large Language Models and Arabic Content: A Review}},
  author = {Rhel, Haneh and Roussinov, Dmitri},
  year = {2025},
  doi = {10.48550/arxiv.2505.08004},
  eprint = {2505.08004},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.08004}
}

References

2 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Y. Liu et al., ‘Summary of ChatGPT-Related research and perspective towards the future of large language models’, Meta-Radiology, vol. 1, no. 2, p. 100017, Sep. 2023, doi: 10.1016/j.metrad.2023.100017. Z. Alyafeai, M. S. Alshaibani, B. AlKhamissi, H. Luqman, E. Alareqi, and A. Fadel, ‘Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Models’, Jun. 2023, [Online]. Available: http://arxiv.org/abs/2306.16322 L. Qin et al., ‘Large Language Models Meet NLP: A Survey’, May 2024, [Online]. Available: http://arxiv.org/abs/2405.12819 Y. Chang et al., ‘A Survey on Evaluation of Large Language Models’, Jul. 2023, [Online]. Available: http://arxiv.org/abs/2307.03109 S. Y. Kwon, G. Bhatia, E. M. B. Nagoud, and M. Abdul-Mageed, ‘ChatGPT for Arabic Grammatical Error Correction’, Aug. 2023, [Online]. Available: http://arxiv.org/abs/2308.04492 Y. Saoudi and M. M. Gammoudi, ‘TRENDS AND CHALLENGES OF ARABIC CHATBOTS: LITERATURE REVIEW’, Jordanian Journal of Computers and Information Technology, vol. 9, no. 3, pp. 261–286, Sep. 2023, doi: 10.5455/jjcit.71-1685381801. A. Koubaa, A. Ammar, L. Ghouti, O. Najar, and S. Sibaee, ‘ArabianGPT: Native Arabic GPT-based Large Language Model’, Feb. 2024, [Online]. Available: http://arxiv.org/abs/2402.15313 K. Kadaoui et al., ‘TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties’, Aug. 2023, [Online]. Available: http://arxiv.org/abs/2308.03051 N. Boudad, R. Faizi, R. Oulad Haj Thami, and R. Chiheb, ‘Sentiment analysis in Arabic: A review of the literature’, Dec. 01, 2018, Ain Shams University. doi: 10.1016/j.asej.2017.04.007. H. Alshammari and A. Ei-Sayed, ‘AIRABIC: Arabic Dataset for Performance Evaluation of AI Detectors’, in Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023, Institute of Electrical and Electronics Engineers Inc., 2023, pp. 864–870. doi: 10.1109/ICMLA58977.2023.00127. K. Kadaoui et al., ‘TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties’, Aug. 2023, [Online]. Available: http://arxiv.org/abs/2308.03051 OpenAI, ‘ChatGPT’. https://chatgpt.com/?utm_source=chatgpt.com M. Shao et al., ‘An Empirical Evaluation of LLMs for Solving Offensive Security Challenges’, Feb. 2024, [Online]. Available: http://arxiv.org/abs/2402.11814 Google, ‘Gemeni’, [Online]. Available: https://deepmind.google/technologies/gemini/. ‘Meta ’. [Online]. Available: https://about.meta.com/ Anthropic, ‘claude.ai’.[Online]. Available: https://claude.ai/ ↗ our card
  2. Microsoft 365 Copilot, ‘Copilot’. [Online]. Available: https://m365.cloud.microsoft/chat M. Salhab and F. Abu-Khzam, ‘AraSpell: A Deep Learning Approach for Arabic Spelling Correction’. [Online]. Available: https://github.com/msalhab96/AraSpell G. Abandah, A. Suyyagh, and M. Z. Khedher, ‘Correcting Arabic Soft Spelling Mistakes using BiLSTM-based Machine Learning’. [Online]. Available: www.ijacsa.thesai.org N. Habash, R. Eskander, and A. Hawwari, ‘A Morphological Analyzer for Egyptian Arabic’, 2012. W. Antoun, F. Baly, and H. Hajj, ‘AraBERT: Transformer-based Model for Arabic Language Understanding’, Feb. 2020, [Online]. Available: http://arxiv.org/abs/2003.00104 A. S. Alammary, ‘BERT Models for Arabic Text Classification: A Systematic Review’, Jun. 01, 2022, MDPI. doi: 10.3390/app12115720. A. Abdelali, S. Hassan, H. Mubarak, K. Darwish, and Y. Samih, ‘Pre-Training BERT on Arabic Tweets: Practical Considerations’, Feb. 2021, [Online]. Available: http://arxiv.org/abs/2102.10684 F. Qarah, ‘EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corpora’, Aug. 2024, [Online]. Available: http://arxiv.org/abs/2408.03524 A. Alghamdi et al., ‘AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing’, Jun. 2023, [Online]. Available: http://arxiv.org/abs/2306.06800 J. Kaplan et al., ‘Scaling Laws for Neural Language Models’, Jan. 2020, [Online]. Available: http://arxiv.org/abs/2001.08361 A. Alrayzah, F. Alsolami, and M. Saleh, ‘AraFast: Developing and Evaluating a Comprehensive Modern Standard Arabic Corpus for Enhanced Natural Language Processing’, Applied Sciences (Switzerland), vol. 14, no. 12, Jun. 2024, doi: 10.3390/app14125294. O. Nacar et al., ‘Towards Inclusive Arabic LLMs: A Culturally Aligned Benchmark in Arabic Large Language Model Evaluation’, 2025. [Online]. Available: https://huggingface.co/datasets/ M. S. Bari et al., ‘ALLaM: Large Language Models for Arabic and English’, Jul. 2024, [Online]. Available: http://arxiv.org/abs/2407.15390 H. Alshammari, A. El-Sayed, and K. Elleithy, ‘AI-Generated Text Detector for Arabic Language Using Encoder-Based Transformer Architecture’, Big Data and Cognitive Computing, vol. 8, no. 3, Mar. 2024, doi: 10.3390/bdcc8030032. F. Qarah, ‘AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis’, Mar. 2024, [Online]. Available: http://arxiv.org/abs/2403.12392 OpenITI, ‘Open Islamicate Texts Initiative’. [Online]. Available: https://openiti.org/ A. Abdaoui, M. Berrimi, M. Oussalah, and A. Moussaoui, ‘DziriBERT: a Pre-trained Language Model for the Algerian Dialect’, Sep. 2021, [Online]. Available: http://arxiv.org/abs/2109.12346 ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rhel-2025-large-language-models-arabic-content/