Elamur Research⌕
☰
home / papers / 2026 / Automated Scoring of Arabic Text Using Large Language Models: A Literature Review

Automated Scoring of Arabic Text Using Large Language Models: A Literature Review

Summary

This survey maps how Deep Learning and large language models are used to grade Arabic learner writing automatically. Two tasks are in scope: the grading of brief responses (ASAG) and the automated scoring of essays (AES). The authors limit the scope to work from 2020–2025 and propose a five-part taxonomy for comparing studies: application domain, whether feedback is produced, model architecture, alignment with a competency framework, and prompting strategy. Using this grid, they compare the available Arabic datasets and the published systems, including their metrics and reported scores.

The main finding is that the field is small and fragmented. Datasets are few, mostly hand-annotated and modest in size. Because metrics differ between studies, no clear state of the art can be named. Newer generative LLMs used with prompting do not clearly beat a fine-tuned encoder. Explanatory feedback appears in only one system, and no system ties its scoring to curricula or rubric standards.

The review matters because it gives a structured baseline and a research agenda for pedagogically grounded Arabic assessment.

Key points

  • Essay datasets surveyed: ZAEBUC (214 essays, bilingual AR–EN, with POS/morph/lemma and CEFR A2–C1 labels), QAES (195, first public Arabic essay scoring set, 7 trait scores), ZaQQ (1021), AR-AES (2046), TAQEEM (1,265) and LAILA (7,859, the largest).
  • Short answer datasets surveyed: translated ASAP-SAS (17,205 answers), AR-ASAG (2,133), Sociology SAS (270) and ESD (610). All of them provide only one holistic score, with no trait-level scores.
  • AES results (QWK): fine-tuned AraBERT on AR-AES reaches 0.88. LLM approaches score lower: mixed-language prompting of ChatGPT/Llama/Aya/Jais/ACEGPT gets 0.67, GPT-4o with few-shot plus rubric gets 0.65, GPT-4.1 gets 0.612, and Fanar/ALLaM/R7B Arabic on LAILA get 0.58.
  • ASAG results: AraBERT/ELECTRA/Bi-LSTM reach QWK 0.78. Local weight-based LSA reaches F1 82.82%. AraBERT+LSTM reaches 71.31% accuracy. A Siamese AraBERT reaches Pearson 0.79. GPT-4/Llama 3/DeepSeek-V3 on ESD reach QWK 0.83, and this is the only system that generates feedback.
  • Open gaps:
    • no scoring of visual or multimodal answers;
    • no use of competency frameworks;
    • scoring is treated as a static one-shot prediction;
    • similarity-based graders miss correct answers that are worded differently.

Interesting ideas

  • I1. QWK is used as the main metric because it corrects for chance and penalizes a prediction more the farther it lands from the true ordinal score, so near misses still earn partial credit. — For Mova (answer): Graded school-test answers from Mova's student readers could be scored with QWK rather than exact-match accuracy, separating "almost right" from "far off".
  • I2. Essay datasets that provide trait-specific scores (organization, vocabulary, grammar, mechanics, etc.) next to a holistic grade support diagnosis of what exactly went wrong, which a single number cannot. — For Mova (back): Generated or round-tripped text from the compressor could be evaluated per trait (grammaticality, coherence, content preservation), so regressions are pinned to one dimension.
  • I3. The review criticizes graders built on lexical overlap or semantic similarity because they fail on correct answers phrased in a different way. This is an argument for comparing meaning rather than surface form. — For Mova (answer): Mova could check a student's answer by matching its frames and world state against the reference answer's graph, accepting paraphrases and still giving an explainable verdict.
  • I4. The authors suggest giving the model the assignment text, its target competencies and the marking scheme as explicit inputs, so scoring becomes assignment-aware instead of a generic prediction. — For Mova (answer): Question-answering readers could take the test's rubric as a structured input that decides which facts in the text world must appear in an acceptable answer.
  • I5. Comparing systems on a fixed grid of dimensions, with each one marked present, absent or partial, quickly shows which capabilities nobody has built yet. — For Mova (learn): A similar capability matrix across Mova's evaluation suites (stages × explainability × negative controls) could expose untested areas before choosing training priorities.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2606.09830 · arXiv 2606.09830

@misc{dahimi2026automated,
  title = {{Automated Scoring of Arabic Text Using Large Language Models: A Literature Review}},
  author = {Dahimi, Khaoula and Cherroun, Hadda and Belabbaci, Amel},
  year = {2026},
  doi = {10.48550/arxiv.2606.09830},
  eprint = {2606.09830},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2606.09830}
}

References

22 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dimitrios Alikaniotis, Helen Yannakoudakis, and Marek Rei. Automatic text scoring using neural networks. arXiv preprint arXiv:1606.04289, 2016. ↗ our card
  2. Nada Almarwani, Alaa Alharbi, and Samah Aloufi. Taibah at taqeem 2025: Leveraging gpt-4o for arabic essay scoring. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, pages 989–997, 2025. find
  3. Mohamad Alnajjar, Ahmad Almoustafa, Tomohiro Nishiyama, Shoko Wakamiya, Eiji Aramaki, and Takuya Matsuzaki. Arxhyoka at taqeem2025: Comparative approaches to arabic essay trait scoring. In Proceedings of the Third Arabic Natural Language Processing Conference: Shared Tasks, pages 977–982, 2025. 10.18653/v1/2025.arabicnlp-sharedtasks.135. ↗
  4. Afnan Alqurashi, Basma Alharbi, and Sahar Sabbeh. An automatic grading system for arabic language short-answer questions using deep learning. Engineering, Technology & Applied Science Research, 15(5):26665–26675, 2025. find
  5. Rasha M. Badry, Mostafa Ali, Esraa Rslan, and Mostafa R. Kaseb. Automatic arabic grading system for short answer questions. IEEE Access, 11:39457–39465, 2023. ISSN 2169-3536. 10.1109/ACCESS.2023.3267407. ↗
  6. May Bashendy, Salam Albatarni, Sohaila Eltanbouly, Eman Zahran, Hamdo Elhuseyin, Tamer Elsayed, Walid Massoud, and Houda Bouamor. Qaes: First publicly-available trait-specific annotations for automated scoring of arabic essays. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 337–351, 2024. find
  7. May Bashendy, Salam Albatarni, Sohaila Eltanbouly, Walid Massoud, Houda Bouamor, and Tamer Elsayed. Taqeem 2025: Overview of the first shared task for arabic quality evaluation of essays in multi-dimensions. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, pages 966–976, 2025a. find
  8. May Bashendy, Walid Massoud, Sohaila Eltanbouly, Salam Albatarni, Marwan Sayed, Abrar Abir, Houda Bouamor, and Tamer Elsayed. Laila: A large trait-based dataset for arabic automated essay scoring. arXiv preprint arXiv:2512.24235, 2025b. ↗
  9. Afrizal Doewes, Nughthoh Kurdhi, and Akrati Saxena. Evaluating quadratic weighted kappa as the standard performance metric for automated essay scoring. In 16th International Conference on Educational Data Mining, EDM 2023, pages 103–113. International Educational Data Mining Society (IEDMS), 2023. find
  10. Yomna Elsayed, Emad Nabil, Marwan Torki, Safiullah Faizullah, and Ayman Khalafallah. Zaqq: A new arabic dataset for automatic essay scoring via a novel human–ai collaborative framework. Data, 10(9):148, 2025. find
  11. Oluwole Fagbohun, Nwaamaka Pearl Iduwe, Mustapha Abdullahi, Adeseye Ifaturoti, and OM Nwanna. Beyond traditional assessment: Exploring the impact of large language models on grading practices. Journal of Artificial Intelligence, Machine Learning and Data Science, 2(1): 1–8, 2024. find
  12. Rayed Ghazawi and Edwin Simpson. Automated essay scoring in arabic: a dataset and analysis of a bert-based system. (arXiv:2407.11212), 2024. 10.48550/arXiv.2407.11212. URL http://arxiv.org/abs/2407.11212. arXiv:2407.11212 [cs]. ↗ our card
  13. Rayed Ghazawi and Edwin Simpson. How well can llms grade essays in arabic? Computers and Education: Artificial Intelligence, page 100449, 2025. find our card
  14. Nizar Habash and David Palfreyman. Zaebuc: An annotated arabic-english bilingual writer corpus. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 79–88, 2022. find
  15. Nourmeen Lotfy, Abdulaziz Shehab, Mohammed Elhoseny, and Ahmed Abu-Elfetouh. An enhanced automatic arabic essay scoring system based on machine learning algorithms. Computers, Materials, & Continua, 77(1):1227, 2023. find
  16. Emad Nabil, Mostafa Mohamed Saeed, Rana Reda, Safiullah Faizullah, and Wael Hassan Gomaa. Leveraging large language models for arabic short answer grading and feedback generation. International Journal of Information and Education Technology, 15(12), 2025. find
  17. Omar Nael, Youssef ELmanyalawy, and Nada Sharaf. Arascore: A deep learning-based system for arabic short answer scoring. Array, 13:100109, 2022. find
  18. Hatem M Noaman, Mohsen Rashwan, and Hazem Raafat. Leveraging distance-optimized transformers for high-performance arabic short answers grading. International Journal of Advanced Computer Science & Applications, 16(9), 2025. find
  19. Leila Ouahrani and Djamal Bennouar. Ar-asag an arabic dataset for automatic short answer grading evaluation. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2634–2643, 2020. find our card
  20. Dadi Ramesh and Suresh Kumar Sanampudi. An automated essay scoring systems: a systematic literature review. Artificial intelligence review, 55(3):2495–2527, 2022. find
  21. Shehab Shehab. Arabic short-answer similarity and grading dataset for automated assessment research, 2026. URL https://doi.org/10.5281/zenodo.18630360. ↗
  22. Younes Alaoui Soulimani, Lotfi El Achaak, and Mohammed Bouhorma. Deep learning based arabic short answer grading in serious games. International Journal of Electrical & Computer Engineering (2088-8708), 14(1), 2024. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dahimi-2026-automated-scoring-arabic-text-using/