Elamur Research⌕
☰
home / papers / 2018 / A Leveled Reading Corpus of Modern Standard Arabic

A Leveled Reading Corpus of Modern Standard Arabic

Paper · 2018 · Muhamed Al Khalil, Hind Saddiki, Nizar Habash et al.

Summary

Most corpora for Modern Standard Arabic (MSA) come from news, so educational resources are scarce. This matters especially for readability assessment, text simplification and language learning. The authors built a reading corpus with level labels and two parts. The first part is the complete UAE school curriculum, grades 1–12, covering Arabic language, social studies and Islamic studies. The second part is 129 full-length fiction works from the Hindawi Foundation, all assigned a Post-secondary level. The source files (InDesign and epub) were converted to plain text, and every token was tagged with a lemma and POS using MADAMIRA.

To test whether the grade labels make sense, the authors checked how much of each grade's text uses lemmas already seen in earlier grades. This coverage rises steadily across the curriculum. They also compared vocabulary overlap with news text (Gigaword) and legal/diplomatic text (UN Corpus) using the Dice coefficient. The curricular texts appear different enough from both to count as a genre of their own. The resource supports work on simplifying Arabic fiction for school readers, which is the aim of the SAMER project.

Key points

  • Size: the curricular part has about 1.4M tokens across 12 grades and is also annotated by subject, term and unit. The fiction part has about 5.6M tokens from 129 public-domain books, with author, title and year metadata.
  • Tagging quality: MADAMIRA was trained on news. On literary text its lemma+POS accuracy dropped by about 4 points absolute, to roughly 92%.
  • Lexical coverage:
    • It stays above the 95% "adequate comprehension" threshold throughout the curriculum.
    • It reaches the 98% "optimal" threshold from Grade 8 onward.
    • For fiction, the full 12-grade vocabulary covers 97.1% of tokens on average per book.
  • Genre similarity (Dice on unique lemma sets):
    • The lowest pair is UN vs Fiction at 57.3%.
    • Gigaword vs Fiction is 65.5%.
    • Curriculum differs from Fiction, Gigaword and UN by about 23%, 29% and 36% respectively.
  • Scale compared with earlier work: previous leveled Arabic corpora were much smaller. One had 150 Saudi texts; another had 1196 Jordanian texts totalling about 400K words.

Interesting ideas

  • I1. Leveling is validated cumulatively: lemmas from all earlier grades count as "known vocabulary", and each new grade is scored by the share of its tokens covered, giving a cheap, interpretable difficulty check without reader studies.
  • I2. Genre distance is computed as Dice overlap of unique lemma sets on size-matched random samples, and is calibrated against a pair of corpora known to differ (news vs fiction) so the numbers have a reference scale.
  • I3. Fiction coverage is averaged per book rather than pooled over all tokens, so a few long books do not dominate the estimate of what reading an arbitrary book requires.
  • I4. Before relying on a news-trained tagger, the authors ran a small in-house check on literary text to measure the out-of-domain accuracy drop explicitly.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): course texts and test passages could be ordered by lemma coverage relative to what earlier lessons supplied, with uncovered lemmas listed explicitly as the reason a passage is harder.
  • I2 (learn): when judging whether silver data or a new text domain drifts from the training treebank, a known-different corpus pair can serve as a negative-control baseline for what "real" shift looks like.
  • I3 (answer): accuracy on fable and tale QA could be reported as a per-story average, so no single long text hides weakness on the others.
  • I4 (tokens): lemma and POS accuracy for crate en could be measured separately on fables, textbooks and word problems, so genre-specific errors are known before they propagate into trees and frames.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓New York UniversityNew York · United States201 works · 100,180 citations🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations🎓Mohammed V UniversityRabat · Morocco8 works · 103 citations

Also on the byline: Ministry of Education.

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 worksEducation · 203 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Muhamed Al Khalil, Hind Saddiki, Nizar Habash and Latifa Alfalasi. 2018. A Leveled Reading Corpus of Modern Standard Arabic. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{khalil2018leveled,
  title = {{A Leveled Reading Corpus of Modern Standard Arabic}},
  author = {Muhamed {Al Khalil}, Hind Saddiki, Nizar Habash and Latifa Alfalasi},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/2vxe57mwjr49},
  url = {https://doi.org/10.63317/2vxe57mwjr49}
}

References

26 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul-Mageed, M. and Diab, M. T. (2012). Awatif: A multi-genre corpus for modern standard Arabic subjectivity and sentiment analysis. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 3907–3914.
  2. Al-Khalifa, H. S. and Al-Ajlan, A. A. (2010). Automatic readability measurements of the Arabic text: An exploratory study. Arabian Journal for Science and Engineering, 35(2 C):103–124. find
  3. Al Khalil, M., Habash, N., and Saddiki, H. (2017). Simplification of Arabic masterpieces for extensive reading: A project overview. Procedia Computer Science, 117:192– 198. find
  4. Al Tamimi, A. K., Jaradat, M., Al-Jarrah, N., and Ghanem, S. (2014). Aari: automatic Arabic readability index. Int. Arab J. Inf. Technol., 11(4):370–378. find
  5. Alfaifi, A., Atwell, E., and Abuhakema, G. (2013). Error annotation of the Arabic learner corpus. In Language Processing and Knowledge in the Web, pages 14–22. Springer. find
  6. Bensalem, I., Rosso, P., and Chikhi, S. (2013). A new corpus for the evaluation of Arabic intrinsic plagiarism detection. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 53–58. Springer. find
  7. Biber, D. (2011). Corpus linguistics and the study of literature: Back to the future? Scientific Study of Literature, 1(1):15–23. ↗
  8. Braun, S., Kohn, K., et al. (2006). Corpus technology and language pedagogy: New resources, new tools, new methods. English corpus linguistics vol 3. find
  9. Collins-Thompson, K. (2014). Computational assessment of text readability: A survey of current and future research. ITL-International Journal of Applied Linguistics, 165(2):97–135. ↗
  10. François, T. (2014). An analysis of a French as a foreign language corpus for readability assessment. In Proceedings of the third workshop on NLP for computer-assisted language learning at SLTC 2014, Uppsala University, number 107. Linköping University Electronic Press. find
  11. Habash, N. Y. (2010). Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
  12. Jarrar, M., Habash, N., Alrimawi, F., Akra, D., and Zalmout, N. (2016). Curras: an annotated corpus for the Palestinian Arabic dialect. Language Resources and Evaluation, pages 1–31. ↗
  13. Khalifa, S., Habash, N., Abdulrahim, D., and Hassan, S. (2016). A Large Scale Corpus of Gulf Arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Portorož, Slovenia.
  14. Laufer, B. and Ravenhorst-Kalovski, G. C. (2010). Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a foreign language, 22(1):15. ↗
  15. McEnery, T., Xiao, R., and Tono, Y. (2006). Corpus-based language studies: An advanced resource book. Taylor & Francis. find
  16. Nation, I. (2006). How large a vocabulary is needed for reading and listening? Canadian Modern Language Review, 63(1):59–82. ↗
  17. Nation, I. S. (2013). Learning Vocabulary in Another Language Google eBook. Cambridge University Press. find
  18. Parker, R., Graff, D., Chen, K., Kong, J., and Maeda, K. (2011). Arabic Gigaword Fifth Edition. LDC catalog number No. LDC2011T11, ISBN 1-58563-595-2. find
  19. Pasha, A., Al-Badrashiny, M., Diab, M., El Kholy, A., Eskander, R., Habash, N., Pooleery, M., Rambow, O., and Roth, R. M. (2014). MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In LREC 2014, Reykjavik, Iceland.
  20. Pravec, N. A. (2002). Survey of learner corpora. ICAME journal, 26(1):8–14. find
  21. Shoufan, A. and Al-Ameri, S. (2015). Natural language processing for dialectical Arabic: A survey. In ANLP Workshop 2015, page 36.
  22. Volodina, E., Pilán, I., Borin, L., and Tiedemann, T. L. (2014). A flexible language learning platform based on language resources and web services. In LREC, pages 3973–3978. find
  23. Zaghouani, W., Mohit, B., Habash, N., Obeid, O., Tomeh, N., Rozovskaya, A., Farra, N., Alkuhlani, S., and Oflazer, K. (2014). Large scale Arabic error annotation: Guidelines and framework. In LREC 2014, Reykjavik, Iceland.
  24. Zaghouani, W. (2014). Critical survey of the freely available Arabic corpora. In Proceedings of the Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools, LREC, pages 1–8. find
  25. Zalmout, N., Saddiki, H., and Habash, N. (2016). Analysis of foreign language teaching methods: An automatic readability approach. NLPTEA 2016, page 122. find
  26. Ziemski, M., Junczys-Dowmunt, M., and Pouliquen, B. (2016). The united nations parallel corpus v1. 0. In LREC.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khalil-2018-leveled-reading-corpus-modern-standard/