Elamur Research⌕
☰
home / papers / 2025 / LIS at BAREC Shared Task 2025: Multi-Scale Curriculum Learning for Arabic Sentence-Level Readability Assessment Using Pre-trained Language Models

LIS at BAREC Shared Task 2025: Multi-Scale Curriculum Learning for Arabic Sentence-Level Readability Assessment Using Pre-trained Language Models

Paper · 2025 · Anya Amel Nait Djoudi, Patrice Bellot and Adrian-Gabriel Chifu

Summary

Arabic sentence-level readability has received far less attention than English. The language is morphologically rich, and most Arabic datasets offer only two or three difficulty classes. Competing in the Strict Track of the BAREC Shared Task 2025, the authors train only on the BAREC corpus: roughly 69K sentences from 1,922 documents, each labelled on a 19-step scale from kindergarten to postgraduate. After cleaning and dediacritizing the text with CAMeL Tools, they compare five systems:

  • MLP classifiers on mean-pooled embeddings from a general encoder (bert-base-arabic-camelbert-mix)
  • the same MLP on embeddings from a readability-tuned encoder (readability-camelbert-word-CE)
  • a progressive MLP trained through 3→5→7→19-level stages
  • progressive fine-tuning of the whole encoder through those stages
  • plain end-to-end fine-tuning

Plain fine-tuning scores best on the blind test, with QWK 82.4. It narrowly beats the curriculum variant and the official baseline. The main lesson is that encoder choice matters far more than training schedule. Coarse-to-fine curricula give only local, level-specific gains, which is a useful caution before adding multi-stage training complexity.

Key points

  • Blind-test results (QWK / Acc19):
    • baseline: 81.5 / 58.1
    • CAMeLBERTMix_MLP: 41.2 / 21.2
    • CAMeLBERTWCE_MLP: 80.5 / 55.7
    • P_CAMeLBERTWCE_MLP: 80.7 / 55.9
    • PFT_CAMeLBERTWCE: 82.0 / 56.7
    • FT_CAMeLBERTWCE: 82.4 / 57.5
    • The baseline still holds the top exact accuracy.
  • Preprocessing dropped 2.5–5% of rows (placeholders such as #NAME?, extra whitespace, duplicates). The final splits are about 52k train, 7.1k validation and 7k test.
  • The MLP curriculum collapses the 19 labels with fixed bins: [0,7,13,19] for 3 levels, [0,4,8,12,16,19] for 5, and [0,3,6,9,12,15,17,19] for 7. Hidden-layer weights carry over between stages, and the network grows to 512-256-128 for the final stage.
  • Progressive fine-tuning keeps the encoder between stages but starts each granularity with a new classification head. It uses AdamW with linear warmup, a learning rate of 2e-5 that drops to 1e-5 in transfer stages, and dropout 0.3.
  • The task is scored as ordinal classification:
    • QWK is the primary metric.
    • Accuracy is reported at 19, 7, 5 and 3 levels.
    • Adjacent (±1) accuracy and mean distance (MAE) are also reported.
  • Errors are concentrated at level 2 and at levels 15–19. The curriculum model makes fewer errors at levels 3–6, 9, 11, 12, 16 and 17.

Interesting ideas

  • I1. Train on a coarse version of the label set first, then refine it stage by stage, reusing the encoder but attaching a new head each time; here it paid off mainly when the encoder itself was trainable.
  • I2. Run the "no curriculum" ablation with exactly the optimizer, warmup and dropout of the final curriculum stage, so any difference can be credited to the schedule alone.
  • I3. Compute per-class conditional error rates next to aggregate scores; the overall loser can still win on particular classes.
  • I4. Treat graded labels as ordinal and score them with QWK, adjacent (±1) accuracy and mean distance, so that near misses cost less than distant ones.
  • I5. Comparing a general and a task-tuned encoder under the same light classifier quickly shows how much representation quality dominates (41.2 vs 80.5 QWK).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Fine-grained taggers or parser label sets could be warmed up on collapsed categories first. Results should be checked separately for frozen and trainable backbones.
  • I2 (learn): Every claimed training trick in our pipeline deserves a matched-hyperparameter control run before we keep it.
  • I3 (learn): Breaking parser or tagger errors down per relation or tag would show where two models complement each other, and could guide witness weighting in silver-data agreement.
  • I4 (answer): Grading of difficulty-graded school-course tests could report how far off a predicted level is, not just whether it matched exactly.
  • I5 (tree): Before tuning heads in gparse, swap encoders under a fixed small probe to see which backbone is worth investing in.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🎓Aix-Marseille UniversitéMarseille · France24 works · 39,160 citations🔬Laboratoire d’Informatique et SystèmesMarseille · France8 works · 106 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Anya Amel Nait Djoudi, Patrice Bellot and Adrian-Gabriel Chifu. 2025. LIS at BAREC Shared Task 2025: Multi-Scale Curriculum Learning for Arabic Sentence-Level Readability Assessment Using Pre-trained Language Models. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.51

@inproceedings{djoudi2025lis,
  title = {{LIS at BAREC Shared Task 2025: Multi-Scale Curriculum Learning for Arabic Sentence-Level Readability Assessment Using Pre-trained Language Models}},
  author = {Anya Amel Nait Djoudi, Patrice Bellot and Adrian-Gabriel Chifu},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.51},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.51}
}

References

39 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M Al-Dawsari. 2004. The assessment of readability books content (boys-girls) of the first grade of intermediate school according to readability standards. Sultan Qaboos University, Muscat. find
  2. Hend S Al-Khalifa and Amani A Al-Ajlan. 2010. Automatic readability measurements of the arabic text: An exploratory study. Arabian Journal for Science and Engineering, 35(2 C):103–124. find
  3. Muhamed Al-Khalil, Nizar Habash, and Zhengyang Jiang. 2020. A large-scale leveled readability lexicon for standard arabic. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3053–3062.
  4. Abdel Karim Al Tamimi, Manar Jaradat, Nuha Al-Jarrah, and Sahar Ghanem. 2014. Aari: automatic arabic readability index. Int. Arab J. Inf. Technol., 11(4):370–378. find
  5. Bashar Alhafni, Reem Hazim, Juan David Pineros Liberato, Muhamed Al Khalil, and Nizar Habash. 2024. The samer arabic text simplification corpus. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16079–16093.
  6. Yuki Arase, Satoru Uchida, and Tomoyuki Kajiwara. 2022. Cefr-based sentence difficulty annotation and assessment. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6206–6219. find
  7. Ion Madrazo Azpiazu and Maria Soledad Pera. 2019. Multiattentive recurrent neural network architecture for multilingual readability assessment. Transactions of the Association for Computational Linguistics, 7:421–436. ↗
  8. May Bashendy, Salam Albatarni, Sohaila Eltanbouly, Eman Zahran, Hamdo Elhuseyin, Tamer Elsayed, Walid Massoud, and Houda Bouamor. 2024. Qaes: First publicly-available trait-specific annotations for automated scoring of arabic essays. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 337–351.
  9. Dominique Brunato, Lorenzo De Mattei, Felice Dell’Or-letta, Benedetta Iavarone, Giulia Venturi, and 1 others. 2018. Is this sentence difficult? do you agree? In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, pages 2690–2699. Association for Computational Linguistics. find
  10. Jacob Cohen. 1968. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin, 70(4):213. ↗
  11. Kevyn Collins-Thompson and Jamie Callan. 2004. Information retrieval for language tutoring: An overview of the reap project. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, pages 544–545. find
  12. Edgar Dale and Jeanne S Chall. 1948. A formula for predicting readability: Instructions. Educational research bulletin, pages 37–54. find
  13. Orphée De Clercq and Véronique Hoste. 2016. All mixed up? finding the optimal feature set for general readability prediction and its application to english and dutch. Computational Linguistics, 42(3):457–490. find
  14. Tovly Deutsch, Masoud Jasbi, and Stuart Shieber. 2020. Linguistic features for readability assessment. arXiv preprint arXiv:2006.00377. ↗
  15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186. ↗
  16. Anya Amel Nait Djoudi, Patrice Bellot, and Adrian-Gabriel Chifu. 2025. Bioreadnet: A transformer-driven hybrid model for target audience-aware biomedical text readability assessment. In Proceedings of the 2025 ACM Symposium on Document Engineering, DocEng ’25, page 1–10. ACM. find
  17. Mahmoud El-Haj and Paul Rayson. 2016. Osman―a novel arabic readability metric. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 250–255. find
  18. Mo El-Haj, Sultan Almujaiwel, Damith Premasiri, Tharindu Ranasinghe, and Ruslan Mitkov. 2024. Dares: Dataset for arabic readability estimation of school materials. In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context@ LREC-COLING 2024, pages 103–113.
  19. Khalid N. Elmadani, Bashar Alhafni, Hanada Taha, and Nizar Habash. 2025a. BAREC shared task 2025 on Arabic readability assessment. In Proceedings of the Third Arabic Natural Language Processing Conference, Suzhou, China. Association for Computational Linguistics.
  20. Khalid N. Elmadani, Nizar Habash, and Hanada Taha-Thomure. 2025b. A large and balanced corpus for fine-grained Arabic readability assessment. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16376–16400, Vienna, Austria. Association for Computational Linguistics.
  21. Jonathan Neil Forsyth. 2014. Automatic readability prediction for modern standard Arabic. Ph.D. thesis, Brigham Young University. Department of Linguistics and English Language. find
  22. Susannah Fox and Maeve Duggan. 2013. Health online 2013. Health, 2013:1–55. find
  23. Nizar Habash and David Palfreyman. 2022. Zaebuc: An annotated arabic-english bilingual writer corpus. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 79–88. find
  24. Nizar Habash, Hanada Taha-Thomure, Khalid N. Elmadani, Zeina Zeino, and Abdallah Abushmaes. 2025. Guidelines for fine-grained sentence-level Arabic readability annotation. In Proceedings of the 19th Linguistic Annotation Workshop (LAW-XIX-2025), pages 359–376, Vienna, Austria. Association for Computational Linguistics.
  25. Reem Hazim, Hind Saddiki, Bashar Alhafni, Muhamed Al Khalil, and Nizar Habash. 2022. Arabic word-level readability visualization for assisted text simplification. arXiv preprint arXiv:2210.10672. ↗
  26. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92–104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  27. Adam Kilgarriff, Frieda Charalabopoulou, Maria Gavrilidou, Janne Bondi Johannessen, Saussan Khalil, Sofie Johansson Kokkinakis, Robert Lew, Serge Sharoff, Ravikiran Vadlapudi, and Elena Volodina. 2014. Corpus-based vocabulary lists for language learners for nine languages. Language resources and evaluation, 48(1):121–163. ↗
  28. J Peter Kincaid, Robert P Fishburne Jr, Richard L Rogers, and Brad S Chissom. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. ↗
  29. Juan Piñeros Liberato, Bashar Alhafni, Muhamed Al Khalil, and Nizar Habash. 2024. Strategies for arabic readability modeling. arXiv preprint arXiv:2407.03032. ↗
  30. Tarek Naous, Michael J Ryan, Anton Lavrouk, Mohit Chandra, and Wei Xu. 2024. Readme++: Benchmarking multilingual language models for multi-domain readability assessment. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2024, page 12230. find
  31. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. Camel tools: An open source python toolkit for arabic natural language processing. In Proceedings of the twelfth language resources and evaluation conference, pages 7022–7032.
  32. Xinying Qiu, Yuan Chen, Hanwu Chen, Jian-Yun Nie, Yuming Shen, and Dawei Lu. 2021. Learning syntactic dense embedding with correlation graph for automatic readability assessment. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3013–3025, Online. Association for Computational Linguistics. ↗
  33. Rasha Soliman and Laila Familiar. 2024. Creating a cefr arabic vocabulary profile: A frequency-based multi-dialectal approach. Critical Multilingualism Studies, 11(1):266–286. ↗
  34. Sanja Štajner, Simone Paolo Ponzetto, and Heiner Stuckenschmidt. 2017. Automatic assessment of absolute sentence complexity. In Proceedings of the 26th international joint conference on artificial intelligence, ijcai, volume 17, pages 4096–4102. find
  35. Hanada Taha-Thomure. 2017. Arabic language text leveling(). Educational Book House (). find
  36. Sowmya Vajjala and Ivana Lučić. 2018. Onestopenglish corpus: A new corpus for automatic readability assessment and text simplification. In Proceedings of the thirteenth workshop on innovative use of NLP for building educational applications, pages 297–304.
  37. Sowmya Vajjala and Detmar Meurers. 2012. On improving the accuracy of readability classification using insights from second language acquisition. In Proceedings of the seventh workshop on building educational applications using NLP, pages 163–173.
  38. Menglin Xia, Ekaterina Kochmar, and Ted Briscoe. 2016. Text readability assessment for second language learners. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pages 12–22.
  39. Wei Xu, Chris Callison-Burch, and Courtney Napoles. 2015. Problems in current text simplification research: New data can help. Transactions of the Association for Computational Linguistics, 3:283–297. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/djoudi-2025-lis-barec-shared-task-2025/