Elamur Research⌕
☰
home / papers / 2025 / Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction

Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction

Summary

The paper predicts how hard Arabic texts are to read. It was created as an entry for the 2025 BAREC competition, competing in its constrained setting, where each sentence and each document gets one of 19 readability levels. The authors join two resources. BAREC gives sentences with level labels. The SAMER lexicon gives graded lemmas. Lemmas are extracted with CAMeL Tools, looked up in SAMER and stored as node features. Each document becomes a heterogeneous graph of sentences, lemmas, difficulty classes and subject domains, connected by typed edges. A stack of SAGEConv layers classifies the sentence nodes. Separately, a fine-tuned AraBERTv2 readability model produces its own probabilities, and the two outputs are blended with a tunable weight. A document's level is its hardest predicted sentence. At document level the blend beats the graph alone on QWK and accuracy. At sentence level the graph alone is clearly better on exact accuracy. The work shows that explicit lexical structure can be paired with contextual embeddings for fine-grained readability in a morphologically rich language with little data.

Key points

  • The task has 19 ordinal readability levels at sentence and document level. The data are the BAREC corpus and the SAMER lexicon, which gives each lemma an average readability level, a frequency and a POS tag. Lemmas come from the CAMeL Tools morphology analyzer, and diacritic counts are also recorded.
  • Nodes:
    • sentences, with 768-dim embeddings from readability-arabertv2-d3tok-CE plus linguistic features;
    • lemmas;
    • classes (Foundational, Advanced, Specialized);
    • domains (Arts & Humanities, STEM, Social Sciences).
  • Edges: HAS_LEMMA, OCCUR_WITH, IN_CLASS and IN_DOMAIN. The IN_CLASS edge links a sentence to its labeled difficulty class. The paper does not explain how this edge is handled on unlabeled test data.
  • Model: a linear input projection for each node type, then 4 SAGEConv layers with ReLU, residual connections and LayerNorm, then an MLP classifier. The transformer branch is trained independently, and the two are combined only at the probability level.
  • Document-level results (Late Fusion vs GNN only): QWK 76.9 vs 75.6, accuracy 42.0 vs 40.0.
  • Sentence-level results: both reach QWK 78.5, but the GNN alone has much higher accuracy (50.0 vs 41.4). The conclusion's claim that fusion is stronger at both levels does not fully match this table.

Interesting ideas

  • I1. Lexicon entries as node features: the lexicon's graded attributes are attached to the lemma nodes themselves rather than kept as a side lookup table, so the model can propagate lexical difficulty through the structure. — For Mova (world): lemma entities in the world graph could carry frequency or grade attributes from compiled lexicons, giving later reasoning an explainable "how rare or hard is this word" signal.
  • I2. Typed edges with separate aggregation: neighbours are aggregated per edge type and node features are projected per node type, so different relations keep their own meaning instead of being merged. — For Mova (world): this is a good template for any learned component over the knowledge graph, where holdings, events and ontology links should stay distinguishable and traceable.
  • I3. Late fusion with a tunable weight: two independently trained models are combined only at their probability outputs, so each stays inspectable and the blend weight can be tuned per task. — For Mova (learn): witness parsers could be fused this way when building silver data, keeping each witness's vote visible and explaining any disagreement.
  • I4. Hardest sentence as document level: max pooling treats the hardest sentence as the barrier to understanding the whole text, which is a simple and interpretable aggregation rule. — For Mova (answer): Mova could pick or grade school-course texts and tales for a target reader level by flagging the single sentence that blocks comprehension.
  • I5. Ordinal evaluation at several granularities: the paper reports QWK, exact accuracy, ±1 accuracy and accuracy after collapsing the scale to 7, 5 and 3 levels, showing where errors are near misses. — For Mova (learn): any graded output, such as difficulty or confidence tiers, could be evaluated at several coarsenings so that near misses are separated from real failures.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2509.22870 · arXiv 2509.22870

@misc{elchafei2025lexicon,
  title = {{Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction}},
  author = {Elchafei, Passant and Osama, Mayar and Rageh, Mohamed and Abuelkheir, Mervat},
  year = {2025},
  doi = {10.48550/arxiv.2509.22870},
  eprint = {2509.22870},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2509.22870}
}

References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhamed Al Khalil, Nizar Habash, and Zhengyang Jiang. 2020. A large-scale leveled readability lexicon for Standard Arabic. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3053–3062, Marseille, France. European Language Resources Association. find our card
  2. Abdel-Karim Al-Tamimi, Manar Jaradat, Nuha Aljarrah, and Sahar Ghanim. 2014. Aari: Automatic arabic readability index. International Arab Journal of Information Technology, 11:370–378. find
  3. Bashar Alhafni, Reem Hazim, Juan David Pineros Liberato, Muhamed Al Khalil, and Nizar Habash. 2024. The SAMER Arabic text simplification corpus. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16079–16093, Torino, Italia. ELRA and ICCL. find our card
  4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. find our card
  5. Ibtehal Baazeem, Hend Al-Khalifa, and Abdulmalik Al-Salman. 2025. Araeyebility: Eye-tracking data for arabic text readability. Computation, 13(5). find
  6. Violetta Cavalli-Sforza, Hind Saddiki, and Naoual Nassiri. 2018. Arabic readability research: Current state and future directions. Procedia Computer Science, 142:38–49. Arabic Computational Linguistics. find
  7. Mo El-Haj, Sultan Almujaiwel, Damith Premasiri, Tharindu Ranasinghe, and Ruslan Mitkov. 2024. DARES: Dataset for Arabic readability estimation of school materials. In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context @ LREC-COLING 2024, pages 103–113, Torino, Italia. ELRA and ICCL. find
  8. Khalid N. Elmadani, Bashar Alhafni, Hanada Taha, and Nizar Habash. 2025a. BAREC shared task 2025 on Arabic readability assessment. In Proceedings of the Third Arabic Natural Language Processing Conference, Suzhou, China. Association for Computational Linguistics. find
  9. Khalid N. Elmadani, Nizar Habash, and Hanada Taha-Thomure. 2025b. A large and balanced corpus for fine-grained arabic readability assessment. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16376–16400, Vienna, Austria. Association for Computational Linguistics. find
  10. William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, volume 30, pages 1024–1034. Curran Associates, Inc. find our card
  11. Reem Hazim, Hind Saddiki, Bashar Alhafni, Muhamed Al Khalil, and Nizar Habash. 2022. Arabic word-level readability visualization for assisted text simplification. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 242–249, Abu Dhabi, UAE. Association for Computational Linguistics. find our card
  12. Joseph Marvin Imperial and Ekaterina Kochmar. 2023. Automatic readability assessment for closely related languages. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5371–5386, Toronto, Canada. Association for Computational Linguistics. find
  13. Juan Liberato, Bashar Alhafni, Muhamed Khalil, and Nizar Habash. 2024. Strategies for Arabic readability modeling. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 55–66, Bangkok, Thailand. Association for Computational Linguistics. find
  14. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association. find our card
  15. Qi Sun, Kun Zhang, Kun Huang, Tiancheng Xu, Xun Li, and Yaodi Liu. 2023. Document-level relation extraction with two-stage dynamic graph attention networks. Knowledge-Based Systems, 267:110428. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elchafei-2025-lexicon-enriched-graph-modeling-arabic/