Elamur Research⌕
☰
home / papers / 2025 / GNNinjas at BAREC Shared Task 2025: Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction

GNNinjas at BAREC Shared Task 2025: Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction

Paper · 2025 · Passant Elchafei, Mayar Osama, Mohamad Rageh et al.

Summary

The paper describes the GNNinjas submission to the BAREC 2025 Shared Task, entered in that competition's Constrained Track. The task is to assign Arabic sentences and documents to one of 19 fine-grained readability levels. The authors argue that sequence-only transformers miss explicit lexical and structural cues. They therefore lemmatize the BAREC sentences with CAMeL Tools and look up each lemma in the SAMER lexicon to obtain graded difficulty and frequency attributes. From this they build a heterogeneous graph whose nodes are sentences, lemmas, difficulty classes and subject domains, linked by several typed edges. A SAGEConv-based GNN runs over this graph, and a separate fine-tuned AraBERTv2 readability classifier is trained alongside it. At inference, the two output distributions are mixed with a tunable weight. A document receives the level of its hardest predicted sentence. Fusion gives small gains at the document level, but the GNN alone is clearly better at exact sentence-level accuracy. The work matters as a case study in injecting a graded lexicon into structured, inspectable features rather than relying only on embeddings.

Key points

  • Resources used:
    • BAREC corpus, with 19 levels at sentence and document granularity.
    • SAMER lexicon, which gives each lemma an average readability level, frequency and POS.
    • CAMeL Tools morphological analyzer, used for lemma extraction and diacritic counts.
  • Node types:
    • Sentence nodes carry 768-dimensional embeddings from readability-arabertv2-d3tok-CE plus linguistic features.
    • The other node types are lemmas, classes (Foundational, Advanced, Specialized) and domains (Arts & Humanities, STEM, Social Sciences).
  • Edge types are HAS_LEMMA, OCCUR_WITH (lemma co-occurrence), IN_CLASS and IN_DOMAIN.
  • Architecture:
    • Each node type gets its own linear input projection.
    • These feed into 4 SAGEConv layers with per-edge-type aggregation, ReLU, and a residual connection followed by LayerNorm.
    • An MLP head performs the final classification.
  • Results (GNN-only vs. Late Fusion):
    • Document level: QWK 75.6 vs. 76.9; accuracy 40.0 vs. 42.0.
    • Sentence level: QWK 78.5 for both; accuracy 50.0 vs. 41.4.
    • The conclusion claims fusion helps at both levels, which the sentence-level numbers in the table do not support.

Interesting ideas

  • I1. Matching each lemma against a graded vocabulary resource turns an external lexicon into per-node features, rather than leaving it as a side lookup table.
  • I2. A single graph mixes sentence, lemma and category nodes, and neighbors are aggregated separately for each edge type, so each relation kind keeps its own meaning.
  • I3. Two branches are trained independently and combined only at the probability level with one tunable weight. This keeps each branch intact and makes the contribution of each model easy to measure.
  • I4. A document's difficulty is set by its hardest sentence (max pooling) rather than by an average. This reflects the idea that one barrier sentence can block comprehension.
  • I5. Evaluation for ordinal labels reports several measures together: QWK, exact accuracy, accuracy within ±1, distance, and accuracy after collapsing to 7, 5 or 3 levels. This shows whether errors are near misses or gross mistakes.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): After lemmatization, the en crate could attach graded-difficulty and frequency attributes to each lemma, giving later stages an explainable lexical-difficulty signal.
  • I2 (world): The world graph could keep typed relations (contains-lemma, co-occurs, belongs-to-domain) distinct when propagating information, rather than flattening them into one neighbor set.
  • I3 (learn): The perceptron parser and gparse could be combined by weighted late fusion of their arc or label distributions, so each witness remains separately auditable.
  • I4 (answer): When grading school texts or fables for a student reader, Mova could report the hardest sentence as the document's difficulty and point to it as the explanation.
  • I5 (learn): Evaluations of any graded output, such as difficulty or confidence tiers, could report tolerance-based and coarsened accuracies alongside exact match, so near misses can be told apart from real failures.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓German University in CairoCairo · Egypt6 works · 74 citations🎓Universität UlmUlm · Germany4 works · 89 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Passant Elchafei, Mayar Osama, Mohamad Rageh and Mervat Abu-Elkheir. 2025. GNNinjas at BAREC Shared Task 2025: Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.36

@inproceedings{elchafei2025gnninjas,
  title = {{GNNinjas at BAREC Shared Task 2025: Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction}},
  author = {Passant Elchafei, Mayar Osama, Mohamad Rageh and Mervat Abu-Elkheir},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.36},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.36}
}

References

15 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhamed Al Khalil, Nizar Habash, and Zhengyang Jiang. 2020. A large-scale leveled readability lexicon for Standard Arabic. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3053–3062, Marseille, France. European Language Resources Association.
  2. Abdel-Karim Al-Tamimi, Manar Jaradat, Nuha Aljarrah, and Sahar Ghanim. 2014. Aari: Automatic arabic readability index. International Arab Journal of Information Technology, 11:370–378. find
  3. Bashar Alhafni, Reem Hazim, Juan David Pineros Liberato, Muhamed Al Khalil, and Nizar Habash. 2024. The SAMER Arabic text simplification corpus. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16079–16093, Torino, Italia. ELRA and ICCL.
  4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  5. Ibtehal Baazeem, Hend Al-Khalifa, and Abdulmalik Al-Salman. 2025. Araeyebility: Eye-tracking data for arabic text readability. Computation, 13(5). ↗
  6. Violetta Cavalli-Sforza, Hind Saddiki, and Naoual Nassiri. 2018. Arabic readability research: Current state and future directions. Procedia Computer Science, 142:38–49. Arabic Computational Linguistics. modeling. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 55–66, Bangkok, Thailand. Association for Computational Linguistics. find
  7. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  8. Qi Sun, Kun Zhang, Kun Huang, Tiancheng Xu, Xun Li, and Yaodi Liu. 2023. Document-level relation extraction with two-stage dynamic graph attention networks. Knowledge-Based Systems, 267:110428. ↗
  9. Mo El-Haj, Sultan Almujaiwel, Damith Premasiri, Tharindu Ranasinghe, and Ruslan Mitkov. 2024. DARES: Dataset for Arabic readability estimation of school materials. In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context @ LREC-COLING 2024, pages 103–113, Torino, Italia. ELRA and ICCL.
  10. Khalid N. Elmadani, Bashar Alhafni, Hanada Taha, and Nizar Habash. 2025a. BAREC shared task 2025 on Arabic readability assessment. In Proceedings of the Third Arabic Natural Language Processing Conference, Suzhou, China. Association for Computational Linguistics.
  11. Khalid N. Elmadani, Nizar Habash, and Hanada Taha-Thomure. 2025b. A large and balanced corpus for fine-grained arabic readability assessment. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16376–16400, Vienna, Austria. Association for Computational Linguistics.
  12. William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, volume 30, pages 1024–1034. Curran Associates, Inc. ↗
  13. Reem Hazim, Hind Saddiki, Bashar Alhafni, Muhamed Al Khalil, and Nizar Habash. 2022. Arabic word-level readability visualization for assisted text simplification. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 242–249, Abu Dhabi, UAE. Association for Computational Linguistics.
  14. Joseph Marvin Imperial and Ekaterina Kochmar. 2023. Automatic readability assessment for closely related languages. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5371–5386, Toronto, Canada. Association for Computational Linguistics. ↗
  15. Juan Liberato, Bashar Alhafni, Muhamed Khalil, and Nizar Habash. 2024. Strategies for Arabic readability find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elchafei-2025-gnninjas-barec-shared-task-2025/