Elamur Research⌕
☰
home / papers / 2025 / Pixels at BAREC Shared Task 2025: Visual Arabic Readability Assessment

Pixels at BAREC Shared Task 2025: Visual Arabic Readability Assessment

Paper · 2025 · Ben Sapirstein

Summary

This paper describes a BAREC 2025 Shared Task submission (Sentence-level Strict track) that assigns Arabic sentences to one of 19 readability levels without a conventional tokenizer. Each sentence is rendered as an image in Noto Sans Arabic and fed to the PIXEL Vision Transformer, which reads the image as 16×16 patches. A linear classification head trained with cross-entropy sits on top. The motivation is that Arabic's rich morphology, inconsistent spelling and ambiguity without diacritics cause vocabulary sparsity for token-based models.

The author compares two kinds of input variation:

  • Orthographic: normalization, dediacritization, MLE-based diacritization, and the Buckwalter and HSB transliterations.
  • Morphological: D3TOK clitic segmentation, with ASCII markers, tatweel or spaces as visible boundaries.

Transliterated input beats native script, with HSB plus diacritics reaching 69.3% QWK. Diacritics help and segmentation gives modest gains. The official blind-test score was 68.4% QWK, well below the AraBERTv2 baseline at 76.2%. The work still shows that a pixel-based model can serve as a test bed for comparing text representations.

Key points

  • The data is the BAREC corpus: 1,922 documents, 69,441 sentences, over 1M words and 19 levels from kindergarten to postgraduate. The model is the English-pretrained PIXEL-base, fine-tuned for classification.

  • On orthographic variants, the best QWK by script was:

    • HSB diacritized: 69.3%
    • Buckwalter diacritized: 68.0%
    • Arabic default: 66.5%

    This is a "script gap" of about 3–4 points. Diacritics added 1.3 points for Buckwalter and 2.4 for HSB, but their effect on Arabic script was mixed.

  • On morphological segmentation, QWK was:

    • D3TOK with tatweel markers: 67.4%
    • D3TOK with space markers: 67.0%
    • Unsegmented words: 66.3%
    • ASCII +/+ markers: 65.4%, which fell below the unsegmented baseline

  • On the official blind test, PIXEL reached 68.4% QWK, 41.5% accuracy and an MAE of 1.6. AraBERTv2 (WORD) scored 76.2% QWK, 51.1% accuracy and an MAE of 1.31.

  • The author names two limitations. English pretraining likely favours Latin glyphs, and the default rendering sometimes pushed Arabic diacritics outside the image.

Interesting ideas

  • I1. The model is used as a probe: identical training is run on many systematically altered versions of the same input, and score differences show which surface information actually carries signal.
  • I2. How a morpheme boundary is marked matters, not only whether it is marked: script-native or space markers helped, while foreign ASCII symbols hurt.
  • I3. HSB transliteration maps letter variants to visually similar Latin glyphs while keeping every distinction recoverable. It normalizes the look without discarding information.
  • I4. Diacritics predicted by an automatic disambiguator are injected into the input, so later stages get disambiguation cues they would otherwise have to infer.
  • I5. Ordinal graded labels are scored with QWK and ±1 accuracy instead of plain accuracy, so near misses are penalized less than distant ones.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova's evaluation could train or run the same component on controlled input variants, such as normalized versus raw punctuation or with and without lemma hints. This would measure which representation choices matter, in the same spirit as negative controls.
  • I2 (tokens): When the en tokenizer splits clitics and contractions (e.g. "can't", possessive "'s"), several boundary encodings could be compared for downstream tagging. A naive marker should not be assumed neutral.
  • I3 (tokens): Typographic variants such as quote and apostrophe forms, dashes and ligatures could be mapped to a shared canonical form plus a recoverable variant tag. Normalized lookup then stays fully explainable and reversible.
  • I4 (tree): The tagger's resolved morphology, for example noun-versus-verb readings of ambiguous English forms, could be passed explicitly into gparse or perceptron parser features. Each enriched variant's effect on LAS would then be checked.
  • I5 (answer): When Mova grades reading levels or partial correctness on school-course tests, distance-aware metrics would give a fairer, more informative picture than exact-match scoring.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Ben Sapirstein. 2025. Pixels at BAREC Shared Task 2025: Visual Arabic Readability Assessment. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.48

@inproceedings{sapirstein2025pixels,
  title = {{Pixels at BAREC Shared Task 2025: Visual Arabic Readability Assessment}},
  author = {Ben Sapirstein},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.48},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.48}
}

References

17 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhamed Al Khalil, Nizar Habash, and Zhengyang Jiang. 2020. A large-scale leveled readability lexicon for Standard Arabic. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3053–3062, Marseille, France. European Language Resources Association.
  2. Bashar Alhafni, Reem Hazim, Juan David Pineros Liberato, Muhamed Al Khalil, and Nizar Habash. 2024. The SAMER Arabic text simplification corpus. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16079–16093, Torino, Italia. ELRA and ICCL.
  3. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  4. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, and 1 others. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. System Demonstrations, pages 242–249, Abu Dhabi, UAE. Association for Computational Linguistics. ↗
  5. Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2021. Masked autoencoders are scalable vision learners. Preprint, arXiv:2111.06377. ↗
  6. Salam Khalifa, Nasser Zalmout, and Nizar Habash. 2016. Yamama: Yet another multi-dialect arabic morphological analyzer. In Proceedings of COLING 2016, the 26th international conference on computational linguistics: system demonstrations, pages 223–227.
  7. Juan Liberato, Bashar Alhafni, Muhamed Khalil, and Nizar Habash. 2024. Strategies for Arabic readability modeling. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 55–66, Bangkok, Thailand. Association for Computational Linguistics.
  8. William H DuBay. 2004. The principles of readability. Online submission. find
  9. Khalid N. Elmadani, Bashar Alhafni, Hanada Taha, and Nizar Habash. 2025a. BAREC shared task 2025 on Arabic readability assessment. In Proceedings of the Third Arabic Natural Language Processing Conference, Suzhou, China. Association for Computational Linguistics.
  10. Khalid N. Elmadani, Nizar Habash, and Hanada Taha-Thomure. 2025b. A large and balanced corpus for fine-grained Arabic readability assessment. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16376–16400, Vienna, Austria. Association for Computational Linguistics.
  11. Nizar Habash, Abdelhadi Soudi, and Timothy Buckwalter. 2007. On Arabic Transliteration. In Abdelhadi Soudi, Antal Van Den Bosch, and Günter Neumann, editors, Arabic Computational Morphology, volume 38, pages 15–22. Springer Netherlands, Dordrecht. Series Title: Text, Speech and Language Technology. ↗
  12. Nizar Habash, Hanada Taha-Thomure, Khalid Elmadani, Zeina Zeino, and Abdallah Abushmaes. 2025. Guidelines for fine-grained sentence-level Arabic readability annotation. In Proceedings of the 19th Linguistic Annotation Workshop (LAW-XIX-2025), pages 359–376, Vienna, Austria. Association for Computational Linguistics.
  13. Nizar Y Habash. 2010. Introduction to Arabic natural language processing. Morgan & Claypool Publishers. ↗
  14. Reem Hazim, Hind Saddiki, Bashar Alhafni, Muhamed Al Khalil, and Nizar Habash. 2022. Arabic word-level readability visualization for assisted text simplification. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Alberto Muñoz-Ortiz, Verena Blaschke, and Barbara Plank. 2024. Evaluating pixel language models on non-standardized languages. arXiv preprint arXiv:2412.09084. ↗
  15. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. Camel tools: An open source python toolkit for arabic natural language processing. In Proceedings of the twelfth language resources and evaluation conference, pages 7022–7032.
  16. Phillip Rust, Jonas F Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott. 2022. Language modelling with pixels. arXiv preprint arXiv:2207.06991. ↗
  17. Hanada Taha-Thomure. 2017. Arabic Language Text  ñ J‚Ó J ’ JË é£ @XA Jë Q K AªÓ Leveling ( HAK  K QªË@ ñ’ JË@ ). Educational Book House ( P@X éJ . ©K PñJË@ð Qå„ JÊË ø ñK. QË@ H. AJºË@ ). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sapirstein-2025-pixels-barec-shared-task-2025/