Elamur Research⌕
☰
home / papers / 2026 / Extractive Summarization for Arabic Documents Using SAraBERT with a Semantic Siamese Similarity Evaluation Metric

Extractive Summarization for Arabic Documents Using SAraBERT with a Semantic Siamese Similarity Evaluation Metric

Summary

Research on extractive summarization has focused mostly on English. Arabic is harder to process because of its rich derivational morphology, missing diacritics and lack of capitalization. The authors propose SAraBERT, which adapts AraBERT for extractive summarization. Each sentence gets its own [CLS] marker, sentences receive alternating interval segment embeddings, and inter-sentence transformer layers on top score each sentence for inclusion in the summary. Training data has no large native Arabic source, so they machine-translate CNN/Daily Mail into Modern Standard Arabic. They evaluate on the Kalimat dataset. They also propose Semantic Siamese Similarity (SSS), a metric that combines contextual embedding similarity with exact lexical match, because n-gram scores alone can miss whether a summary covers the document's ideas. Using BLEU, ROUGE and SSS, the authors report that SAraBERT beats earlier Arabic models and reaches state-of-the-art results. The work matters for low-resource summarization: it shows how a translated benchmark and an architecture that knows sentence boundaries can fill the gap.

Key points

  • Each sentence in SAraBERT is wrapped in [CLS] … [SEP]. Sentences alternate between segment embeddings E_A and E_B by odd/even position. The vector at each sentence's [CLS] is that sentence's representation, and it is labeled y ∈ {0,1}.
  • Training corpus: CNN/Daily Mail translated into Modern Standard Arabic, intended for extractive summarization and Question Answering. It is available on request.
  • Test data: the Kalimat dataset. Metrics: BLEU, ROUGE and the new Semantic Siamese Similarity (SSS).
  • To check the translated data, a sample Arabic passage is scored with the Osman readability metric and gets 88.01 ("easy to read").
  • Prior hybrid work (Reda et al.: AraBERT extraction followed by MT5) was tested only on ESAC, which has 153 articles. It reached precision 53%, recall 55% and F1 49%.

Interesting ideas

  • I1. A boundary marker at the start of every sentence, plus segment embeddings that alternate between neighbours, lets one encoder pass over a whole document while still producing a separate vector per sentence. This avoids encoding sentences in isolation. — For Mova (tree): gparse could parse a whole paragraph in one pass with per-sentence markers and alternating segment ids, so cross-sentence context reaches each sentence's tree without merging sentence boundaries.
  • I2. Framing summarization as a 0/1 decision per sentence keeps the output traceable to the source text. Every chosen sentence is an original span, never a rewrite. — For Mova (answer): the fable and tale QA readers could first label which sentences support an answer, then reason only over those. The answer then cites the exact sentences it came from.
  • I3. SSS combines a meaning-level embedding score with a strict exact-match score. Paraphrases still earn credit, and real lexical overlap is still rewarded. — For Mova (back): the text → graph → text compressor could be scored with a two-part metric like this. A round trip that keeps meaning but changes wording would then not be unfairly penalized.
  • I4. When no native training corpus existed, the authors translated an established English benchmark. They then checked the result with an independent quality signal, a readability score. — For Mova (learn): silver data produced by machine could pass through a separate quality check before training. That check could be readability or agreement statistics, acting as a negative control on the generated data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2608.20964 · arXiv 2608.20964

@misc{deen2026extractive,
  title = {{Extractive Summarization for Arabic Documents Using SAraBERT with a Semantic Siamese Similarity Evaluation Metric}},
  author = {Deen, Sami Shames El and Awad, Mariette},
  year = {2026},
  doi = {10.48550/arxiv.2608.20964},
  eprint = {2608.20964},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2608.20964}
}

References

34 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. H. Jing, “Using hidden markov modeling to decompose human-written summaries,” Computational linguistics, vol. 28, no. 4, pp. 527–543, 2002. find
  2. N. Hajj and M. Awad, “Weighted entropy cortical algorithms for isolated arabic speech recognition,” IEEE, 2013. find
  3. A. B. Al-Saleh and M. E. B. Menai, “Automatic arabic text summarization: a survey,” Artificial Intelligence Review, vol. 45, no. 2, pp. 203–234, 2016. find
  4. K. S. Al Harazin, “Multi-document arabic text summarization,” 2015. find
  5. V. Patil, M. Krishnamoorthy, P. Oke, and M. Kiruthika, “A statistical approach for document summarization,” Department of Computer Engineering Fr. C. Rodrigues Institute of Technology, Vashi, Navi Mumbai, Maharashtra, India, 2004. find
  6. F. Alotaiby, S. Foda, and I. Alkharashi, “New approaches to automatic headline generation for arabic documents,” Journal of Engineering and Computer Innovations, vol. 3, no. 1, pp. 11–25, 2012. find
  7. R. Ferreira, L. de Souza Cabral, R. D. Lins, G. P. e Silva, F. Freitas, G. D. Cavalcanti, R. Lima, S. J. Simske, and L. Favaro, “Assessing sentence scoring techniques for extractive text summarization,” Expert systems with applications, vol. 40, no. 14, pp. 5755–5764, 2013. find
  8. Q. Al-Radaideh and M. Afif, “Arabic text summarization using aggregate similarity,” in International Arab conference on information technology (ACIT2009), Yemen, 2009. find
  9. A. Haboush, M. Al-Zoubi, A. Momani, and M. Tarazi, “Arabic text summarization model using clustering techniques,” World of Computer Science and Information Technology Journal (WCSIT) ISSN, pp. 2221– 0741, 2012. find
  10. F. El-Ghannam and T. El-Shishtawy, “Multi-topic multi-document summarizer,” arXiv preprint arXiv:1401.0640, 2014. ↗
  11. H. N. Fejer and N. Omar, “Automatic arabic text summarization using clustering and keyphrase extraction,” in Proceedings of the 6th International Conference on Information Technology and Multimedia, pp. 293– 298, IEEE, 2014. find
  12. N. M. Hewahi and K. A. Kwaik, “Automatic arabic text summarization system (aatss) based on semantic features extraction,” International Journal of Technology Diffusion (IJTD), vol. 3, no. 2, pp. 12–27, 2012. find
  13. M. El-Haj, U. Kruschwitz, and C. Fox, “Multi-document arabic text summarisation,” in 2011 3rd Computer Science and Electronic Engineering Conference (CEEC), pp. 40–44, IEEE, 2011. find
  14. D. Miller, “Leveraging bert for extractive text summarization on lectures,” arXiv preprint arXiv:1906.04165, 2019. ↗
  15. M. A. Fattah and F. Ren, “Ga, mr, ffnn, pnn and gmm based models for automatic text summarization,” Computer Speech & Language, vol. 23, no. 1, pp. 126–144, 2009. find
  16. R. Belkebir and A. Guessoum, “A supervised approach to arabic text summarization using adaboost,” in New contributions in information systems and technologies, pp. 227–236, Springer, 2015. find
  17. Q. A. Al-Radaideh and D. Q. Bataineh, “A hybrid approach for arabic text summarization using domain knowledge and genetic algorithms,” Cognitive Computation, vol. 10, no. 4, pp. 651–669, 2018. find
  18. A. Nenkova and K. McKeown, “A survey of text summarization techniques,” in Mining text data, pp. 43–76, Springer, 2012. find
  19. D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014. ↗ our card
  20. I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems, pp. 3104–3112, 2014. find
  21. A. M. Rush, S. Chopra, and J. Weston, “A neural attention model for abstractive sentence summarization,” arXiv preprint arXiv:1509.00685, 2015. ↗
  22. R. Nallapati, B. Zhou, and M. Ma, “Classify or select: Neural architectures for extractive document summarization,” arXiv preprint arXiv:1611.04244, 2016. ↗
  23. S. Chopra, M. Auli, and A. M. Rush, “Abstractive sentence summarization with attentive recurrent neural networks,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 93–98, 2016. find our card
  24. M. Al-Maleh and S. Desouki, “Arabic text summarization using deep learning approach,” Journal of Big Data, vol. 7, no. 1, pp. 1–17, 2020. find
  25. W. Antoun, F. Baly, and H. Hajj, “Arabert: Transformer-based model for arabic language understanding,” arXiv preprint arXiv:2003.00104, 2020. ↗ our card
  26. A. M. Abu Nada, E. Alajrami, A. A. Al-Saqqa, and S. S. Abu-Naser, “Arabic text summarization using arabert model using extractive text summarization approach,” 2020. find
  27. Y. Liu, “Fine-tune bert for extractive summarization,” arXiv preprint arXiv:1903.10318, 2019. ↗
  28. G. Inoue, B. Alhafni, N. Baimukan, H. Bouamor, and N. Habash, “The interplay of variant, size, and task type in arabic pre-trained language models,” 2021. find our card
  29. N. Zalmout and N. Habash, “Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for arabic,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 704–713, 2017. find our card
  30. A. Reda, N. Salah, J. Adel, M. Ehab, I. Ahmed, M. Magdy, G. Khoriba, and E. H. Mohamed, “A hybrid arabic text summarization approach based on transformers,” in 2022 2nd International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pp. 56–62, 2022. find
  31. C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out, pp. 74–81, 2004. find
  32. Y. Tang, C. Tran, X. Li, P.-J. Chen, N. Goyal, V. Chaudhary, J. Gu, and A. Fan, “Multilingual translation with extensible multilingual pretraining and finetuning,” arXiv preprint arXiv:2008.00401, 2020. ↗
  33. Z. Barqawi, “Shakkala, arabic text vocalization,” 2017. find
  34. M. El-Haj and P. E. Rayson, “Osman: A novel arabic readability metric,” 2016. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/deen-2026-extractive-summarization-arabic-documents-using/