Elamur Research⌕
☰
home / papers / 2018 / An Enhanced Latent Semantic Analysis Approach for Arabic Document Summarization

An Enhanced Latent Semantic Analysis Approach for Arabic Document Summarization

Paper · 2018 · Kamal Al-Sabahi, Zuping Zhang, Jun Long et al.

Summary

The paper addresses extractive single-document summarization, with Arabic as the main focus. Arabic has far fewer tools and resources than English. The authors start from latent semantic analysis (LSA), which builds a term–sentence matrix and factorizes it with SVD. They name three weaknesses of earlier LSA summarizers: they ignore word order and syntax, the term space is large and noisy, and their sentence selection is crude. They make three changes. A POS tagger filters terms to shrink the matrix. Each term's weight also includes how it is used in four adjacent sentences, which adds local context and eases sparsity. A new selection procedure draws on all three SVD outputs (U, Σ, V) and combines term-level and sentence-level descriptions of each latent topic. It picks a variable number of sentences per topic. The method is tested on LDC Arabic Newswire-a, EASC, DUC2002 and MultiLing MSS 2015, scored with ROUGE-1/2. It reaches the top ROUGE-1 among compared systems on all of them. ROUGE-2 results are mixed on English. This suggests that a carefully engineered classic LSA pipeline can stay competitive for a lower-resource language.

Key points

  • Pipeline stages. The pipeline runs in three steps: building the input matrix, applying SVD, and then choosing the sentences. The changes touch the first stage (POS-based term filtering, an adjacent weight over four neighbouring sentences) and the third (a new selection algorithm). The authors say they are the first to use U, Σ and V together for choosing sentences.

  • DUC2002 setup. DUC2002 has 567 news articles with 100-word reference summaries. For the English runs the POS tagger was not used.

  • DUC2002 results (ROUGE-1 / ROUGE-2).

    • Proposed model: 0.500 / 0.241
    • DUC-best: 0.498 / 0.252
    • Wang2013: 0.472 / 0.261
    • LEAD-3: 0.436 / 0.210
    • Steinberger2004: 0.433 / 0.180

  • MultiLing MSS 2015 setup. The data are Wikipedia articles, 30 per language. The variant RAWEF4ADJ is compared with the best and worst of 23 participating systems.

  • MSS 2015 results.

    LanguageRAWEF4ADJ (R-1 / R-2)Best participant (R-1 / R-2)
    English0.5159 / 0.14410.5038 / 0.1412
    Arabic0.5759 / 0.22580.5741 / 0.2064

    The paper's text says English ROUGE-2 was an exception, although the table shows a small lead.

  • Future work. The authors suggest improving the input-matrix weights with word embeddings such as word2vec or GloVe.

Interesting ideas

  • I1. Filtering terms by part of speech before building the matrix reduces dimensionality and noise, instead of keeping every surface word.
  • I2. A term's weight in a sentence also counts its use in a small window of neighbouring sentences, which brings word order and local discourse into an otherwise unordered matrix.
  • I3. Each latent topic is described both by its terms and by its sentences, and a candidate sentence must fit both views before it is chosen; this guards against picking sentences that score high by accident.
  • I4. The number of sentences taken per topic follows that topic's share of the singular values, rather than one per topic or a fixed cap.
  • I5. The authors note that a comparison was hard to interpret because another paper did not report its ROUGE toolkit options, and they publish their own exact settings.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): tagger output could mark which lemmas count as content-bearing, so later salience or topic statistics are computed only over nouns and verbs that matter.
  • I2 (world): entity and state salience could be smoothed across adjacent sentences, helping decide which referents are the current discourse topic when tracking the world of a story.
  • I3 (back): when compressing a meaning graph to text, a frame could be kept only if both its participating entities and its own centrality support the topic, giving explainable choices of what to keep.
  • I4 (back): a summary or compression budget could be divided among event chains in proportion to their measured weight, so major storylines get more space than side episodes.
  • I5 (learn): every evaluation report should log scorer versions and flags next to the numbers, so regressions and cross-system comparisons stay reproducible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Central South UniversityChangsha · China15 works · 582 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksSummarization · 232 works

Cite

Cite as Kamal Al-Sabahi, Zuping Zhang, Jun Long and Khaled Alwesabi. 2018. An Enhanced Latent Semantic Analysis Approach for Arabic Document Summarization. arXiv:1807.11618 · K. Al-Sabahi, Z. Zhang, J. Long, and K. Alwesabi, "An Enhanced Latent Semantic Analysis Approach for Arabic Document Summarization," Arabian Journal for Science and Engineering, journal article May 05 2018.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1007/s13369-018-3286-z · arXiv 1807.11618

@misc{alsabahi2018enhanced,
  title = {{An Enhanced Latent Semantic Analysis Approach for Arabic Document Summarization}},
  author = {Kamal Al-Sabahi, Zuping Zhang, Jun Long and Khaled Alwesabi},
  year = {2018},
  doi = {10.1007/s13369-018-3286-z},
  eprint = {1807.11618},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/s13369-018-3286-z}
}

References

42 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Binwahlan, M.S.; Salim, N.; Suanmali, L.: Fuzzy swarm diversity hybrid model for text summarization. Inf. Process. Manag. 46(5), 571–588 (2010). https://doi.org/10.1016/j.ipm.2010.03.004 ↗
  2. Khan, K.; Baharudin, B.B.; Khan, A.: Semantic-based unsupervised hybrid technique for opinion targets extraction from unstructured reviews. Arab. J. Sci. Eng. 39(5), 3681–3689 (2014). https://doi.org/10.1007/s13369-014-0990-1 ↗
  3. Imam, I.; Nounou, N.; Hamouda, A.; Khalek, H.A.A.: An ontology-based summarization system for Arabic documents (OSSAD). Int. J. Comput. Appl. 74(17), 38–43 (2013) find
  4. Qumsiyeh, R.; Ng, Y.-K.: Enhancing web search by using query-based clusters and multi-document summaries. Knowl. Inf. Syst. 47(2), 355–380 (2016). https://doi.org/10.1007/s10115-015-0852-5 ↗
  5. Sarkar, D.: Text Summarization. In: Text Analytics with Python: A Practical Real-World Approach to Gaining Actionable Insights from your Data, pp. 217–263. Apress, Berkeley, CA (2016) find
  6. Heu, J.-U.; Qasim, I.; Lee, D.-H.: FoDoSu: multi-document summarization exploiting semantic analysis based on social Folksonomy. Inf. Process. Manag. 51(1), 212–225 (2015). https://doi.org/10.1016/j.ipm.2014.06.003 ↗
  7. Hammo, B.H.: A hybrid arabic text summarization technique based on text structure and topic identification. Int. J. Comput. Process. Lang. 23(1), 39–65 (2011) find
  8. Al Qassem, L.M.; Wang, D.; Al Mahmoud, Z.; Barada, H.; Al-Rubaie, A.; Almoosa, N.I.: Automatic Arabic summarization: a survey of methodologies and systems. Proc. Comput. Sci. 117, 10–18 (2017). https://doi.org/10.1016/j.procs.2017.10.088 ↗
  9. Ferreira, R.; de Souza Cabral, L.; Lins, R.D.; Pereirae Silva, G.; Freitas, F.; Cavalcanti, G.D.C.; Lima, R.; Simske, S.J.; Favaro, L.: Assessing sentence scoring techniques for extractive text summarization. Expert Syst. Appl. 40(14), 5755–5764 (2013). https://doi.org/10.1016/j.eswa.2013.04.023 ↗
  10. Zhu, J.; Jiang, Y.; Li, B.; Sun, M.: Ontology-based automatic summarization of web document. Int. J. Adv. Comput. Technol. 4(14), 289–309 (2012). https://doi.org/10.4156/ijact.vol4.issue14.34 ↗
  11. Jeong, H.; Ko, Y.; Seo, J.: How to improve text summarization and classification by mutual cooperation on an integrated framework. Expert Syst. Appl. 60, 222–233 (2016). https://doi.org/10.1016/j. eswa.2016.05.001 ↗
  12. Isonuma, M.; Fujino, T.; Mori, J.; Matsuo, Y.; Sakata, I.: Extractive Summarization Using multi-task learning with document classification. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (2017), pp. 2091–2100 find
  13. Triantafillou, E.; Kiros, J.R.; Urtasun, R.; Zemel, R.: Towards generalizable sentence embeddings. In: Proceedings of the 1st Workshop on Representation Learning for NLP, Berlin, Germany, 2016, pp. 239–248 find
  14. Wu, Z.; Lei, L.; Li, G.; Huang, H.; Zheng, C.; Chen, E.; Xu, G.: A topic modeling based approach to novel document automatic summarization. Expert Syst. Appl. 84, 12–23 (2017). https://doi.org/10.1016/j.eswa.2017.04.054 ↗
  15. Gong, Y.; Liu, X.: Generic text summarization using relevance measure and latent semantic analysis. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 19–25, ACM (2001) find
  16. Steinberger, J.; Jezek, K.: Using latent semantic analysis in text summarization and summary evaluation. In: Proceedings of the ISIM’04, pp. 93–100 (2004) find
  17. Murray, G.; Renals, S.; Carletta, J.: Extractive summarization of meeting recordings. In: INTERSPEECH (2005) find
  18. Ozsoy, M.G.; Alpaslan, F.N.; Cicekli, I.: Text summarization using latent semantic analysis. J. Inf. Sci. 37(4), 405–417 (2011). https://doi.org/10.1177/0165551511408848 ↗
  19. Wang, Y.; Ma, J.: A comprehensive method for text summarization based on latent semantic analysis. In: Zhou, G., Li, J., Zhao, D., Feng, Y. (eds.) Natural Language Processing and Chinese Computing: Second CCF Conference, NLPCC 2013, Chongqing, China, November 15–19, 2013, Proceedings, pp. 394–401. Springer, Berlin (2013) find
  20. Shen, Y.; He, X.; Gao, J.; Deng, L.; Gr, #233, Mesnil, g.: A latent semantic model with convolutional-pooling structure for information retrieval. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, Shanghai, China, pp. 101–110. ACM, 2661935 (2014) find
  21. Al-Saleh, A.B.; Menai, M.E.B.: Automatic Arabic text summarization: a survey. Artif. Intell. Rev. 45(2), 203–234 (2016). https://doi.org/10.1007/s10462-015-9442-x ↗
  22. El-Haj, M.; Kruschwitz, U.; Fox, C.: Multi-document Arabic text summarisation. In: 2011 3rd Computer Science and Electronic Engineering Conference (CEEC), pp. 40–44 (2011) find
  23. Froud, H.; Lachkar, A.; Ouatik, S.A.: Arabic text summarization based on latent semantic analysis to enhance Arabic documents clustering. arXiv preprint arXiv:1302.1612 (2013) ↗
  24. Ba-Alwi, F.M.; Gaphari, G.H.; Al-Duqaimi, F.N.: Arabic text summarization using latent semantic analysis. Br. J. Appl. Sci. Technol. 10(2), 1–14 (2015) find
  25. Althobaiti, M.; Kruschwitz, U.; Poesio, M.: AraNLP: A Java-Based Library for the Processing of Arabic Text, pp. 4134–4138. University of Essex, Colchester (2013)
  26. Farghaly, A.; Shaalan, K.: Arabic natural language processing: challenges and solutions. ACM Trans. Asian Lang. Inf. Process. 8(4), 1–22 (2009). https://doi.org/10.1145/1644879.1644881 ↗
  27. Nadera, B.: The Arabic natural language processing: introduction and challenges. Int. J. Engl. Lang. Transl. Stud. 2(3), 106–112 (2014) find
  28. El-Khair, I.A.: Effects of stop words elimination for Arabic information retrieval: a comparative study. Int. J. Comput. Inf. Sci. 4(3), 119–133 (2006) find
  29. Taghva, K.; Elkhoury, R.; Coombs, J.: Arabic stemming without a root dictionary. In: International Conference on Information Technology: Coding and Computing (ITCC’05)—Volume II, 4–6, vol. 151, pp. 152–157 (2005) find
  30. Zahedi, M.-H.; Kahani, M.: SREC: discourse-level semantic relation extraction from text. Neural Comput. Appl. 23, 1573–1582 (2013) find
  31. Köprü, S.: An efficient part-of-speech tagger for Arabic. In: Gelbukh, A.F. (ed.) Computational Linguistics and Intelligent Text Processing: 12th International Conference, CICLing 2011, Tokyo, Japan, February 20–26, 2011. Proceedings, Part I, pp. 202–213. Springer, Berlin (2011) find
  32. Manning, C.D.; Raghavan, P.; Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008) find
  33. Lee, J.-H.; Park, S.; Ahn, C.-M.; Kim, D.: Automatic generic document summarization based on non-negative matrix factorization. Inf. Process. Manag. 45(1), 20–34 (2009). https://doi.org/10.1016/j.ipm.2008.06.002 ↗
  34. Kalman, D.: A singularly valuable decomposition: the SVD of a matrix. Coll. Math. J. 27(1), 2–23 (1996) find
  35. Menéndez, H.D.; Plaza, L.; Camacho, D.: A Genetic Graph-Based Clustering Approach to Biomedical Summarization, pp. 978-1-4503-1850-1. ACM (2013) find
  36. Jing, H.; Barzilay, R.; McKeown, K.; Elhadad, M.: Summarization evaluation methods: experiments and analysis. In: AAAI Symposium on Intelligent Summarization, pp. 51–59 (1998) find
  37. Sobh, I.; Darwish, N.; Fayek, M.: Evaluation Approaches for an Arabic Extractive Generic Text Summarization System, pp. 150– 155. The Research and Development International Company, RDI, Cairo University, Giza, Egypt. http://www.rdi-eg.com (2013) ↗
  38. Lin, C.-Y.: Rouge: a package for automatic evaluation of summaries. In: Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, Barcelona, Spain (2004)
  39. El-Haj, M.: Essex Arabic summaries corpus (EASC). In: Text Analysis Conference (TAC) 2011, vol. 2016, vol. 10/03/2015. Lancaster University (2011) find
  40. El-Haj, M.; Kruschwitz, U.; Fox, C.: Using mechanical turk to create a corpus of Arabic summaries. In: Proceedings of the International Conference on Language Resources and Evaluation. European Language Resources Association (2010) find
  41. Lin, C.-Y.; Hovy, E.: Automatic evaluation of summaries using N-gram co-occurrence statistics. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology— Volume 1, Edmonton, Canada 2003, pp. 71–78. Association for Computational Linguistics, 1073465
  42. Giannakopoulos, G.; Kubina, J.; Conroy, J.; Steinberger, J.; Favre, B.; Kabadjov, M.; Kruschwitz, U.; Poesio, M.: Multiling 2015: multilingual summarization of single and multi-documents, on-line fora, and call-center conversations. In: Proceedings of SIGDIAL, Prague, pp. 270–274 (2015) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/al-sabahi-2018-enhanced-latent-semantic-analysis-approach/