Elamur Research⌕
☰
home / papers / 2024 / RoBERT2VecTM: A Novel Approach for Topic Extraction in Islamic Studies

RoBERT2VecTM: A Novel Approach for Topic Extraction in Islamic Studies

Paper · 2024 · Sania Aftar, Luca Gagliardelli, Amina El Ganadi et al.

Summary

The paper addresses topic discovery in Hadith collections. Classical Arabic morphology is rich, and most topic-modeling tooling assumes English-like languages. The authors take more than 14,940 Ahadith from Sahih Al-Bukhari and Sahih Muslim and keep only the Matn, the actual content, dropping the chain of narrators. The Matn is then cleaned in several steps: diacritics are stripped, characters are normalized, stopwords are removed and words are lemmatized with Farasa. Their model, RoBERT2VecTM, keeps the CombinedTM architecture (a Contextualized Topic Model built on ProdLDA) but changes its input embedding. The usual SBERT vector is replaced by a hybrid representation that merges a document-level Doc2Vec vector with RoBERTa token embeddings, then compresses the result with an autoencoder. On coherence (CV score), the hybrid beats CombinedTM with SBERT, RoBERTa alone, or Doc2Vec alone. The resulting topics can be grouped into recognizable Islamic domains. The authors also report that lemmatization and stopword removal strongly affect topic quality. The work belongs to the Digital Maktaba / ITSERR effort to automate metadata extraction for non-Latin scripts.

Key points

  • Corpus: Unicode editions of Sahih Al-Bukhari and Sahih Muslim, more than 14,940 Ahadith in total.

  • Isnad/Matn split: custom regular expressions separate the narrator chain from the content, and only the Matn is analyzed.

  • Preprocessing pipeline: diacritic removal, Unicode normalization, stopword filtering with nltk.corpus, and lemmatization with Farasa.

  • Embedding construction:

    • Doc2Vec produces a 128-dim document vector.
    • RoBERTa produces 768-dim token vectors, which UMAP reduces to 128 dimensions.
    • The combined vector passes through an autoencoder with a 64-unit ReLU bottleneck, trained for 200 epochs with Adam and MSE loss.
    • The final representation replaces SBERT in CombinedTM.

  • CV coherence (Table 4):

    EmbeddingCV score
    RoBERT2VecTM (Doc2Vec + RoBERTa)0.61
    SBERT0.55
    RoBERTa only0.54
    Doc2Vec only0.35

    The comparative analysis with other approaches (Table 3) uses 50 topics.

Interesting ideas

  • I1. Before any semantic processing, the authors use surface-pattern rules to cut away the predictable "who transmitted it" frame (the Isnad), so content and provenance are handled separately.
  • I2. The hybrid is checked against each of its parts alone (Doc2Vec only, RoBERTa only) and against the standard SBERT setup, which shows the gain comes from the combination itself.
  • I3. The paper finds that lemmatization and stopword choices change topic quality noticeably, so preprocessing is treated as a factor to test rather than a fixed default.
  • I4. A document-level vector and token-level contextual vectors are projected into a shared size and fused, giving a representation with both a global and a local view of the text.
  • I5. The unsupervised clusters become interpretable because each induced topic is linked to a domain category that experts would recognize, for instance "Islamic Law and Society" or the category covering "Prayer and Worship".

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): reporting chains like "X said that Y told us that…" could be parsed as explicit nested attribution frames, so the embedded proposition is passed to the world separately from its source chain.
  • I2 (learn): any new combined component (e.g., merged witness parsers) could be evaluated against each member alone, so improvements are attributed to the merge rather than to one strong member.
  • I3 (tokens): ablate the lemmatizer and morphology settings in the en crate and measure the effect on downstream tree and frame accuracy, not just on token-level scores.
  • I4 (tree): gparse could receive a cheap sentence- or document-level context feature next to its token encodings, which might help resolve attachments that depend on the broader context.
  • I5 (world): clusters induced from texts could be anchored to WordNet/Wikidata ontology nodes, so every grouping carries an explainable label.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Modena and Reggio EmiliaModena · Italy3 works · 3 citations🎓University of PalermoPalermo · Italy2 works · 3 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 405 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Topic modeling · 47 works

Cite

Cite as Sania Aftar, Luca Gagliardelli, Amina El Ganadi, Federico Ruozzi and Sonia Bergamaschi. 2024. RoBERT2VecTM: A Novel Approach for Topic Extraction in Islamic Studies. Findings of the Association for Computational Linguistics: EMNLP 2024.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.findings-emnlp.534

@inproceedings{aftar2024robert2vectm,
  title = {{RoBERT2VecTM: A Novel Approach for Topic Extraction in Islamic Studies}},
  author = {Sania Aftar, Luca Gagliardelli, Amina {El Ganadi}, Federico Ruozzi and Sonia Bergamaschi},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.534},
  url = {https://doi.org/10.18653/v1/2024.findings-emnlp.534}
}

References

43 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hammam M. Abdelaal, Berihan R. Elemary, and Hassan A. Youness. 2019. Classification of hadith according to its content based on supervised learning algorithms. IEEE Access, 7:152379–152387. ↗
  2. Aly Abdelrazek, Yomna Eid, Eman Gawish, Walaa Medhat, and Ahmed Hassan. 2023. Topic modeling algorithms and applications: A survey. Information Systems, 112:102131. ↗
  3. Sania Aftar, Luca Gagliardelli, Amina El Ganadi, Federico Ruozzi, and Sonia Bergamaschi. 2024. A novel methodology for topic identification in hadith. In Proceedings of the 20th Conference on Information and Research science Connecting to Digital and Library science (formerly the Italian Research Conference on Digital Libraries), Bressanone, Brixen, Italy - 22-23 February 2024, volume 3643 of CEUR Workshop Proceedings, pages 117–125. CEUR-WS.org. find
  4. Nikolaos Aletras and Mark Stevenson. 2013. Evaluating topic coherence using distributional semantics. In Proceedings of the 10th international conference on computational semantics (IWCS 2013)–Long Papers, pages 13–22.
  5. Yousif A Alhaj, Jianwen Xiang, Dongdong Zhao, Mohammed AA Al-Qaness, Mohamed Abd Elaziz, and Abdelghani Dahou. 2019. A study of the effects of stemming strategies on arabic document classification. IEEE access, 7:32664–32671. ↗
  6. Mohammad O Alhawarat, Hikmat Abdeljaber, and Anwer Hilal. 2021. Effect of stemming on text similarity for arabic language at sentence level. PeerJ Computer Science, 7:e530. ↗
  7. Sanjeev Arora, Rong Ge, and Ankur Moitra. 2012. Learning topic models–going beyond svd. In 2012 IEEE 53rd annual symposium on foundations of computer science, pages 1–10. IEEE. ↗
  8. Mohammad Hamid Asnawi, Anindya Apriliyanti Pravi-tasari, Tutut Herawan, and Triyani Hendrawati. 2023. The combination of contextualized topic model and mpnet for user feedback topic modeling. IEEE Access, 11:130272–130286. ↗
  9. Aqil M Azmi, Abdulaziz O Al-Qabbany, and Amir Hussain. 2019. Computational and natural language processing based studies of hadith literature: a survey. Artificial Intelligence Review, 52:1369–1414. ↗
  10. Sonia Bergamaschi, Stefania De Nardis, Riccardo Martoglia, Federico Ruozzi, Luca Sala, Matteo Vanzini, and Riccardo Amerigo Vigliermo. 2022. Novel perspectives for the management of multilingual and multialphabetic heritages through automatic knowledge extraction: The digitalmaktaba approach. Sensors, 22(11):3995. ↗
  11. Muzafar Rasool Bhat, Majid A Kundroo, Tanveer A Tarray, and Basant Agarwal. 2020. Deep lda: A new way to topic model. Journal of Information and Optimization Sciences, 41(3):823–834. ↗
  12. Federico Bianchi, Silvia Terragni, and Dirk Hovy. 2020a. Pre-training is a hot topic: Contextualized document embeddings improve topic coherence. arXiv preprint arXiv:2004.03974. ↗
  13. Khitam Mahmoud Abdalla Jbara, Azzam T Sleit, and Bassam H Hammo. 2009. Knowledge discovery in Al-Hadith using text classification algorithm. University of Jordan. find
  14. Hugo Larochelle and Stanislas Lauly. 2012. A neural autoregressive topic model. Advances in Neural Information Processing Systems, 25. find
  15. Riccardo Martoglia, Sonia Bergamaschi, Federico Ruozzi, Matteo Vanzini, Luca Sala, and Riccardo Amerigo Vigliermo. 2023. Knowledge extraction, management and long-term preservation of non-latin cultural heritages - digital maktaba project presentation. In Proceedings of the 19th The Conference on Information and Research science Connecting to Digital and Library science, IRCDL 2023, Bari, Italy, February 23-24, 2023, volume 3365 of CEUR Workshop Proceedings, pages 153–161. CEUR-WS.org. find
  16. Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426. ↗
  17. Mohammed Mghari, Omar Bouras, and Abdelaaziz El Hibaoui. 2022. Sanadset 650k: Data on hadith narrators. Data in Brief, 44:108540. ↗
  18. Federico Bianchi, Silvia Terragni, Dirk Hovy, Debora Nozza, and Elisabetta Fersini. 2020b. Cross-lingual contextualized topic models with zero-shot learning. arXiv preprint arXiv:2004.07737. ↗
  19. Yishu Miao, Lei Yu, and Phil Blunsom. 2016. Neural variational inference for text processing. In International conference on machine learning, pages 1727–1736. PMLR. ↗
  20. Adji B Dieng, Francisco JR Ruiz, and David M Blei. 2020. Topic modeling in embedding spaces. Transactions of the Association for Computational Linguistics, 8:439–453. ↗
  21. Asha Rani Mishra and VK Panchal. 2022. A novel approach to capture the similarity in summarized text using embedded model. International Journal on Smart Sensing and Intelligent Systems, 15(1):1–20. ↗
  22. Kamel Gaanoun and Mohammed Alsuhaibani. 2022. Fabricated hadith detection: A novel matn-based approach with transformer language models. IEEE Access, 10:113330–113342. ↗
  23. Hamdy Mubarak. 2017. Build fast and accurate lemmatization for arabic. arXiv preprint arXiv:1710.06700. ↗
  24. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018).
  25. Maarten R. Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. ArXiv, abs/2203.05794. ↗
  26. Rahul Kumar Gupta, Ritu Agarwalla, Bukya Hemanth Naik, Joythish Reddy Evuri, Apil Thapa, and Thoudam Doren Singh. 2022. Prediction of research trends using lda based topic modeling. Global Transitions Proceedings, 3(1):298–304. ↗
  27. Nassera HABBAT, Houda ANOUN, and Larbi HAS-SOUNI. 2021. Arabertopic: A neural topic modeling approach for news extraction from arabic facebook pages using pre-trained bert transformer model. International Journal Of Computing and Digital System, 14. find
  28. Moath Mustafa Ahmad Najeeb. 2016. Xml database for hadith and narrators. American Journal of Applied Sciences, 13(1):55–63. ↗
  29. Moath Mustafa Ahmad Najeeb. 2020. A novel hadith processing approach based on genetic algorithms. IEEE Access, 8:20233–20244. ↗
  30. Moath Mustafa Ahmad Najeeb. 2021. Towards a deep leaning-based approach for hadith classification. European Journal of Engineering and Technology Research, 6(3):9–15. ↗
  31. Mohammed Naji Al-Kabi, Ghassan Kanaan, Riyad Al-Shalabi, Saja I Al-Sinjilawi, and Ronza S Al-Mustafa. 2005. Al-hadith text classifier. Journal of Applied Sciences, 5(3):584–587. ↗
  32. PNE Nohuddin, Z Zainol, KF Chao, M Tarhamizwan, S Marzukhi, and A Nordin. 2016. Keyword based clustering technique for collections of hadith chapters. International Journal on Islamic Applications in Computer Science And Technologies–IJASAT, 4(3):11–18. find
  33. Michael Röder, Andreas Both, and Alexander Hinneburg. 2015. Exploring the space of topic coherence measures. In Proceedings of the eighth ACM international conference on Web search and data mining, pages 399–408.
  34. Nur Aqilah Paskhal Rostam and Nurul Hashimah Ahamed Hassain Malim. 2021. Text categorisation in quran and hadith: Overcoming the interrelation challenges using machine learning and term weighting. Journal of King Saud University-Computer and Information Sciences, 33(6):658–667. find
  35. Joseph Schacht. 1967. The origins of Muhammadan jurisprudence. Oxford University Press. find
  36. Mohammad Najib SR, Nurazzah Abd Rahman, N Alias, MN Alias, et al. 2017. Comparative study of machine learning approach on malay translated hadith text classification based on sanad. In MATEC Web of Conferences. EDP Sciences. find
  37. Akash Srivastava and Charles Sutton. 2017. Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488. ↗
  38. Silvia Terragni, Elisabetta Fersini, Bruno Giovanni Galuzzi, Pietro Tropeano, and Antonio Candelieri. 2021. Octis: Comparing and optimizing topic models is simple! In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 263–270. find
  39. Felipe Viegas, Sérgio Canuto, Christian Gomes, Washington Luiz, Thierson Rosa, Sabir Ribas, Leonardo Rocha, and Marcos André Gonçalves. 2019. Cluwords: exploiting semantic word clustering representation for enhanced topic modeling. In Proceedings of the twelfth ACM international conference on web search and data mining, pages 753–761. find
  40. Zongda Wu, Li Lei, Guiling Li, Hui Huang, Chengren Zheng, Enhong Chen, and Guandong Xu. 2017. A topic modeling based approach to novel document automatic summarization. Expert Systems with Applications, 84:12–23. ↗
  41. Qing Xie, Xinyuan Zhang, Ying Ding, and Min Song. 2020. Monolingual and multilingual topic analysis using lda and bert embeddings. Journal of Informetrics, 14(3):101055. ↗
  42. Rahmadi Yotenka, Sekti Kartika Dini, Achmad Fauzan, and Atina Ahdika. 2022. Exploring the relationship between hadith narrators in book of bukhari through spade algorithm. MethodsX, 9:101850. ↗
  43. Xiaowei Zhao, Deqing Wang, Zhengyang Zhao, Wei Liu, Chenwei Lu, and Fuzhen Zhuang. 2021. A neural topic model with word vectors and entity vectors for short texts. Information Processing & Management, 58(2):102455. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aftar-2024-robert2vectm-novel-approach-topic-extraction/