Elamur Research⌕
☰
home / papers / 2021 / Quranic Verses Semantic Relatedness Using AraBERT

Quranic Verses Semantic Relatedness Using AraBERT

Paper · 2021 · Abdullah Alsaleh, Eric Atwell and Abdulrahman Altahhan

Summary

The paper asks whether a pre-trained Arabic transformer can decide if two Quranic verses are semantically related. The authors use QurSim, a set of 7679 verse pairs derived from Ibn Kathir's Tafsir and graded as strongly related ('2'), topically related ('1'), or non-obviously related ('0'). They convert verse references to text, delete symmetric duplicates and pairs with conflicting labels, and drop the middle grade '1'. Because QurSim contains no true negatives, they sample random verse pairs absent from the dataset and label them '-1'. They then build three binary datasets: '2' vs '0', '2' vs '-1', and '2' vs a merged '0'+'-1' class. They fine-tune the BASE versions of AraBERTv0.2 and AraBERTv2 on each one. AraBERTv0.2 performs best throughout, peaking at about 92% accuracy on '2' vs '-1'. The error analysis is the most instructive part. Failures come from thoughts split across several short verses, synonyms with little shared vocabulary, and religious idioms the model reads literally. The work is a first BERT baseline for relatedness in Classical Arabic scripture.

Key points

  • QurSim label distribution: '2' = 40.09%, '1' = 48.41%, '0' = 11.49%. Verse text was attached using Aloufi's (2019) Quran dataset.
  • Of 764 duplicate records, 592 were reversed pairs with identical labels (redundant), and 172 had contradictory labels. All were removed.
  • After filtering, 2548 pairs labelled '2' and 857 labelled '0' remained. The authors added 2548 random '-1' pairs to balance the data.
  • Data splits:
    • '2' vs '0': 2052 train, 229 validation, 1124 test.
    • The two balanced datasets: 3072 train, 342 validation, 1682 test.
  • Setup: BASE models only (hardware limits), Google Colab, batch size 32, learning rate 2e-5, 8 epochs. AraBERTv2 relies on Farasa segmentation, while v0.2 does not. The best result was AraBERTv0.2 at ~92% on '2' vs '-1'.

Interesting ideas

  • I1. Because relatedness is symmetric, reversed duplicate pairs are merged, and any pair whose two directions carry different labels is discarded instead of guessing a winner.
  • I2. Random negatives are drawn from outside the annotated set, and the authors state explicitly that "unrelated" then means unrelated only according to the source annotator's judgments.
  • I3. One experiment merges weak, non-obvious relations with random pairs into a single class, testing whether the model treats near-misses like noise. Accuracy for v0.2 dropped, which shows the classes differ.
  • I4. Errors arose when one idea was spread over several short verses, so a single fragment lacked the content that made it match its partner.
  • I5. The model missed that "stand the night" conventionally means praying at night, showing that literal reading fails on domain idioms.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): When building silver data, canonicalise symmetric relations (e.g., coreference, paraphrase) and drop items where witness parsers or teachers contradict themselves under reordering.
  • I2 (learn): Negative controls should record their provenance and their implied assumption, so evaluation reports what a negative actually certifies.
  • I3 (learn): Keep hard negatives (plausible but wrong frames or answers) separate from easy random ones, and report scores on each group to avoid inflated metrics.
  • I4 (world): Compare meaning at the level of a discourse span that completes a thought, not a single sentence, by merging consecutive states and events before relating texts.
  • I5 (frames): Add an explainable idiom-to-frame mapping layer, so conventionalised phrases trigger their intended frame (e.g., worship or activity) instead of a literal posture frame.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Abdullah Alsaleh, Eric Atwell and Abdulrahman Altahhan. 2021. Quranic Verses Semantic Relatedness Using AraBERT. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alsaleh2021quranic,
  title = {{Quranic Verses Semantic Relatedness Using AraBERT}},
  author = {Abdullah Alsaleh, Eric Atwell and Abdulrahman Altahhan},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.19/}
}

References

19 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. ## Conclusion The paper presented experiments on the QurSim dataset using fine-tuned AraBERT model to classify pairs of verses either to be semantically related Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9.
  2. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. ↗
  4. I. A. El-Khair. 2016. 1.5 billion words arabic corpus. ArXiv, abs/1611.04033. ↗
  5. Nizar Habash, Owen Rambow, and Ryan Roth. 2009. Mada+tokan: A toolkit for arabic tokenization, diacritization, morphological disambiguation, pos tagging, stemming and lemmatization. In Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt. The MEDAR Consortium. find
  6. Chadi Helwe, Ghassan Dib, Mohsen Shamas, and Shady Elbassuoni. 2020. A semi-supervised BERT approach for Arabic named entity recognition. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 49–57, Barcelona, Spain (Online). Association for Computational Linguistics.
  7. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. ↗
  8. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013b. Distributed representations of words and phrases and their compositionality. ↗
  9. Reham Mohamed, Maha Ragab, Heba Abdelnasser, Nagwa M. El-Makky, and Marwan Torki. 2015. Al-bayan: A knowledge-based system for Arabic answer selection. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 226–230, Denver, Colorado. Association for Computational Linguistics. ↗
  10. Matteo Pagliardini, Prakhar Gupta, and Martin Jaggi. 2018. Unsupervised learning of sentence embeddings using compositional n-gram features. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).
  11. Nicole Peinelt, Dong Nguyen, and Maria Liakata. 2020. tBERT: Topic models and BERT joining forces for semantic similarity detection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7047–7055, Online. Association for Computational Linguistics. find
  12. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations.
  13. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks.
  14. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in bertology: What we know about how bert works. ↗
  15. Haitham Seelawi, Ahmad Mustafa, Hesham Al-Bataineh, Wael Farhan, and Hussein T. Al-Natsheh. 2019a. Nsurl-2019 shared task 8: Semantic question similarity in arabic. ↗
  16. Haitham Seelawi, Ahmad Mustafa, Hesham Al-Bataineh, Wael Farhan, and Hussein T. Al-Natsheh. 2019b. NSURL-2019 task 8: Semantic question similarity in Arabic. In Proceedings of The First International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2019) co-located with ICNLSP 2019 - Short Papers, pages 1–8, Trento, Italy. Association for Computational Linguistics.
  17. Abdul-Baquee Sharaf and Eric Atwell. 2012. Qursim: A corpus for evaluation of relatedness in short texts. ↗
  18. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.
  19. I. Zeroual, Dirk Goldhahn, T. Eckart, and A. Lakhouaja. 2019. Osian: Open source international arabic news corpus - preparation and integration into the clarin-infrastructure. In WANLP@ACL 2019. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alsaleh-2021-quranic-verses-semantic-relatedness-arabert/