Elamur Research⌕
☰
home / papers / 2023 / HAQA and QUQA: Constructing Two Arabic Question-Answering Corpora for the Quran and Hadith

HAQA and QUQA: Constructing Two Arabic Question-Answering Corpora for the Quran and Hadith

Paper · 2023 · Sarah Alnefaie, Eric Atwell and Mohammad Ammar Alsalka

Summary

Arabic question answering over Islamic texts has lacked shared, demanding benchmarks. The only public Quran QA sets are small, cover about a fifth of the Quran, and mostly contain questions that reuse the answer's wording. No public Hadith QA set exists, so prior Hadith systems were each tested on private data and cannot be compared. The authors build two corpora.

  • QUQA merges cleaned versions of AQQAC and AyaTEC/QRCD with new material taken from expert question-and-answer books. Questions are in MSA and answers are Quranic verses in CA.
  • HAQA is assembled from expert books whose answers cite hadiths.

The construction pipeline had several steps:

  • design a shared metadata schema, stored as UTF-8 CSV;
  • choose sources that give scriptural evidence, not only scholarly opinion;
  • run OCR, then review the output by hand;
  • extract the question-answer pairs with a script;
  • fill in metadata with Python or manually.

QUACA is not a typo to fix here: QUQA reaches roughly 47% verse coverage with harder, less lexically overlapping questions. Both corpora are released to serve as gold test sets and as training data for QA models.

Key points

  • QUQA has 3382 records, 2189 distinct questions and over 301,000 tokens. Its answers draw on 2930 verses, about 47% of the Quran. 199 questions are single-answer and 1990 are multiple-answer.
  • HAQA has 1598 records and 1359 questions. Its hadiths come from collections such as Al-Bukhari, Muslim, Al-Tirmidhi, Al-Nasai, Ibn Majah and Imam Ahmad.
  • For comparison, the cleaned AQQAC has 616 records, 611 questions and 1232 verses (19%). AyaTEC/QRCD has 1166 records, 169 questions and 1247 verses (20%).
  • Each record stores several fields:
    • question type and the question's opening word;
    • the exact answer span(s), called "Answer Instances", and the full verse or hadith;
    • optional expert commentary;
    • chapter and verse range;
    • source name and link;
    • a credibility flag and the ID from the original dataset.
  • Both datasets are published on GitHub (scsaln/HAQA-and-QUQA).

Interesting ideas

  • I1. Answer IDs separate two cases: one meaning spread over several passages with different wording (numbered 1.1, 1.2…) versus genuinely different answers (1, 2, 3…). This lets scoring treat paraphrased evidence differently from distinct answers.
  • I2. Each record carries a yes/no flag saying whether an expert checked it, so reviewed and unreviewed data can live in one file without being confused.
  • I3. The authors argue that older benchmarks were too easy because questions reused the answer's own words. They therefore favored questions whose phrasing differs from the evidence.
  • I4. Three layers are kept apart: the minimal answer span, the full containing unit (verse or hadith), and free expert commentary. This separates the "what" from the "where" and the "why".
  • I5. Each benchmark is judged by the share of the underlying text that its answers actually touch, not only by question count.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): Fable and school-test QA keys could group equivalent evidence sentences under one answer ID. A reader that cites any of them is then credited once, rather than penalized or double-counted.
  • I2 (learn): Silver items agreed by witness parsers or LLM teachers could carry a similar verified bit. Training can then weight or filter by it, and evaluation can run only on verified rows.
  • I3 (learn): Build a negative-control split where question and answer share few content words. If scores drop sharply there, the reader is leaning on surface matching instead of the frame/world path.
  • I4 (answer): Answer records could store the exact entity or value, the supporting sentence, and an explanation trace as separate fields. Each part of an explainable answer can then be checked on its own.
  • I5 (learn): Measure what fraction of sentences, entities or event links in the fable/tale corpus are exercised by QA items. This shows which parts of the world-building pipeline are never tested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of LeedsLeeds · United Kingdom40 works · 1,822 citations🎓King Abdulaziz UniversityJeddah · Saudi Arabia31 works · 508 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksQuestion answering · 441 works

Cite

Cite as Sarah Alnefaie, Eric Atwell and Mohammad Ammar Alsalka. 2023. HAQA and QUQA: Constructing Two Arabic Question-Answering Corpora for the Quran and Hadith. Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alnefaie2023haqa,
  title = {{HAQA and QUQA: Constructing Two Arabic Question-Answering Corpora for the Quran and Hadith}},
  author = {Sarah Alnefaie, Eric Atwell and Mohammad Ammar Alsalka},
  booktitle = {Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing},
  year = {2023},
  doi = {10.26615/978-954-452-092-2\_010},
  url = {https://doi.org/10.26615/978-954-452-092-2_010}
}

References

19 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asad Abdi, Shafaatunnur Hasan, Mohammad Arshi, Siti Mariyam Shamsuddin, and Norisma Idris. 2020. A question answering system in hadith using linguistic knowledge. Computer Speech & Language, 60:101023. ↗
  2. Mohamed Adany Hamdelsayed Adany et al. 2017. An automatic question answering system for the Arabic Quran. Ph.D. thesis, Sudan University of Science and Technology. find
  3. Faisal bin Misfer bin Moawad Al Alami. 2022. Prayer (1770) Question and Answer. find
  4. Faisal bin Misfer bin Moawad Al-Wadi. 2016. Inference on children’s treasure. Dar Knoz Al-Islam. find
  5. Conclusion and Future Work Fouzia Alakeel. 2018. Quranic questions and answer. find
  6. Duraid ALmuselli. 2020. 900 questions and answers in managing the verses of the book. Altafseer, Erbil. This paper presents the process of building two Islamic religious corpora in Arabic. QUQA and HAQA are two datasets that contain questions and answers about the Holy Quran and the Hadith Sharif, respectively. Since these corpora include more than 4900 records, they are considered to Sarah Alnefaie, Eric Atwell, and Mohammad Ammar Alsalka. 2022. Challenges in the islamic question 3 http://https://github.com/scsaln/ HAQA-and-QUQA answering corpora. International Journal on Islamic Applications in Computer Science And Technology, 10(4):1–10. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 146–153. ↗
  7. Mohammad Mushabbab A Alqahtani. 2019. Quranic Arabic semantic search model based on ontology of concepts. Ph.D. thesis, University of Leeds. find
  8. Muhammad bin Jamil Zeno. 2004. The abbreviation of the Islamic belief from the Qur’an and Sunnah. find
  9. Shatha Altammami, Eric Atwell, and Ammar Alsalka. 2020. Constructing a bilingual hadith corpus using a segmentation tool. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 3390–3398. The European Language Resources Association (ELRA). find
  10. Muhammad bin Jamil Zeno. 2007. The doctrine of every Muslim in a question and answer. find
  11. Eric Atwell, Nizar Habash, Bill Louw, Bayan Abu Shawar, Tony McEnery, Wajdi Zaghouani, and Mahmoud El-Haj. 2010. Understanding the quran: A new grand challenge for computer science and artificial intelligence. ACM-BCS Visions of Computer Science 2010. find
  12. Mohamed Adany Hamdelsayed and Eric Atwell. 2016. Islamic applications of automatic question-answering. Journal of Engineering and Computer Science, 17(2):51–57. find
  13. Bothaina Hamoud and Eric Atwell. 2017. Evaluation corpus for restricted-domain question-answering systems for the holy quran. International Journal of Science and Research, 6(8):1133–1138. find
  14. Rana Malhas and Tamer Elsayed. 2020. Ayatec: building a reusable verse-based test collection for arabic question answering on the holy qur’an. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 19(6):1–21. find
  15. Rana Malhas and Tamer Elsayed. 2022. Arabic machine reading comprehension on the holy qur’an using cl-arabert. Information Processing & Management, 59(6):103068. ↗
  16. Rana Malhas, Watheq Mansour, and Tamer Elsayed. 2022. Qur’an qa 2022: Overview of the first shared task on question answering over the holy qur’an. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 79–87.
  17. Hajer Maraoui, Kais Haddar, and Laurent Romary. 2021. Arabic factoid question-answering system for islamic sciences using normalized corpora. Procedia Computer Science, 192:69–79. find
  18. NABEEL NEAMAH and SAIDAH SAAD. 2017. Question answering system supporting vector machine method for hadith domain. Journal of Theoretical & Applied Information Technology, 95(7). find
  19. Ahmed Sleem, Eman Mohammed lotfy Elrefai, Marwa Mohammed Matar, and Haq Nawaz. 2022. Stars at qur’an qa 2022: Building automatic extractive question answering systems for the holy qur’an with transformer models and releasing a new dataset.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alnefaie-2023-haqa-quqa-constructing-two-arabic/