Elamur Research⌕
☰
home / papers / 2024 / Question-Answering (QA) Model for a Personalized Learning Assistant for Arabic Language

Question-Answering (QA) Model for a Personalized Learning Assistant for Arabic Language

Summary

War, strikes and pandemics can cut school students off from their teachers, leaving them to study alone from textbooks with no one to ask when questions arise. The authors build a question-answering component for an Arabic personalized learning assistant. They fine-tune a pre-trained BERT model on material from the Palestinian 11th- and 12th-grade biology books. A course team collected the textbook data by hand and organized it by unit, lesson and section, with each section's content, its questions and a summary. A second, public dataset of student records from the Kalboard 360 LMS supplies learner profiles, so that responses can depend on a student's performance band. On 693 question–answer pairs from Arabic SQuAD, the model reaches an Exact Match of 20% and a word-overlap F1 of 51%. The authors read the gap between these scores as a sign that many answers are partly right but worded differently, and they link this to Arabic's rich morphology. The work is an early, curriculum-specific step toward Arabic tutoring tools, though the scores leave substantial room for improvement.

Key points

  • The base model is BERT, fine-tuned for extractive QA on Arabic biology textbook content from the Palestinian secondary curriculum (grades 11–12).
  • The textbook dataset has 111 observations and 7 attributes: id, unit_title, lesson_title, section_title, section_content, questions and a summary field.
  • The student-profile data is the Kaggle xAPI-Edu-Data set from Kalboard 360 (University of Jordan), with 480 records and 16 features. Learners are binned into Low (0–69), Middle (70–89) and High (90–100) levels.
  • Evaluation used 693 Arabic SQuAD pairs, with only the questions fed to the model. Results averaged EM 20% and F1 51%.
  • F1 is computed from shared words: precision is shared words over predicted words, and recall is shared words over gold-answer words.

Interesting ideas

  • I1. Reporting strict exact match together with a word-overlap F1 separates fully wrong answers from answers that are right but worded differently, which matters in morphologically rich languages. — For Mova (answer): Fable and school-test readers could report both strict and overlap scores, ideally computed over lemmas, so that inflection or word-order variation is not counted as a reasoning failure.
  • I2. The source corpus keeps the textbook's own hierarchy (unit → lesson → section) and stores each section's questions and summary alongside it, so the context for a question can be scoped by structure. — For Mova (world): The world of a text could record which section or episode each entity and event came from, letting answers point to an explainable, structurally local source.
  • I3. Answers are meant to be conditioned on a learner profile (performance band) as well as on the question and context, so the same question can get different explanations for different students. — For Mova (back): Generation from the meaning graph could accept a reader-level parameter that controls how much detail or how many intermediate steps to verbalize.
  • I4. The authors explicitly lower their expectation for exact match because of Arabic word variety, which is a reminder to calibrate metrics to the language's morphology before judging a system. — For Mova (learn): Evaluation could include a control that measures how much score is lost purely to surface-form variation, by comparing exact-string and lemma-level scoring on the same outputs.
  • I5. The model is trained on curriculum text but tested on a different public QA set (Arabic SQuAD), which probes transfer beyond the narrow training domain. — For Mova (learn): Keeping a held-out set from another genre alongside in-domain tests would show whether gains on school texts carry over to fables and general prose.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Question answering · Semitic languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2406.08519 · arXiv 2406.08519

@misc{ashqar2024question,
  title = {{Question-Answering (QA) Model for a Personalized Learning Assistant for Arabic Language}},
  author = {Ashqar;, Mohammad Sammoudi; Ahmad Habaybeh; Huthaifa I. and Elhenawy, Mohammed},
  year = {2024},
  doi = {10.48550/arxiv.2406.08519},
  eprint = {2406.08519},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2406.08519}
}

References

18 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Farea, Amer and Yang, Zhen and Duong, Kien and Perera, Nadeesha and Emmert-Streib, Frank, "Evaluation of Question Answering Systems: Complexity of judging a natural language," arXiv preprint arXiv:2209.12617, 2022. ↗
  2. "What is Question Answering?," Face Hugging, [Online]. Available: https://huggingface.co/tasks/question-answering. [Accessed 5 February 2024]. ↗
  3. Sharma, Yashvardhan and Gupta, Sahil, "Deep learning approaches for question answering system," Procedia computer science, vol. 132, pp. 785-794, 2018. find
  4. Tang, Ying and Liang, Joleen and Hare, Ryan and Wang, Fei-Yue, "A personalized learning system for parallel intelligent education," IEEE Transactions on Computational Social Systems, vol. 7, pp. 352-361, 2020. find
  5. McHugh, David and Shaw, Sarah and Moore, Travis R and Ye, Leafia Zi and Romero-Masters, Philip and Halverson, Richard, "ncovering themes in personalized learning: Using natural language processing to analyze school interviews," Journal of Research on Technology in Education, vol. 52, pp. 391-402, 2020. find
  6. Qaiser, Shahzad and Ali, Ramsha, "Text mining: use of TF-IDF to examine the relevance of words to documents," International Journal of Computer Applications, vol. 181, pp. 25-29, 2018. find
  7. Yilmaz, Seyhmus and Toklu, Sinan, "A deep learning analysis on question classification task using Word2vec representations," Neural Computing and Applications, vol. 32, pp. 2909-2928, 2020. find
  8. Ashok, M and Ramasamy, Kumar and Snehitha, G and Keerthi, SR, "A systematic survey of cognitive chatbots in personalized learning framework," in 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), IEEE, 2021, pp. 241-245. find
  9. Wellnhammer, Natalie and Dolata, Mateusz and Steigler, Susanne and Schwabe, Gerhard, "Studying with the help of digital tutors: Design aspects of conversational agents that influence the learning process," 2020. find
  10. Kochmar, Ekaterina and Vu, Dung Do and Belfer, Robert and Gupta, Varun and Serban, Iulian Vlad and Pineau, Joelle, "Automated data-driven generation of personalized pedagogical interventions in intelligent tutoring systems," International Journal of Artificial Intelligence in Education, vol. 32, pp. 323-349, 2022. find
  11. Mathew, Ann Neethu and Rohini, V and Paulose, Joy, "NLP-based personal learning assistant for school education," Int. J. Electr. Comput. Eng, vol. 11, pp. 4522-4530, 2021. find
  12. Aljameel, Sumayh S and O'Shea, James D and Crockett, Keeley A and Latham, Annabel and Kaleem, Mohammad, "Development of an Arabic conversational intelligent tutoring system for education of children with ASD," in 2017 IEEE international conference on computational intelligence and virtual environments for measurement systems and applications (CIVEMSA), IEEE, 2017, pp. 24-29. find
  13. Schmohl, Tobias and Schelling, Kathrin and Go, Stefanie and Freier, Carolin and Hunger, Marianne and Hoffmann, Franziska and Helten, Anne-Kathrin and Richter, Florian and others, "Combining NLP, speech recognition, and indexing: an AI-based learning assistant for higher education," in Conference Proceedings. The Future of Education 2023, 2023. find
  14. Gillioz, Anthony and Casas, Jacky and Mugellini, Elena and Abou Khaled, Omar, "Overview of the Transformer-based Models for NLP Tasks," in 2020 15th Conference on Computer Science and Information Systems (FedCSIS), IEEE, 2020, pp. 179-183. find
  15. Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, Łukasz and Polosukhin, Illia, "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017. find
  16. Radford, Alec and Narasimhan, Karthik and Salimans, Tim and Sutskever, Ilya and others, "Improving language understanding by generative pre-training," 2018. find
  17. J. a. C. M.-W. a. L. K. a. T. K. Devlin, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018. ↗
  18. Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sammoudi-2024-question-answering-qa-model-personalized/