Elamur Research⌕
☰
home / papers / 2025 / AraMinds at AraHealthQA 2025: A Retrieval-Augmented Generation System for Fine-Grained Classification and Answer Generation of Arabic Mental Health Q&A

AraMinds at AraHealthQA 2025: A Retrieval-Augmented Generation System for Fine-Grained Classification and Answer Generation of Arabic Mental Health Q&A

Paper · 2025 · Mohamed Zaytoon, Ahmed Mahmoud Salem, Ahmed Sakr et al.

Summary

The paper describes the AraMinds entry to the AraHealthQA 2025 shared task on Arabic mental health Q&A. The task had three parts: multi-label tagging of patient questions (seven intent classes), multi-label tagging of doctor answers (three style classes), and generating replies to new questions. The data is MentalQA, collected from the altibbi platform. The team followed two routes. In the first, they built an external knowledge base by having Gemini gather web articles for each training question and condense related ones into a single text. They indexed these with BAAI/bge-m3 in chroma-db and used them as retrieval context for google/medgemma-4b-it. In the second, they adapted MARBERTv2 with masked-LM training on Arabic psychiatric books and articles, then fine-tuned it for the two classification tracks. Which route worked best depended on the track. The encoder did better on question tagging (macro-F1 0.558). Retrieval prompting did better on answer tagging (0.760). Retrieval also lifted generation to a BERTScore of 0.663. The system placed 3rd in answer classification and 4th in generation, which shows that small, domain-tuned models can stay competitive in a low-resource, high-stakes setting.

Key points

  • MentalQA comes from altibbi and has 500 manually annotated samples. Questions carry 7 labels and answers carry 3, and both tracks are multi-label.
  • The continued-pretraining corpus has about 4.5M tokens: 72 Arabic or professionally translated books (including DSM-5) and 32 Royal College of Psychiatrists articles. MARBERTv2 was trained on it with AdamW, lr 2e-5, for 20 epochs on one RTX-3090.
  • Macro-F1 results by track:
    • Question classification: RAG scored 0.490, the encoder without pretraining 0.549, and the encoder with pretraining 0.558.
    • Answer classification: RAG scored 0.760, ahead of the encoder at 0.735 with pretraining and 0.733 without.
  • On validation, MedGemma3-4b (0.620) narrowly beat Gemma3-4b, Qwen2.5 7b, Phi-mini 4b and Llama3.2 3b. Adding RAG raised its test BERTScore from 0.632 to 0.663.
  • In the error analysis, a request for sleep medication was tagged as diagnosis or epidemiology because the model latched onto the word "insomnia". Generated answers also tended to be wordy and generic rather than concrete.

Interesting ideas

  • I1. Instead of indexing every fetched article, the team merged the articles relevant to one question into a single condensed document, which keeps the index small and retrieval cleaner.
  • I2. They ran the same tasks through two different routes, retrieval-prompted LLM and fine-tuned encoder, and found each route won on a different track, so the choice was made per subtask.
  • I3. Continued masked-LM training on a modest in-domain corpus (around 4.5M tokens) gave small but consistent gains before fine-tuning.
  • I4. Their failure analysis showed the classifier over-weighting a single salient symptom word instead of the user's actual request.
  • I5. The generator was picked by a cheap validation bake-off of several small open LLMs before RAG was built around it.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Compiled knowledge could store one consolidated entry per concept or verb class rather than many overlapping sources, which reduces conflicting facts when building a text's world.
  • I2 (answer): School-test and fable QA readers could be scored per question type against an alternative pipeline, keeping the better route per type instead of assuming one design fits all.
  • I3 (tree): Briefly adapting the gparse transformer encoder on raw domain text (e.g., school textbooks) before parser training is a cheap thing to try and measure.
  • I4 (frames): This argues for deriving intent from the speech act and the requested action in the frame (asking for a drug), not from salient nouns, and for adding such keyword traps to evaluation as negative controls.
  • I5 (learn): LLM teacher tiers could be chosen the same way, with a quick scored comparison on a held-out slice before committing to one teacher for silver data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Alexandria UniversityAlexandria · Egypt39 works · 287 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksInformation retrieval & search · 473 works

Cite

Cite as Mohamed Zaytoon, Ahmed Mahmoud Salem, Ahmed Sakr and Hossam Elkordi. 2025. AraMinds at AraHealthQA 2025: A Retrieval-Augmented Generation System for Fine-Grained Classification and Answer Generation of Arabic Mental Health Q&A. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.28

@inproceedings{zaytoon2025araminds,
  title = {{AraMinds at AraHealthQA 2025: A Retrieval-Augmented Generation System for Fine-Grained Classification and Answer Generation of Arabic Mental Health Q\&A}},
  author = {Mohamed Zaytoon, Ahmed Mahmoud Salem, Ahmed Sakr and Hossam Elkordi},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.28},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.28}
}

References

33 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Sina Elahimanesh, Shayan Salehi, Sara Zahedi Movahed, Lisa Alazraki, Ruoyu Hu, and Abbas Edalat. 2023. From words and exercises to wellness: Farsi chatbot for self-attachment technique. arXiv preprint arXiv:2310.09362. ↗
  3. Maryam Farzan, Hamid Ebrahimi, Maryam Pourali, and Fatemeh Sabeti. 2025. Artificial intelligence-powered cognitive behavioral therapy chatbots, a systematic review. Iranian journal of psychiatry, 20(1):102. ↗
  4. Ashwag Alasmari. 2025. A scoping review of arabic natural language processing for mental health. In Healthcare, volume 13-9, page 963. MDPI. ↗
  5. Gerald N Grob. 1998. A history of psychiatry: From the era of the asylum to the age of prozac. Bulletin of the History of Medicine, 72(1):153–155. find
  6. Hassan Alhuzali, Ashwag Alasmari, and Hamad Al-saleh. 2024. Mentalqa: An annotated arabic corpus for questions and answers of mental healthcare. IEEE Access, 12:101155–101165. ↗
  7. Zhijun Guo, Alvina Lai, Johan H Thygesen, Joseph Farrington, Thomas Keen, Kezhi Li, and 1 others. 2024. Large language models for mental health applications: systematic review. JMIR mental health, 11(1):e57400. find
  8. Hassan Alhuzali, Farah Shamout, Muhammad Abdul-Mageed, Chaimae Abouzahir, Mouath Abu-Daoud, Ashwag Alasmari, Walid Al-Eisawi, Renad Al-Monef, Ali Alqahtani, Lama Ayash, Nizar Habash, and Leen Kharouf. 2025. Arahealthqa 2025 shared task description paper. In Proceedings of Arabic-NLP 2025. find
  9. Obieda Altobaishat, Mohamed Abouzid, Deemah Omari, Walid Sange, Ahmad K Al-Zoubi, Abdallah Bani-Salameh, and Yazan A Al-Ajlouni. 2025. Examining the burden of mental disorders in jordan: an ecological study over three decades. BMC psychiatry, 25(1):218. ↗
  10. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Preprint, arXiv:2402.03216. ↗
  11. Arman Cohan, Bart Desmet, Andrew Yates, Luca Soldaini, Sean MacAvaney, and Nazli Goharian. 2018. Smhd: a large-scale resource for exploring online language usage for multiple mental health conditions. arXiv preprint arXiv:1806.05258. ↗
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171– 4186. ↗
  13. Nizar Y Habash. 2010. Introduction to Arabic natural language processing. Morgan & Claypool Publishers. ↗
  14. Song Han, Huizi Mao, and William J. Dally. 2015. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. arXiv: Computer Vision and Pattern Recognition. find
  15. Sooji Han, Rui Mao, and Erik Cambria. 2022. Hierarchical attention network for explainable depression detection on Twitter aided by metaphor concept mappings. In Proceedings of the 29th International Conference on Computational Linguistics, pages 94–104, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. ↗
  16. Mariam Hassib, Nancy Hossam, Jolie Sameh, and Marwan Torki. 2022. Aradepsu: Detecting depression and suicidal ideation in arabic tweets using transformers. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 302–311.
  17. Shaoxiong Ji, Xue Li, Zi Huang, and Erik Cambria. 2022. Suicidal ideation and mental disorder detection with attentive relation networks. Neural Comput. Appl., 34(13):10309–10319. ↗
  18. Ashwag Maghraby and Hosnia Ali. 2022. Modern standard arabic mood changing and depression dataset. Data in Brief, 41:107999. ↗
  19. Nina H Di Cara, Valerio Maggio, Oliver SP Davis, and Claire MA Haworth. 2023. Methodologies for monitoring mental health on twitter: systematic review. Journal of Medical Internet Research, 25:e42734. ↗
  20. Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large language models: A survey. arXiv preprint arXiv:2402.06196. ↗
  21. FIFTH EDITION. 1980. Diagnostic and statistical manual of mental disorders. American Psychiatric Association, Washington, DC, pages 205–224. find
  22. Zabir Al Nazi and Wei Peng. 2024. Large language models in healthcare and medical domain: A review. In Informatics, volume 11, page 57. MDPI. ↗
  23. Sarah Aly Nazmy. 2025. Cultural Factors and Mental Health Help-Seeking Behaviors Among Middle Eastern/North African Adults in the United States. Ph.D. thesis, Alliant International University. find
  24. Aske Plaat, Max van Duijn, Niki van Stein, Mike Preuss, Peter van der Putten, and Kees Joost Batenburg. 2025. Agentic large language models, a survey. arXiv preprint arXiv:2503.23037. ↗
  25. Suraj Racha, Prashant Joshi, Anshika Raman, Nikita Jangid, Mridul Sharma, Ganesh Ramakrishnan, and Nirmal Punjabi. 2025. Mhqa: A diverse, knowledge intensive mental health question answering challenge for language models. arXiv preprint arXiv:2502.15418. ↗
  26. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, and 1 others. 2025. Medgemma technical report. arXiv preprint arXiv:2507.05201. ↗
  27. Yong Shan, Jinchao Zhang, Zekang Li, Yang Feng, and Jie Zhou. 2022. Mental health assessment for the chatbots. arXiv preprint arXiv:2201.05382. ↗
  28. Eric J Topol. 2019. High-performance medicine: the convergence of human and artificial intelligence. Nature medicine, 25(1):44–56. ↗
  29. Juraj Vladika and Florian Matthes. 2024. Improving health question answering with reliable and time-aware evidence retrieval. arXiv preprint arXiv:2404.08359. ↗
  30. Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, and Sophia Ananiadou. 2023. Mentalllama: Interpretable mental health analysis on social media with large language models. arXiv preprint arXiv:2309.13567. ↗
  31. Mohamed Zahran and 1 others. 2025. A comprehensive evaluation of large language models on mental illnesses in arabic context. arXiv preprint arXiv:2501.06859. ↗
  32. A Zaki and R Hassan. 2023. Optimizing large language models for arabic healthcare communication: A focus on patient-centered nlp applications. Multimodal Technologies and Interaction, 8(11):157. find
  33. Tianyi Zhang, Kishore, Wu, Q. Weinberger, and Artzi. 2020. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaytoon-2025-araminds-arahealthqa-2025-retrieval-augmented/