Elamur Research⌕
☰
home / papers / 2025 / Arabic Mental Health Question Answering: A Multi-Task Approach with Advanced Retrieval-Augmented Generation

Arabic Mental Health Question Answering: A Multi-Task Approach with Advanced Retrieval-Augmented Generation

Paper · 2025 · AbdelAziz Amr, Mamdouh Koritam, Mohamed Youssef et al.

Summary

This shared-task paper (AraHealthQA 2025, Track 1: MentalQA) tackles Arabic mental health QA, where dialect variety, scarce annotated data and cultural or religious norms make generic systems weak. The authors cover three subtasks. They classify question categories and answer strategies with fine-tuned AraBERTv2, MARBERTv2 and an English RoBERTa baseline. They also generate answers through a RAG pipeline.

The knowledge base contains 1,000 QA pairs. Half came from the organizers, and half were LLM-generated and then checked by human reviewers. Retrieval mixes dense vectors from Arabic-SBERT-100K with sparse rank_bm25 scores. It adds query normalization, expansion of dialect words to formal ones, a weighted reranker and a filter that demotes culturally unacceptable advice. Answers come from Saka-14B tuned with QLoRA.

The headline figures are weighted F1 of 0.742 and 0.718 for the two classification tasks and a BERTScore F1 of 0.821 for generation, about 15% above retrieval-only baselines. The result tables, however, report higher classification numbers (roughly 0.83). The work shows how culture-aware constraints can be built into a domain QA system for a low-resource language.

Key points

  • The data has 1,000 Arabic QA pairs: 500 from the organizers and 500 synthetic. Three native speakers with mental-health vocabulary reviewed every synthetic item.
  • Question categories are Treatment, Diagnosis, Healthy Lifestyle, Epidemiology and Other. Answer strategies are Information, Direct Guidance and Emotional Support. Emotional Support is rare (7.4%) and was the hardest class to detect.
  • The tables report weighted F1 of 0.832 (MARBERTv2 ensemble) for questions and 0.8289 with Jaccard 0.7667 (AraBERTv2) for answer strategies. RoBERTa-base lagged at 0.771. These figures conflict with the 0.742/0.718 given in the abstract.
  • The reranker weights are 0.4 semantic similarity, 0.2 BM25, 0.2 text length and 0.2 question similarity, tuned on validation BERTScore.
  • Saka-14B was trained in 4-bit NF4 with LoRA (r=8, alpha 16, dropout 0.05) and sees the top-5 contexts. It reached BERTScore F1 0.821, and a paired bootstrap test with 1,000 samples found it significantly better than a Qwen14 baseline.

Interesting ideas

  • I1. The predicted answer-strategy label is saved as query metadata, and retrieval then favors passages written in that strategy, so classification steers generation instead of running separately.
  • I2. Context ranking uses a transparent linear mix of a few named signals, with weights tuned on validation data, rather than an opaque reranker.
  • I3. Before retrieval, colloquial or dialect wording is mapped to formal equivalents and spelling variants are standardized, so the same meaning lands on the same key.
  • I4. An English-only RoBERTa is kept as a baseline specifically to measure how much language-specific pretraining adds.
  • I5. The authors observe that retrieving more contexts made answers more detailed but less focused, so they capped retrieval at five.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): A reader could first label a question's type (count, cause, who-did-what) and then pick which world facts and which answer template to use, with the label logged as part of the explanation.
  • I2 (world): Candidate entity or event matches could be scored the same way from inspectable features (frame overlap, role agreement, recency), so every choice can be traced back to its weights.
  • I3 (tokens): A normalization layer mapping informal spellings and colloquial verbs to canonical lemmas would help English inputs such as "gonna" or "kinda" reach the right UD lemmas and later the right VerbNet classes.
  • I4 (learn): Evaluations could routinely include a deliberately mismatched model or resource as a negative control, showing that a gain truly comes from the component being tested.
  • I5 (answer): When readers pull supporting facts from the world of the text, a small tunable cap on evidence could keep answers on point, with the cap reported in the explanation trace.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Nile UniversityGiza · Egypt14 works · 158 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 worksPsychology · 1,552 worksSocial Psychology · 306 works

🧭 Topics 3Research subjects and languages the work is about.Semitic languages · 2,908 worksQuestion answering · 441 worksInformation retrieval & search · 473 works

Cite

Cite as AbdelAziz Amr, Mamdouh Koritam, Mohamed Youssef, Marwa Aldeeb and Ensaf H. Mohamed. 2025. Arabic Mental Health Question Answering: A Multi-Task Approach with Advanced Retrieval-Augmented Generation. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.27

@inproceedings{amr2025arabic,
  title = {{Arabic Mental Health Question Answering: A Multi-Task Approach with Advanced Retrieval-Augmented Generation}},
  author = {AbdelAziz Amr, Mamdouh Koritam, Mohamed Youssef, Marwa Aldeeb and Ensaf H. Mohamed},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.27},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.27}
}

References

7 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yu Jin, Jiayi Liu, Pan Li, Baosen Wang, Yangxinyu Yan, Huilin Zhang, Chenhao Ni, Jing Wang, Yi Li, 14B provided verbose but less targeted responses. Yajun Bu, and Yuanyuan Wang. (2025, May). The Trade off Observation: Verbosity in Saka-14B Applications of Large Language Models in Mental outputs was partially controlled by the prompt de-Health: Scoping Review. Journal of Medical Internet Research, vol. 27, p. e69284. JMIR Publications. sign and top 5 retrieval limit; increasing retrieval scope tended to increase detail but sometimes re-[2] Zhijun Guo, Alvina Lai, Johan Hilmar Thygesen, duced focus. Joseph Farrington, Thomas Keen, and Kezhi Li. (2024, Oct.). Large Language Models for Mental Health Applications: Systematic Review. JMIR Mental Health, vol. 11, p. e57400. JMIR Publications. Conclusion and Future work This study presented a multi-task Arabic mental health question answering framework that integrates question categorization, answer strategy classification, and culturally sensitive Retrieval Augmented Generation. The system addresses the linguistic complexity and cultural considerations of Arabic mental health discourse by combining semantic chunking, query enhancement, hybrid retrieval, and a finetuned Saka-14B model aligned with cultural norms. Experimental evaluation demonstrated that the framework achieved weighted F1-scores of 0.742 for question categorization and 0.718 for answer strategy classification, alongside a BERTScore F1 of 0.821 for generative answering—up to 15% higher than retrieval only baselines. Expert evaluations confirmed that the integration of cultural filtering improved trustworthiness and contextual relevance. Despite these promising results, limitations remain. The dataset size is small relative to comparable English language resources, which restricts the model’s generalizability. Dialectal imbalance, find
  2. He Hu, Yucheng Zhou, Juzheng Si, Qianning Wang, Hengheng Zhang, Fuji Ren, Fei Ma, and Laizhong Cui. (2025, May). Beyond Empathy: Integrating Diagnostic and Therapeutic Reasoning with Large Language Models for Mental Health Counseling. arXiv preprint arXiv:2505.15715. ↗
  3. Lei Yin, Yuxuan Zhang, Xiaoyu Wang, and Jing Chen. (2025). Evaluating LLMs and RAG Pipelines for Mental Health App Recommendations. AIMS Applied Computing and Informatics, vol. 2025, no. 1, p. 10. AIMS Press. find
  4. Hassan Alhuzali, Ashwag Alasmari, and Hamad Al-saleh. (2024). MentalQA: An Annotated Arabic Corpus for Questions and Answers of Mental Healthcare. IEEE Access, vol. 12, pp. 101155–101165. doi: 10.1109/ACCESS.2024.3430068. ↗
  5. Noureldin Zahran, Aya Elsayed Fouda, Radwa Jamal Hanafy, and Mohammed Elsayed Fouda. (2025, Jan.). A Comprehensive Evaluation of Large Language Models on Mental Illnesses in Arabic Context. arXiv preprint arXiv:2501.06859. ↗
  6. Hassan Alhuzali, Farah Shamout, Muhammad Abdul-Mageed, Chaymae Abouzahir, Mohammad Abu-Daoud, Abdulrahman Alasmari, Waseem Al-Eisawi, Rawan Al-Monef, Ali Alqahtani, Lina Ayash, Nizar Habash, and Lina Kharouf. (2025). AraHealthQA 2025: The First Shared Task on Arabic Health Question Answering. arXiv preprint arXiv:2508.20047, v2. ↗
  7. Hassan Alhuzali and Ashwag Alasmari. (2025, Apr.). Pre-Trained Language Models for Mental Health: An Empirical Study on Arabic Q&A Classification. Healthcare, vol. 13, no. 9, p. 985. MDPI. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/amr-2025-arabic-mental-health-question-answering/