Elamur Research⌕
☰
home / papers / 2025 / !MSA at AraHealthQA 2025 Shared Task: Enhancing LLM Performance for Arabic Clinical Question Answering through Prompt Engineering and Ensemble Learning

!MSA at AraHealthQA 2025 Shared Task: Enhancing LLM Performance for Arabic Clinical Question Answering through Prompt Engineering and Ensemble Learning

Paper · 2025 · Mohamed Younes, Seif Ahmed and Mohamed Basem

Summary

The authors tackle Arabic medical question answering in Track 2 (MedArabiQ) of the AraHealthQA-2025 shared task. The task covers 12 clinical domains and has two subtasks. Sub-Task 1 is multiple-choice selection over standard, bias-injected and fill-in-the-blank items. Sub-Task 2 is free-text generation for blanks without options, patient–doctor consultations from AraMed, GEC-corrected questions and LLM-paraphrased questions. No model is fine-tuned. The team relies entirely on Gemini 2.5 Flash and careful prompting.

For classification, three differently framed prompts each answer the question:

  • a native Arabic few-shot prompt;
  • a prompt that first translates the question into English;
  • a prompt that first rewrites the question so each option carries a short explanation.

A majority vote with a fixed tie-break order combines their answers. For generation, a single Arabic prompt casts the model as a medical expert, gives format-specific rules and examples, and is followed by output cleanup.

The system placed 2nd in both subtasks, with 76% accuracy and a BERTScore of 86.953%. The result shows that prompt design and light ensembling can be competitive in a low-resource clinical language setting.

Key points

  • Data: ST1 has 300 dev and 100 test items, and ST2 has 400 dev and 100 test items. Dev sets were used only for prompt design and choosing examples. Test scoring went through the organizers' API.
  • Individual ST1 systems scored 74% (RFA, refinement + answer), 71% (AFS, Arabic few-shot) and 69% (ETA, English translation + answer). The vote over all three reached 76%, and ties were resolved in the order RFA > AFS > ETA. A zero-shot Arabic prompt (AZS) served only as a baseline.
  • Leave-one-out ablation of the ensemble lowered accuracy by 3 points without RFA, 2 without AFS and 1 without ETA.
  • In ST2, Arabic normalization plus stripping markdown from outputs raised BERTScore by roughly 2–3%.
  • Planned future work includes retrieval from vetted Arabic medical sources, more diverse models and human-in-the-loop checking.

Interesting ideas

  • I1. Several "views" of the same input (original, translated, clarified) vote on the answer, with a fixed priority order for ties based on which view is strongest alone; diversity of framing reduces the bias of any single prompt.
  • I2. Before answering, the question is rewritten so that every option gets a short explanation, which turned out to be the strongest single strategy.
  • I3. Each ensemble member is removed in turn to measure its marginal contribution, confirming that the members complement each other rather than duplicate each other.
  • I4. Formatting artifacts such as markdown and inconsistent punctuation were shown to shift a similarity metric by a few points, so outputs are normalized before scoring.
  • I5. One unified instruction carries explicit per-format output rules (only the missing words, comma-separated blanks, brief advice with referral) instead of separate prompts per task.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver-data agreement among witness parsers could use the same scheme, resolving deadlocks by a fixed ranking of witness reliability instead of discarding the sentence.
  • I2 (answer): School-test readers could expand each multiple-choice option into a small explicit proposition before matching it against the world of the text.
  • I3 (learn): Evaluation could include leave-one-witness-out runs to prove each parser or teacher tier actually adds signal, alongside negative controls.
  • I4 (back): The text → graph → text compressor should normalize surface formatting before computing similarity scores, so that measured differences reflect meaning rather than layout noise.
  • I5 (answer): Readers could share one answer-rendering layer with declarative per-question-type output contracts, which keeps answers consistent and easy to check.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓October University of Modern Sciences and ArtsCairo · Egypt10 works · 27 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksQuestion answering · 441 works

Cite

Cite as Mohamed Younes, Seif Ahmed and Mohamed Basem. 2025. !MSA at AraHealthQA 2025 Shared Task: Enhancing LLM Performance for Arabic Clinical Question Answering through Prompt Engineering and Ensemble Learning. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.25

@inproceedings{younes2025msa,
  title = {{!MSA at AraHealthQA 2025 Shared Task: Enhancing LLM Performance for Arabic Clinical Question Answering through Prompt Engineering and Ensemble Learning}},
  author = {Mohamed Younes, Seif Ahmed and Mohamed Basem},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.25},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.25}
}

References

11 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. *Table 1: Sub-Task 1 (Classification) official results on test set. All experiments used Gemini 2.5 Flash.* Approach Description English Translation (ETA) Translate to English then answer Arabic Few-Shot (AFS) Arabic instruction + 6 medical examples Refinement + Answer (RFA) Question clarification + option explanation Ensemble (Final) 3-way majority vote (RFA + AFS + ETA) Note: Individual systems not submitted separately; ensemble represents official submission. Accuracy (%) 69.0 71.0 74.0 76.0 Ranking – – – 2 nd *Table 2: Sub-Task 2 (Generation) official results on test set using unified prompting approach.* Approach Unified Arabic Prompting Description Single prompt with Arabic medical expert role-playing, few-shot examples, handles all question formats Final Ranking Official AraHealthQA-2025 shared task BERTScore combines BLEU, ROUGE, and semantic similarity metrics. 2021. Arbert & marbert: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4510–4521.
  2. Mouath Abu Daoud, Chaimae Abouzahir, Leen Kharouf, Walid Al-Eisawi, Nizar Habash, and Farah E Shamout. 2025. Medarabiq: Benchmarking large language models on Arabic medical tasks. arXiv e-prints, pages arXiv–2505.
  3. Youssef Al Hariri and Ibrahim Abu Farha. 2024. SMASH at StanceEval 2024: Prompt engineering LLMs for Arabic stance detection. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 800–806, Bangkok, Thailand. Association for Computational Linguistics.
  4. Ashwag Alasmari, Sarah Alhumoud, and Waad Alshammari. 2024. Aramed: Arabic medical question answering using pretrained transformer language models. In Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT), pages 50–56.
  5. Hassan Alhuzali, Farah Shamout, Muhammad Abdul-Mageed, Chaimae Abouzahir, Mouath Abu-Daoud, Ashwag Alasmari, Walid Al-Eisawi, Renad Al-Monef, Ali Alqahtani, Lama Ayash, and 1 others. 2025. Arahealthqa 2025 shared task description paper. arXiv preprint arXiv:2508.20047. ↗
  6. Jennifer Chu-Carroll, Krzysztof Czuba, John Prager, and Abraham Ittycheriah. 2003. In question answering, two heads are better than one. find
  7. BERTScore (%) 86.953 2 nd place In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 24–31. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186. ↗
  9. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 9459–9474.
  10. Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, and Mark Gerstein. 2024. Struc-Bench: Are large language models good at generating complex structured tabular data? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 12–34, Mexico City, Mexico. Association for Computational Linguistics. find
  11. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824– 24837.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/younes-2025-msa-arahealthqa-2025-shared-task/