Elamur Research⌕
☰
home / papers / 2025 / Binary_Bunch at AraHealthQA Track 1: Arabic Mental Health Q&A Classification Using Data Augmentation and Transformer Models

Binary_Bunch at AraHealthQA Track 1: Arabic Mental Health Q&A Classification Using Data Augmentation and Transformer Models

Paper · 2025 · Sajib Bhattacharjee, Ratnajit Dhar, Kawsar Ahmed et al.

Summary

The paper addresses multi-label classification for Arabic mental-health QA, using the MentalQA data from AraHealthQA 2025 Track 1. Subtask 1 labels user questions with six categories, and Subtask 2 labels answers with three. Each subtask has only 350 training items, so the authors grew the training set in two steps. First, Grok-3 and GPT-4 produced one meaning-preserving paraphrase per item, and each paraphrase was checked against the original's labels. Second, they built synthetic items by joining two random originals and taking the union of their label sets.

They compared four encoders (XLMR-Arabic, AraBERT-Base, mBERT, XLMR-Base), each fully fine-tuned, against Phi-4 and Qwen-14B. The two LLMs were tested both with five-shot prompting and with LoRA tuning in UnSloth. The Arabic-adapted XLMR-Arabic won, with Jaccard scores of 53.00% and 77.44%, and the team placed second in both subtasks. The study suggests that, for small, sensitive, non-English classification tasks, a compact language-specialised encoder plus cheap synthetic data can outperform much larger general LLMs.

Key points

  • Data: 350 labelled training items and 150 test items per subtask. Augmentation raised training to 1,200 items: 700 after paraphrasing, plus 500 merged multi-label items.
  • Best system: XLMR-Arabic (Davlan/XLM-Roberta-base-finetuned-arabic) at learning rate 2×10⁻⁵, chosen from a sweep over 2×10⁻⁴, 2×10⁻⁵ and 2×10⁻⁶.
  • Effect of augmentation: XLMR-Arabic gained +7.00 Jaccard on Subtask 2. Adding label merging on top of paraphrasing gave it +3.22 and +8.44 Jaccard over paraphrasing alone, although weighted F1 sometimes dropped (for example −7.41 on Subtask 2).
  • Comparison with LLMs: Qwen-14B was trained through LoRA with 4-bit quantization, which left 34.9M trainable parameters. XLMR-Arabic updated all 278M of its parameters and beat the tuned LLMs on both subtasks.
  • Few-shot prompting: five-shot Phi-4 and Qwen3-14B stayed below the fine-tuned encoders.

Interesting ideas

  • I1. Joining two labelled examples and taking the union of their labels is a near-free way to create harder multi-label training items when data is scarce.
  • I2. LLM paraphrases were kept only after checking that the original labels still held, which guards against semantic drift in the synthetic data.
  • I3. The ablation stacks augmentation steps (paraphrase alone, then paraphrase plus merging) and reports two metrics that sometimes move in opposite directions.
  • I4. A small encoder specialised for one language, trained on all its parameters, beat much larger LLMs that were compressed with LoRA and 4-bit quantization.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): concatenating sentences that each carry one speech act would give training items with several acts, useful for a speech-act or implicature tagger that must handle mixed utterances.
  • I2 (learn): LLM-teacher paraphrases could be admitted as silver data only if the witness parsers produce the same frame graph as for the source sentence, with that check acting as the filter.
  • I3 (learn): each repair rule or data source could be added as a separate measured step, tracking several metrics so a gain on one does not hide a loss on another.
  • I4 (tree): this supports fully training a modest, English-specialised gparse encoder rather than relying on heavily quantised adapters on top of a big general model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh28 works · 29 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksSocial Psychology · 330 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Sajib Bhattacharjee, Ratnajit Dhar, Kawsar Ahmed and Mohammed Moshiul Hoque. 2025. Binary_Bunch at AraHealthQA Track 1: Arabic Mental Health Q&A Classification Using Data Augmentation and Transformer Models. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.24

@inproceedings{bhattacharjee2025binary,
  title = {{Binary\_Bunch at AraHealthQA Track 1: Arabic Mental Health Q\&A Classification Using Data Augmentation and Transformer Models}},
  author = {Sajib Bhattacharjee, Ratnajit Dhar, Kawsar Ahmed and Mohammed Moshiul Hoque},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.24},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.24}
}

References

18 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. AR-BERT & MARBERT: Deep bidirectional transformers for Arabic. Association for Computational Linguistics. ↗
  2. Asma Abdulsalam, Areej Alhothali, and Saleh Al-Ghamdi. 2023. Detecting suicidality in arabic tweets using machine learning and deep learning techniques. Preprint, arXiv:2309.00246. ↗
  3. Mouath Abu Daoud, Chaimae Abouzahir, Leen Kharouf, Walid Al-Eisawi, Nizar Habash, and Farah E Shamout. 2025. Medarabiq: Benchmarking large language models on arabic medical tasks.
  4. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. ↗
  5. Ashwag Alasmari. 2025. A scoping review of arabic natural language processing for mental health. In Healthcare. MDPI. find
  6. Hassan Alhuzali and Ashwag Alasmari. 2025. Pre- trained language models for mental health: An empirical study on arabic qa classification. Healthcare, 13(9). find
  7. Hassan Alhuzali, Ashwag Alasmari, and Hamad Alsaleh. 2024. Mentalqa: An annotated arabic corpus for questions and answers of mental healthcare. IEEE Access. find
  8. Suzan Elmajali and Irfan Ahmad. 2024. Toward early detection of depression: Detecting depression symptoms in arabic tweets using pretrained transformers. IEEE Access, 12:88134–88145. find
  9. Hassan Alhuzali, Farah Shamout, Muhammad Abdul-Mageed, Chaimae Abouzahir, Mouath Abu-Daoud, Ashwag Alasmari, Walid Al-Eisawi, Renad Al-Monef, Ali Alqahtani, Lama Ayash, Nizar Habash, and Leen Kharouf. 2025. Arahealthqa 2025 shared task description paper. In Proceedings of ArabicNLP 2025. find
  10. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  11. Monira Abdulrahman Almeqren, Latifah Almuqren, Fatimah Alhayan, Alexandra I Cristea, and Diane Pennington. 2023. Using deep learning to analyze the psychological effects of covid-19. Frontiers in Psychology. find
  12. Bushra Alsmadi. 2024. Deberta-bilstm: A multi-label classification model of arabic medical questions using pre-trained models and deep learning. Computers in Biology and Medicine, 170:107921. find
  13. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104. ↗
  14. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. Dora: Weight-decomposed low-rank adaptation. In Forty-first International Conference on Machine Learning. find
  15. Esraa M Rabie, Atef F Hashem, and Fahad Kamal Alsheref. 2025. Recognition model for major depressive disorder in arabic user-generated content. Beni-Suef University Journal of Basic and Applied Sciences. find
  16. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems. ↗
  17. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. ↗
  18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bhattacharjee-2025-binary-bunch-arahealthqa-track-1/