Elamur Research⌕
☰
home / papers / 2025 / EHSAN: Leveraging ChatGPT in a Hybrid Framework for Arabic Aspect-Based Sentiment Analysis in Healthcare

EHSAN: Leveraging ChatGPT in a Hybrid Framework for Arabic Aspect-Based Sentiment Analysis in Healthcare

Summary

Patient reviews written in Arabic are rarely analysed automatically. Two obstacles stand in the way: the many dialects, and the near absence of labels that tie a sentiment to a specific service aspect. The authors build EHSAN from Google Maps reviews of three Saudi hospitals in Riyadh, Jeddah and Dammam. Reviews are split into sentences, and ChatGPT tags each sentence with one aspect, aligned to the Saudi healthcare complaint taxonomy, and a polarity (positive, negative or neutral). ChatGPT also writes a short justification for every label. Human checking is applied at three depths, producing three training sets: fully reviewed, half reviewed, and machine-only. AraBERT and a multilingual DistilBERT are fine-tuned on each set for both aspect and sentiment classification, using either 17 fine-grained or 6 merged aspect classes. AraBERT stays accurate even when trained on ChatGPT-only labels, losing only a little. Collapsing the taxonomy to 6 classes improves every metric. The work suggests that LLM pre-annotation with limited expert checking is a cheap way to build resources for low-resource languages.

Key points

  • EHSAN covers 6,000 Saudi hospital reviews and annotates them per sentence, with one aspect and one sentiment per sentence. Review-level aspect information is obtained by aggregating the sentence labels.
  • The three supervision levels are fully supervised (all labels human-checked), semi-supervised (50% checked) and unsupervised (ChatGPT labels only).
  • Two aspect schemes are compared: a 17-category taxonomy and a consolidated 6-category one.
  • Both AraBERT and DistilBERT were tested, and the Arabic-specific AraBERT was the more robust model under weak supervision.
  • Unlike the earlier PX dataset (19,000 comments, internal, no sentiment) and HEAR (25,156 reviews, review-level), EHSAN is public and includes rationales. It is archived on Zenodo (doi 10.5281/zenodo.15418860).

Interesting ideas

  • I1. Train the same model on graded amounts of human checking (0%, 50%, 100%) to measure how much human review is actually needed, instead of assuming full review is required. — For Mova (learn): Run silver-data experiments with tiers of LLM-teacher labels versus hand-checked labels, to find the cheapest review budget that keeps parser and frame accuracy.
  • I2. Ask the LLM annotator to give a justification alongside every label, so reviewers can verify a decision quickly and the label stays auditable. — For Mova (learn): Require LLM teacher tiers to return a reason with each proposed arc or role, so disagreements with witness parsers can be triaged and turned into repair rules.
  • I3. Keep a fine and a coarse version of the label inventory and report results at both levels, which shows how much error comes from granularity alone. — For Mova (frames): Evaluate semantic-role and frame assignment both at fine VerbNet subclass level and at merged coarse role sets, to separate true confusions from taxonomy hair-splitting.
  • I4. Label at the sentence level and derive document-level judgments by aggregation, rather than labelling whole documents with multiple labels. — For Mova (world): Build per-sentence states and attitudes first, then compose them into the world of the text, so every document-level conclusion traces back to specific sentences.
  • I5. Store a provenance chain for every label: the prompt, the raw model reply, and any human correction. — For Mova (learn): Record which teacher, prompt and correction produced each silver tree, keeping training data explainable and making it easy to roll back bad sources.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2508.02574 · arXiv 2508.02574

@misc{alamoudi2025ehsan,
  title = {{EHSAN: Leveraging ChatGPT in a Hybrid Framework for Arabic Aspect-Based Sentiment Analysis in Healthcare}},
  author = {Alamoudi, Eman and Solaiman, Ellis},
  year = {2025},
  doi = {10.48550/arxiv.2508.02574},
  eprint = {2508.02574},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2508.02574}
}

References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Felix Greaves, Daniel Ramirez-Cano, Christopher Millett, Ara Darzi, and Liam Donaldson. 2013. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res 15, 11 (2013). https://doi.org/10.2196/jmir.2721 ↗
  2. Anastazia Zunic, Padraig Corcoran, and Irena Spasic. 2020. Sentiment analysis in health and well-being: Systematic review. JMIR Medical Informatics 8. https://doi.org/10.2196/16023 ↗
  3. Diogo Cunha Ferreira, Inês Vieira, Maria Isabel Pedro, Paulo Caldas, and Miguel Varela. 2023. Patient Satisfaction with Healthcare Services and the Techniques Used for its Assessment: A Systematic Literature Review and a Bibliometric Analysis. Healthcare (Switzerland) 11. https://doi.org/10.3390/healthcare11050639 ↗
  4. Seba AlNasser and Sarab AlMuhaideb. 2024. Listening to Patients: Advanced Arabic Aspect-Based Sentiment Analysis Using Transformer Models Towards Better Healthcare. Big Data and Cognitive Computing 8, 11 (November 2024). https://doi.org/10.3390/bdcc8110156 ↗
  5. Lumeon. 2020. U.S. Patient Access Leadership Research 2020/21. Technical report. Lumeon Ltd., in partnership with Sage Growth Partners. Retrieved April 10, 2025 from https://info.lumeon.com/patient-access-leadership-research-report ↗
  6. Centers for Medicare & Medicaid Services (CMS). n.d. CAHPS Hospital Survey. Centers for Medicare & Medicaid Services. Retrieved April 10, 2025 from https://www.hcahpsonline.org/en/search-results/?q=CAHPS+Hospital+Survey ↗
  7. Ministry of Health, Kingdom of Saudi Arabia. n.d. Ministry of Health Official Portal. Ministry of Health. Retrieved April 10, 2025 from https://www.moh.gov.sa/en/Pages/Default.aspx ↗
  8. Michelle Beattie, Douglas J. Murphy, Iain Atherton, and William Lauder. 2015. Instruments to measure patient experience of healthcare quality in hospitals: A systematic review. Syst Rev 4, 1 (July 2015). https://doi.org/10.1186/s13643-015-0089-0 ↗
  9. Mátyás Osváth, Zijian Győző Yang, and Karolina Kósa. 2023. Analyzing Narratives of Patient Experiences: A BERT Topic Modeling Approach. Acta Polytech. Hungarica 20, 7 (2023), 153–171. https://doi.org/10.12700/APH.20.7.2023.7.9 ↗
  10. Ankita Bansal and Niranjan Kumar. 2022. Aspect-Based Sentiment Analysis Using Attribute Extraction of Hospital Reviews. New Gener Comput 40, 4 (December 2022), 941–960. https://doi.org/10.1007/s00354-021-00141-3 ↗
  11. Najla Z. Alhazzani, Isra M. Al-Turaiki, and Sarah A. Alkhodair. 2023. Text Classification of Patient Experience Comments in Saudi Dialect Using Deep Learning Techniques. Applied Sciences (Switzerland) 13, 18 (September 2023). https://doi.org/10.3390/app131810305 ↗
  12. Usman Malik, Simon Bernard, Alexandre Pauchet, Clement Chatelain, Romain Picot-Clemente, and Jerome Cortinovis. 2024. Pseudo-Labeling with Large Language Models for Multi-Label Emotion Classification of French Tweets. IEEE Access 12, (2024), 15902–15916. https://doi.org/10.1109/ACCESS.2024.3354705 ↗
  13. Jiaxuan Li, Yunchu Yang, Rong Chen, Dashun Zheng, Patrick Cheong Iao Pang, Chi Kin Lam, Dennis Wong, and Yapeng Wang. 2025. Identifying healthcare needs with patient experience reviews using ChatGPT. PLoS One 20, 3 March (March 2025). https://doi.org/10.1371/journal.pone.0313442 ↗
  14. Abdulmohsen Al-Thubaity, Sakhar Alkhereyf, Hanan Murayshid, Nouf Alshalawi, Maha Bin Omirah, Raghad Alateeq, Rawabi Almutairi, Razan Alsuwailem, Manal Alhassoun, and Imaan Alkhanen. 2023. Evaluating ChatGPT and Bard AI on Arabic Sentiment Analysis. In Proceedings of the Third Arabic Natural Language Processing Workshop (ArabicNLP 2023). Association for Computational Linguistics, Singapore (Hybrid), 335–349. https://doi.org/10.18653/v1/2023.arabicnlp-1.27 ↗
  15. Saudi Central Board for Accreditation of Healthcare Institutions (CBAHI). 2025. Hospitals Accreditation Status till 1 May 2025. Saudi Central Board for Accreditation of Healthcare Institutions. Retrieved April 10, 2025 from https://portal.cbahi.gov.sa/Library/Assets/hos-ar-035644.pdf ↗
  16. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python Natural Language Processing Toolkit for Many Human Languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, Online, 101–108. https://doi.org/10.18653/v1/2020.acl-demos.14 ↗ our card
  17. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 1 (1960), 37–46. https://doi.org/10.1177/001316446002000104 ↗
  18. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based Model for Arabic Language Understanding. arXiv preprint arXiv:2003.00104 (February 2020). Retrieved from http://arxiv.org/abs/2003.00104 ↗ our card
  19. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. (October 2019). Retrieved from http://arxiv.org/abs/1910.01108 ↗
  20. Eman Alamoudi and Ellis Solaiman, EHSAN: Leveraging ChatGPT in a Hybrid Framework for Arabic Aspect-Based Sentiment Analysis in Healthcare, in Proceedings of the International Conference on Intelligent Data Engineering and Automated Systems (IDEAS), Newcastle, UK, Springer, 2025. find our card
  21. Hissah Almousa and Ellis Solaiman, A Vision for Robust and Human-Centric LLM-Based QR Code Security, in Proceedings of the International Conference on Intelligent Data Engineering and Automated Systems (IDEAS), Newcastle, UK, Springer, 2025. find
  22. Banan Alkhateeb and Ellis Solaiman, Context-Aware Visualization for Explainable AI Recommendations in Social Media: A Vision for User-Aligned Explanations, in Proceedings of the International Conference on Intelligent Data Engineering and Automated Systems (IDEAS), Newcastle, UK, Springer, 2025. find
  23. Maryam Mosleh, Ellis Solaiman, and Marie Delvin, Transparent Adaptive Learning via Data-Centric Multimodal Explainable AI, in Proceedings of the International Conference on Intelligent Data Engineering and Automated Systems (IDEAS), Newcastle, UK, Springer, 2025. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alamoudi-2025-ehsan-leveraging-chatgpt-hybrid-framework/