Elamur Research⌕
☰
home / papers / 2021 / New Arabic Medical Dataset for Diseases Classification

New Arabic Medical Dataset for Diseases Classification

Summary

Arabic NLP has few training datasets, and the ones available for text classification are general news collections (politics, sports, economics) rather than specialist material. The authors address this with a medical topic-classification resource. Using Request, Beautiful Soup and Selenium, they semi-automatically collected 2,000 Arabic documents from several medical websites and the Arab Medical Encyclopedia. Each document belongs to one of ten disease groups, and three physicians from Syrian university hospitals checked the labels.

As baselines, they fine-tuned three Transformer encoders: multilingual BERT, AraBERT v2, and ABioNER, an AraBERT model further pre-trained on Arabic biomedical literature. They compared these with SVM and Naive Bayes over TF-IDF features. The domain-adapted ABioNER scored highest, with test F1 near 96. Generic BERT reached about 92, and the shallow methods reached 86–87. The study shows that pre-training closer to the target language and domain gives measurable gains. It also supplies an Arabic medical benchmark, planned to grow into NER and QA tasks.

Key points

  • Dataset size and classes: 2,000 documents, each assigned to one of ten disease categories: Nephrological, Liver, Immune, Gastrointestinal, Eye, Endocrine, Ear, Cardiovascular, Bone and Blood.

    • Classes are roughly balanced at 180–218 documents each.
    • Average document length is about 1,080–1,750 words and 22–27 sentences.

  • Sources: altibbi.com, webteb.com, mayoclinic.org, dailymedicalinfo.com and the Arab Medical Encyclopedia (arab-ency.com.sy).

  • Data split and setup: 80/10/10 train/validation/test.

    • Each encoder is topped with a 1024-unit ReLU dense layer and a 10-way softmax.
    • Training used Adam, batch size 16 and maximum sequence length 512.
    • The paper gives two different learning rates: 1e-4 in the setup description and 2e-5 as the best found value.

  • Test F1 scores:

    ModelTest F1
    ABioNER95.91
    AraBERT v294.54
    BERT (multilingual cased)92.29
    SVM87.35
    Naive Bayes85.69

  • Shallow pipeline: NLTK tokenization, stop-word removal, Snowball stemming and TF-IDF, trained with scikit-learn. The released data itself is meant to stay unpreprocessed.

Interesting ideas

  • I1. The three encoders form a ladder of increasing closeness to the target: general multilingual, then language-specific, then language plus domain. This isolates how much each adaptation step adds. — For Mova (learn): Train parser or tagger variants on general silver data versus domain-specific silver data (e.g., fables, school texts) and report each step's gain, so domain adaptation is measured rather than assumed.
  • I2. The corpus is distributed as raw text, on the argument that steps like stop-word removal can strip context that contextual models rely on. — For Mova (tokens): Store the untouched source text next to every tokenized and lemmatized layer, so later stages can recover anything that normalization would otherwise discard.
  • I3. Labels gathered by scraping are confirmed by several domain experts, combining cheap collection with targeted human verification. — For Mova (learn): Silver trees and frames agreed by witness parsers could be spot-checked by an LLM-teacher tier or human reviewer on a sample, giving an estimate of the silver data's error rate.
  • I4. TF-IDF with SVM and Naive Bayes is kept as a floor that the stronger models must clearly beat. The authors also read the baselines' decent scores as a sign that the dataset is well balanced. — For Mova (answer): Use a bag-of-words baseline as a negative control on school-test and fable QA; if the explainable pipeline does not beat it, the questions may be solvable by surface cues.
  • I5. The dataset card lists, per class, the sample count and the average words and sentences per document, which makes length and balance biases visible at once. — For Mova (answer): Publish similar per-category length and size statistics for Mova's test suites, so differences in accuracy across categories can be checked against differences in text length.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2106.15236 · arXiv 2106.15236

@misc{hammoud2021new,
  title = {{New Arabic Medical Dataset for Diseases Classification}},
  author = {Hammoud, Jaafar and Vatian, Aleksandra and Dobrenko, Natalia and Vedernikov, Nikolai and Shalyto, Anatoly and Gusarova, Natalia},
  year = {2021},
  doi = {10.48550/arxiv.2106.15236},
  eprint = {2106.15236},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2106.15236}
}

References

45 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Li, Q., Peng, H., Li, J., Xia, C., Yang, R., Sun, L., ... & He, L. (2020). A Text Classification Survey: From Shallow to Deep Learning. arXiv preprint arXiv:2008.00364. ↗
  2. Yao, L., Mao, C., & Luo, Y. (2019). Clinical text classification with rule-based features and knowledge-guided convolutional neural networks. BMC medical informatics and decision making, 19(3), 71. find
  3. Suzdaltseva, M., Shamakhova, A., Dobrenko, N., Alekseeva, O., Hammud, J., Gusarova, N., Vatian, A., & Shalyto, A. (2021). De-identification of medical information for forming multimodal datasets to train neural networks: Proceedings of the 7th International Conference on Information and Communication Technologies for Ageing Well and E-Health, 163– 170. https://doi.org/10.5220/0010406001630170 4. Hammoud, J., Dobrenko, N., & Gusarova, N. (2020). Named entity recognition and information extraction for Arabic medical text. In Multi Conference on Computer Science and Information Systems, MCCSIS 2020-Proceedings of the International Conference on e-Health (pp. 121-127). ↗
  4. Alalyani, N., & Marie-Sainte, S. L. (2018). NADA: New Arabic dataset for text classification. International Journal of Advanced Computer Science and Applications, 9(9). find
  5. Zhang, Y., Jin, R., & Zhou, Z. H. (2010). Understanding bag-of-words model: a statistical framework. International Journal of Machine Learning and Cybernetics, 1(1-4), 43-52. find
  6. Pennington, J., Socher, R., & Manning, C. D. (2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543). find
  7. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  8. Cavnar, W. B., & Trenkle, J. M. (1994, April). N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval (Vol. 161175). 10. “Term frequency by inverse document frequency,” in Encyclopedia of Database Systems, p. 3035, 2009. find
  9. Zhang, M. L., & Zhang, K. (2010, July). Multi-label learning by exploiting label dependency. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 999-1008). find
  10. A. van den Bosch, “Hidden markov models,” in Encyclopedia of Machine Learning and Data Mining, pp. 609–611, 2017. find
  11. Maron, M. E. (1961). Automatic indexing: an experimental inquiry. Journal of the ACM (JACM), 8(3), 404-417. find
  12. O'Donnell, M. (2008). CATALOGING AND CLASSIFICATION: AN INTRODUCTION. Lois Mai Chan. Lanham, MD: Scarecrow Press, 2007, 580 pp., ISBN 0-8108-6000-7. Technical Services Quarterly, 26(1), 86-87. find
  13. Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1), 21-27. find
  14. Dramé, K., Mougin, F., & Diallo, G. (2016). Large scale biomedical texts classification: a kNN and an ESA-based approaches. Journal of biomedical semantics, 7(1), 1-12. find
  15. Cortes, C., & Vapnik, V. (1995). Support-vector networks Machine learning (pp. 237–297), Vol. 20. Boston, MA: Kluwer Academic Publisher. find
  16. Bui, D. D. A., & Zeng-Treitler, Q. (2014). Learning regular expressions for clinical text classification. Journal of the American Medical Informatics Association, 21(5), 850-857. find
  17. Chern, C. C., Chen, Y. J., & Hsiao, B. (2019). Decision tree–based classifier in providing telehealth service. BMC medical informatics and decision making, 19(1), 1-15. find
  18. Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., & Gao, J. (2021). Deep Learning--based Text Classification: A Comprehensive Review. ACM Computing Surveys (CSUR), 54(3), 1-40. find
  19. Tai, K. S., Socher, R., & Manning, C. D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075. ↗
  20. Liu, P., Qiu, X., Chen, X., Wu, S., & Huang, X. J. (2015, September). Multi-timescale long short-term memory neural network for modelling sentences and documents. In Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 2326-2335). find
  21. Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., & Xu, B. (2016). Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling. arXiv preprint arXiv:1611.06639. ↗
  22. Conneau, A., Schwenk, H., Barrault, L., & Lecun, Y. (2016). Very deep convolutional networks for text classification. arXiv preprint arXiv:1606.01781. ↗ our card
  23. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. ↗ our card
  24. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). find
  25. Mou, L., Men, R., Li, G., Xu, Y., Zhang, L., Yan, R., & Jin, Z. (2015). Natural language inference by tree-based convolution and heuristic matching. arXiv preprint arXiv:1512.08422. ↗ our card
  26. Karimi, S., Dai, X., Hassanzadeh, H., & Nguyen, A. (2017, August). Automatic diagnosis coding of radiology reports: a comparison of deep learning and conventional classification methods. In BioNLP 2017 (pp. 328-332). find
  27. Peng, S., You, R., Wang, H., Zhai, C., Mamitsuka, H., & Zhu, S. (2016). DeepMeSH: deep semantic representation for improving large-scale MeSH indexing. Bioinformatics, 32(12), i70-i79. find
  28. Rios, A., & Kavuluru, R. (2015, September). Convolutional neural networks for biomedical text classification: application in indexing biomedical articles. In Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics (pp. 258-267). find
  29. Hughes, M., Li, I., Kotoulas, S., & Suzumura, T. (2017). Medical text classification using convolutional neural networks. Stud Health Technol Inform, 235, 246-50. find
  30. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762. ↗
  31. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  32. Hammoud, J., Eisab, A., Dobrenkoa, N., & Gusarovaa, N. (2021). Using a New Nonlinear Gradient Method for Solving Large Scale Convex Optimization Problems with an Application on Arabic Medical Text. arXiv preprint arXiv:2106.04383. ↗ our card
  33. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. find
  34. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9. find
  35. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  36. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. ↗
  37. Marie-Sainte, S. L., & Alalyani, N. (2020). Firefly algorithm based feature selection for Arabic text classification. Journal of King Saud University-Computer and Information Sciences, 32(3), 320-328. find
  38. Saad, M. K., & Ashour, W. M. (2010). Osac: Open source arabic corpora. In 6th ArchEng Int. Symposiums, EEECS (Vol. 10). find
  39. Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015). Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE international conference on computer vision (pp. 19-27). find our card
  40. Antoun, W., Baly, F., & Hajj, H. (2020). Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104. ↗ our card
  41. El-Khair, I. A. (2016). 1.5 billion words arabic corpus. arXiv preprint arXiv:1611.04033. ↗
  42. Zeroual, I., Goldhahn, D., Eckart, T., & Lakhouaja, A. (2019, August). OSIAN: Open source international Arabic news corpus-preparation and integration into the CLARIN-infrastructure. In Proceedings of the Fourth Arabic Natural Language Processing Workshop (pp. 175-182). find
  43. Boudjellal, N., Zhang, H., Khan, A., Ahmad, A., Naseem, R., Shang, J., & Dai, L. (2021). ABioNER: a BERT-based model for Arabic biomedical named-entity recognition. Complexity, 2021. find
  44. WHO EMRO, “EMHJ home,” Eastern Mediterranean Health Journal, vol. 27, 2021, http://www.emro.who.int/emhjournal/eastern-mediterranean-health-journal/home.html. ↗
  45. Adhikari, A., Ram, A., Tang, R., & Lin, J. (2019). Docbert: Bert for document classification. arXiv preprint arXiv:1904.08398. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hammoud-2021-new-arabic-medical-dataset-diseases/