📄 Olga Snissarenko at AbjadMed: Arabic Clinical Text Classification with AraBERT: Results from the AbjadMed Shared Task

Summary

This paper describes a system for the AbjadMed Shared Task. The task is to assign Arabic medical consultation texts (question–answer pairs in Modern Standard Arabic) to one of 82 categories. The label distribution is heavily long-tailed. The author does not propose a new architecture. Instead, she fine-tunes AraBERT in four configurations and compares the effects of:

• input length,
• Arabic orthographic normalization,
• mean vs. learnable attention pooling,
• class-weighted cross-entropy.

The best setup uses bert-base-arabertv02 with mean pooling, 256-token inputs and balanced class weights. It reaches macro-F1 of 0.411 on the private test set, placing 12th.

The added components did not help. Attention pooling with normalization scored lower, as did a variant with a smaller learning rate. The author concludes that handling imbalance through the loss matters more than architectural additions. The comparison is only partly controlled, however. Model variant, sequence length and weighting change together between the first two runs, which makes the effect of each factor hard to isolate.

The work offers practical, reproducible guidance for low-resource, imbalanced Arabic clinical NLP, and the code is released on GitHub.

Key points

• Data: 27,951 labeled training samples and 18,634 unlabeled DEVTEST samples, spanning 82 classes with 7 to 600 examples each. A stratified 90/10 train/validation split (seed 42) was used, and evaluation is by macro-F1.
• Models compared:
  • aubmindlab/bert-base-arabertv02-twitter (128 tokens, no weighting): 0.356 private.
  • aubmindlab/bert-base-arabertv02 with balanced class weights (scikit-learn style), 256 tokens: 0.387 public / 0.411 private.
• Effect of class weighting: the weighted run gained +0.055 private F1 over the unweighted baseline. This gain is confounded with the longer input and the switch of AraBERT variant.
• Weaker variants:
  • Attention pooling plus normalization reached 0.360 private.
  • A normalized mean-pooling run with learning rate 1×10⁻⁵ reached 0.326 private, which the author attributes to underfitting.
• Inconsistency in the paper: the method section gives the best run's learning rate as 1×10⁻⁵, while the results table lists 2×10⁻⁵.
• Hardware and future work: training ran on an Apple M4 Max (MPS backend). Proposed next steps are data augmentation, hierarchical label modeling and domain-adaptive pretraining on Arabic medical corpora.