Elamur Research⌕
☰
home / papers / 2025 / AraNLP at MAHED 2025 Shared Task: Using AraBERT for Text-based Hate and Hope Speech Classification

AraNLP at MAHED 2025 Shared Task: Using AraBERT for Text-based Hate and Hope Speech Classification

Paper · 2025 · Wafaa S. El-Kassas and Enas A. Hakim Khalil

Summary

The paper describes AraNLP, a system for Sub-task 1 of the MAHED 2025 shared task. Each Arabic tweet, written in MSA or dialect, must be labelled as hate, hope, or not_applicable. The authors fine-tune the tweet-pretrained AraBERT checkpoint (aubmindlab/bert-base-arabertv02-twitter) on the organisers' dataset of 6893 training, 1476 validation and 1477 test tweets. The class distribution is skewed toward not_applicable. Their pipeline is ArabertPreprocessor cleaning, AutoTokenizer, a 128-token limit chosen from a length analysis, and two epochs on Colab.

The submitted run reached about 65% macro F1, which placed 20th. Two later changes raised the score:

  • switching how output labels were extracted from the saved model;
  • retraining on merged train and validation data with 5-fold cross-validation and 5 epochs.

The final macro F1 of about 70% would sit roughly at rank 4. The error analysis shows the model falls back on not_applicable when uncertain, and hope is the hardest class. The work matters as a simple, reproducible baseline for Arabic hate and hope detection, where resources and dialect coverage are limited.

Key points

  • Data and labels:
    • Training split: 1892 hope, 1304 hate and 3697 not_applicable tweets.
    • Test split: 422 hope, 287 hate and 768 not_applicable tweets.
  • Model and settings:
    • The model is AraBERTv02-twitter, a BERT-base model (12 layers, 768 hidden units, 110M parameters) pretrained on 60M Arabic tweets.
    • Hyperparameters: learning rate 2e-5, train batch 16, gradient accumulation 2, seed 25.
    • The best checkpoint was chosen by macro_f1.
  • Truncation: sequences were cut at 128 tokens, which affected only 27 training and 5 validation tweets.
  • Results across runs:
    • Test 1 (leaderboard): macro F1 0.649.
    • Test 2 (different label-prediction code, same model): about 0.67.
    • Test 3 (train+val merged, 5-fold CV, 5 epochs): about 0.70 macro F1 and 0.70 accuracy.
  • Hope-class weakness: hope recall rose from 0.51 (Test 2) to 0.63 (Test 3), but hope remained the weakest class. Hate recall was about 0.68–0.69 and not_applicable recall 0.75–0.78.

Interesting ideas

  • I1. Validation loss crept up slightly between epochs while macro F1 still rose, so picking checkpoints by the target metric rather than by loss gave the better model.
  • I2. With the same trained weights, swapping the label-extraction code at inference changed the test score noticeably, so the inference wrapper itself is a hidden source of error.
  • I3. Before fixing a maximum input length, the authors plotted the length distribution and counted exactly how many examples would be truncated.
  • I4. They chose a checkpoint pretrained on the same register as the task (tweets with slang, hashtags, emojis) to get better vocabulary coverage.
  • I5. The per-class confusion analysis exposed a fallback toward the majority "nothing" label under uncertainty, which hid poor recall on the minority hope class.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): gparse and tagger training could save checkpoints by LAS or per-label F1 instead of loss, logging both so divergences between them become visible.
  • I2 (learn): add an evaluation check that runs the training-time scorer and the deployed prediction path on one fixed set and flags any disagreement as a negative-control failure.
  • I3 (tree): gparse's sequence limit could be set from treebank length statistics, reporting which sentences get cut so long-sentence parsing failures are not silently hidden.
  • I4 (tokens): measure the en tokenizer's and tagger's coverage on social-media style input separately and add register-specific lexicon entries where coverage drops.
  • I5 (frames): speech-act and irony detection in prag should report per-class recall, so a default "neutral/assertion" fallback cannot mask missed rarer acts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬National Research CentreGiza · Egypt2 works · 11 citations

Also on the byline: Egyptian e-Learning University.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Wafaa S. El-Kassas and Enas A. Hakim Khalil. 2025. AraNLP at MAHED 2025 Shared Task: Using AraBERT for Text-based Hate and Hope Speech Classification. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.97

@inproceedings{elkassas2025aranlp,
  title = {{AraNLP at MAHED 2025 Shared Task: Using AraBERT for Text-based Hate and Hope Speech Classification}},
  author = {Wafaa S. El-Kassas and Enas A. Hakim Khalil},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.97},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.97}
}

References

13 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelsamie, Mahmoud Mohamed, Shahira Shaaban Azab, and Hesham A. Hefny. 2026. “The Dialects Gap: A Multi-Task Learning Approach for Enhancing Hate Speech Detection in Arabic Dialects.” Expert Systems with Applications 295:128584. doi:https://doi.org/10.1016/j.eswa.2025.128584. ↗
  2. Al-Saqqa, Samar. 2024. “Hate Speech Detection of Arabic Text Using Deep Learning and Transfer Learning Models.” PhD Thesis, Princess Sumaya University for Technology (Jordan). find
  3. Al-Saqqa, Samar, Arafat Awajan, and Bassam Hammo. 2024. “A Survey of Hate Speech Detection for Arabic Social Media: Methods and Datasets.” Procedia Computer Science 251:224– 31. doi:https://doi.org/10.1016/j.procs.2024.11.104. ↗
  4. Alwateer, M., I. Gad, M. Elmarhomy, G. Elmarhomy, H. Hashim, M. Almaliki, and E. S. Atlam. 2025. “Interpretable Arabic Hate Speech Detection Using Large Language Model.” Pp. 1–8 in 2025 2nd International Conference on Advanced Innovations in Smart Cities (ICAISC). find
  5. Antoun, Wissam, Fady Baly, and Hazem Hajj. 2020. “AraBERT: Transformer-Based Model for Arabic Language Understanding.” Pp. 9–15 in Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, edited by H. Al-Khalifa, W. Magdy, K. Darwish, T. Elsayed, and H. Mubarak. Marseille, France: European Language Resource Association. ↗
  6. HuggingFace. n.d. “Bert-Base-Arabertv02-Twitter.” Retrieved September 12, 2025. https://huggingface.co/aubmindlab/bert-base-arabertv02-twitter. ↗
  7. Khezzar, Ramzi, Abdelrahman Moursi, and Zaher Al Aghbari. 2023. “ArHateDetector: Detection of Hate Speech from Standard and Dialectal Arabic Tweets.” Discover Internet of Things 3(1):1. doi:10.1007/s43926-023-00030-9. ↗
  8. Ramos, Gil, Fernando Batista, Ricardo Ribeiro, Pedro Fialho, Sérgio Moro, António Fonseca, Rita Guerra, Paula Carvalho, Catarina Marques, and Cláudia Silva. 2024. “A Comprehensive Review on Automatic Hate Speech Detection in the Age of the Transformer.” Social Network Analysis and Mining 14(1):204. doi:10.1007/s13278-024-01361-3. ↗
  9. Zaghouani, Wajdi, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgio Mikros, Abul Hasnat, and Firoj Alam. 2025. “MAHED Shared Task: Multimodal Detection of Hope and Hate Emotions in Arabic Content.” in Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025). Suzhou, China: Association for Computational Linguistics. Zaghouani, Wajdi, Hamdy Mubarak, and Md.
  10. Rafiul Biswas. 2024. “So Hateful! Building a Multi-Label Hate Speech Annotated Arabic Dataset.” Pp. 15044–55 in Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Torino, Italia: ELRA and ICCL. Biswas, Md.
  11. Rafiul, and Wajdi Zaghouani. 2025a. “An Annotated Corpus of Arabic Tweets for Hate Speech Analysis.” CoRR. https://arxiv.org/abs/2505.11969. Biswas, Md. ↗
  12. Rafiul, and Wajdi Zaghouani. 2025b. “EmoHopeSpeech: An Annotated Dataset of Emotions and Hope Speech in English and Arabic.” CoRR abs/2505.11959. https://arxiv.org/abs/2505.11959. ↗
  13. Fat’hAlalim, Ahmed, Yongjian Liu, Qing Xie, and Nahla Ibrahim. 2025. “Advancements in Transformer-Based Models for Enhanced Hate Speech Detection in Arabic: Addressing Dialectal Variations and Cross-Platform Challenges.” ACM Trans. Asian Low-Resour. Lang. Inf. Process. 24(8). doi:10.1145/3748492. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kassas-2025-aranlp-mahed-2025-shared-task/