Elamur Research⌕
☰
home / papers / 2025 / HTU at MAHED Shared Task: Ensemble-Based Classification of Arabic Hate and Hope Speech Using Pre-trained Dialectal Arabic Models

HTU at MAHED Shared Task: Ensemble-Based Classification of Arabic Hate and Hope Speech Using Pre-trained Dialectal Arabic Models

Paper · 2025 · Abdallah Saleh and Mariam M Biltawi

Summary

The paper describes the winning system for sub-task 1 of MAHED 2025. The task is three-way classification of Arabic social media text into hate, hope, or neither, and the data contains both the formal standard form of the language (MSA) and its regional spoken varieties (DA). The authors argue that most Arabic encoders are trained mostly on one variety and that pre-training from scratch is expensive, so they rely on continual pre-training instead. They introduce two models, BERT-DA and ArabicDeBERTa-DA, which were first trained with MLM on MSA and then further trained on dialectal text. They combine these with MARBERTV2, which went the opposite way: dialect first, then MSA. Each encoder receives a small classification head. The three outputs are merged by averaging logits, and training uses a class-weighted cross-entropy loss. Single models reach about 67% macro F1. The ensemble gains roughly five to six points, and a final hard vote over several seeded ensemble runs reached 72.3% macro F1, first place in the competition. The authors note that a system of about 0.5B parameters is too heavy for edge or CPU deployment.

Key points

  • Training data was imbalanced: 3,697 not_applicable, 1,892 hope, and 1,301 hate instances. Preprocessing stripped English, emoticons, symbols and diacritics, and inputs were fixed at 128 tokens.
  • BERT-DA and ArabicDeBERTa-DA were pre-trained on about 2.2B MSA tokens, then continually pre-trained on DA text from Al-Fetyani et al. (2023). MARBERTV2 started from about 15.6B dialectal tokens and was then adapted to MSA.
  • Each encoder's classification head is linear → Tanh → linear over three classes. Training ran for 5 epochs with batch size 16, AdamW, and a learning rate of 7e-6. Class weights were 1.02 for hate, 0.98 for hope and 1.0 for not_applicable.
  • The ensemble peaked after only about three quarters of the first epoch, at 72.2% F1. Hard voting across the best seeded ensemble runs raised this to 72.3%, which ranked first.
  • The main stated limitation is the system size of nearly half a billion parameters, which is impractical for production and resource-constrained settings.

Interesting ideas

  • I1. Opposite pre-training orders. Ensemble members were adapted in reverse orders (MSA then dialect versus dialect then MSA), so they likely make different errors, which is where voting gains come from.
  • I2. Two voting levels. Members inside one ensemble are fused by averaging logits, and complete ensemble runs are then combined by a discrete majority vote.
  • I3. Seeds as a cheap source of diversity. Changing only the initialization seed of the heads was enough to give the final vote something to gain.
  • I4. Validating within an epoch. The best checkpoint appeared before the first epoch ended, so evaluating only at epoch boundaries would have missed it.
  • I5. Continuing training instead of retraining. Adapting an existing model to a new language variety was presented as cheaper than training from scratch while keeping the original skills.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): train gparse variants with opposite treebank curricula (for example, formal-genre treebank then web-genre, and the reverse) so the parsers disagree in useful, non-redundant ways.
  • I2 (learn): when building silver data, average scores within one parser family but require a hard vote across independent witness families before accepting an arc.
  • I3 (learn): run the perceptron parser with several seeds and data shuffles as extra low-cost witnesses, and treat the size of their disagreement as a confidence signal.
  • I4 (learn): validate gparse fine-tuning runs several times per epoch, so an early peak is kept instead of being overwritten by later overfitting.
  • I5 (tokens): adapt the en tagger and lemmatizer to fable or school-test text by continuing training on in-domain silver data, then check that accuracy on the original domain has not dropped.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Al Hussein Technical University.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Abdallah Saleh and Mariam M Biltawi. 2025. HTU at MAHED Shared Task: Ensemble-Based Classification of Arabic Hate and Hope Speech Using Pre-trained Dialectal Arabic Models. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.89

@inproceedings{saleh2025htu,
  title = {{HTU at MAHED Shared Task: Ensemble-Based Classification of Arabic Hate and Hope Speech Using Pre-trained Dialectal Arabic Models}},
  author = {Abdallah Saleh and Mariam M Biltawi},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.89},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.89}
}

References

18 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Mohammad Al-Fetyani, Muhammad Al-Barham, Gheith Abandah, Adham Alsharkawi, and Maha Dawas. 2023. Masc: Massive arabic speech corpus. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 1006–1013. IEEE. find
  3. Malik Almaliki, Abdulqader M Almars, Ibrahim Gad, and El-Sayed Atlam. 2023. Abmm: Arabic bert-mini model for hate-speech detection on social media. Electronics, 12(4):1048. ↗
  4. Faisal Yousif Al Anezi. 2022. Arabic hate speech detection using deep recurrent neural networks. Applied Sciences, 12(12):6010. ↗
  5. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104. ↗
  6. S ArunaDevi and B Bharathi. 2024. Machine learning based approach for hope speech detection. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS. org. find
  7. Bharathi Raja Chakravarthi. 2022. Hope speech detection in youtube comments. Social Network Analysis and Mining, 12(1):75. ↗
  8. Ankita Gandhi, Param Ahir, Kinjal Adhvaryu, Pooja Shah, Ritika Lohiya, Erik Cambria, Soujanya Poria, and Amir Hussain. 2024. Hate speech detection: A comprehensive review of recent works. Expert Systems, 41(8):e13562. ↗
  9. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org. ↗
  10. Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, and Irina Rish. 2024. Simple and scalable strategies to continually pre-train large language models. arXiv preprint arXiv:2403.08763. ↗
  11. Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. 2023. Continual pre-training of language models. arXiv preprint arXiv:2302.03241. ↗
  12. Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24):8788–8790. ↗
  13. Kuan-Ying Lee, Yuanyi Zhong, and Yu-Xiong Wang. 2023. Do pre-trained models benefit equally in continual learning? In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6485–6493. find
  14. Soumajyoti Sarkar, Kaixiang Lin, Sailik Sengupta, Leonard Lausen, Sheng Zha, and Saab Mansour. 2022. Parameter and data efficient continual pre-training for robustness to dialectal variance in arabic. arXiv preprint arXiv:2211.03966. ↗
  15. Wajdi Zaghouani and Md Rafiul Biswas. 2025a. An annotated corpus of arabic tweets for hate speech analysis. arXiv preprint arXiv:2505.11969. ↗
  16. Wajdi Zaghouani and Md Rafiul Biswas. 2025b. Emohopespeech: An annotated dataset of emotions and hope speech in english and arabic. arXiv preprint arXiv:2505.11959. ↗
  17. Wajdi Zaghouani, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgios Mikros, Abul Hasnat, and Firoj Alam. 2025. MAHED shared task: Multimodal detection of hope and hate emotions in arabic content. In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics.
  18. Wajdi Zaghouani, Hamdy Mubarak, and Md Rafiul Biswas. 2024. So hateful! building a multi-label hate speech annotated arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/saleh-2025-htu-mahed-shared-task-ensemble/