Elamur Research⌕
☰
home / papers / 2025 / MAHED Shared Task: Multimodal Detection of Hope and Hate Emotions in Arabic Content

MAHED Shared Task: Multimodal Detection of Hope and Hate Emotions in Arabic Content

Paper · 2025 · Wajdi Zaghouani, Md. Rafiul Biswas, Mabrouka Bessghaier et al.

Summary

MAHED 2025 is a shared task on Arabic social media content. It covers two opposite affective signals: hateful speech and hopeful, prosocial speech. The organizers argue that Arabic moderation is made harder by dialect variety, code-switching and image-based formats. They also note that hope has received far less Arabic annotation than toxicity. The task has three subtasks:

  • a three-way text label (hate, hope, not_applicable);
  • joint prediction of an emotion together with offense and hate labels, under an explicit hierarchy;
  • harmful-content detection in memes that combine image and text.

The released resources total more than 22,000 items, labeled by native speakers with Cohen's Kappa above 0.85, along with baseline code. The leaderboard received 46 submissions. Participating systems used Arabic encoders (AraBERT, MARBERT), LLMs (GPT-4, Gemini) and CLIP-based fusion. The best macro-F1 scores were 0.723, 0.578 and 0.796 for the three subtasks. The error analysis points to weak spots in dialect handling, recall on the hope class and OCR quality. This gives a concrete agenda for future work on Arabic content safety.

Key points

  • The Task 1 corpus has 9,843 texts split 6,890 / 1,476 / 1,477. It merges three earlier datasets and is skewed: roughly 19% hate, 28% hope and 53% not_applicable.
  • Task 2 asks for one of 12 emotion labels (neutral, anger, joy, optimism, pessimism, trust and others) alongside offense and hate. Hate is treated as implying offense.
  • A typical meme system paired a MARBERTv2 [CLS] vector with a CLIP ViT-B/32 image embedding. Both were projected into a shared 512-d space and fused by concatenation or cross-attention.
  • The best macro-F1 scores were 0.723 (text hope/hate), 0.578 (multitask) and 0.796 (memes). Task 2 came out lowest, which the authors attribute to competing optimization pressures between the label heads.
  • The error analysis found:
    • hope recall averaged only 0.52;
    • OCR noise caused about 28% of meme errors;
    • Gulf and Levantine phrasing was frequently misclassified, and among misclassified items the dialect-related difference reached as much as 34%.

Interesting ideas

  • I1. Treating hope as its own target, rather than folding it into general positive sentiment, keeps encouraging, future-oriented speech from blurring with plain positivity.
  • I2. Declaring implications between labels (hate entails offense) turns several heads into one consistent structure rather than independent guesses.
  • I3. Model selection used macro-F1 with class-weighted loss and oversampling, so rare classes count as much as the dominant "nothing applies" class.
  • I4. The error analysis attributes failures to their source, such as dialect or OCR extraction noise, instead of reporting only a single score.
  • I5. Separate learning rates for the shared encoder and the task heads, with heads adapting about 1.8× faster, let the heads specialize without disturbing the shared representation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Speech-act tagging in prag could separate encouragement or aspiration from simple positive evaluation, so frames record the intent of a line and not just its polarity.
  • I2 (world): Compiled knowledge could store such entailments between state or attitude labels and check that inferred states in a text's world never violate them.
  • I3 (learn): Evaluation of frame or role labelers should report macro scores, so frequent defaults such as no-role or neutral cannot hide failures on rare classes.
  • I4 (learn): Because each Mova step is explainable, its error reports could assign each wrong answer to tokens, tree, frames or world, showing which stage to repair first.
  • I5 (tree): gparse could use a slower rate for the transformer body and a faster one for the arc and label heads when finetuning on a new treebank or UD dialect.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hamad bin Khalifa UniversityDoha · Qatar141 works · 14,175 citations🎓Northwestern University in QatarDoha · Qatar24 works · 148 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 worksVision & language · 657 works

Cite

Cite as Wajdi Zaghouani, Md. Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, George Mikros, Abul Hasnat and Firoj Alam. 2025. MAHED Shared Task: Multimodal Detection of Hope and Hate Emotions in Arabic Content. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.75

@inproceedings{zaghouani2025mahed,
  title = {{MAHED Shared Task: Multimodal Detection of Hope and Hate Emotions in Arabic Content}},
  author = {Wajdi Zaghouani, Md. Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, George Mikros, Abul Hasnat and Firoj Alam},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.75},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.75}
}

References

57 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lamiaa Abdel-Hamid. 2020. Egyptian arabic speech emotion recognition using prosodic, spectral and wavelet features. Speech Communication, 122:19– 30. ↗
  2. Mahmoud Mohamed Abdelsamie, Shahira Shaaban Azab, and Hesham A. Hefny. 2024. A comprehensive review on arabic offensive language and hate speech detection on social media: methods, challenges and solutions. Social Network Analysis and Mining, 14(1):111. ↗
  3. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. Arbert & marbert: Deep bidirectional transformers for arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088– 7105, Online. Association for Computational Linguistics. ↗
  4. Ibrahim Abu Farha and Walid Magdy. 2020. Multi-task learning for Arabic offensive language and hate-speech detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 86–90, Marseille, France. European Language Resource Association. find
  5. Firoj Alam, Md Rafiul Biswas, Uzair Shah, Wajdi Zaghouani, and Georgios Mikros. 2024a. Propaganda to hate: A multimodal analysis of arabic memes with multi-agent llms. In International Conference on Web Information Systems Engineering, pages 380– 390. Springer. Firoj Alam, Abul Hasnat, Fatema Ahmad, Md.
  6. Arid Hasan, and Maram Hasanain. 2024b. ArMeme: Propagandistic content in Arabic memes. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21071–21090, Miami, Florida, USA. Association for Computational Linguistics.
  7. Firoj Alam, Abul Hasnat, Fatema Ahmed, Md Arid Hasan, and Maram Hasanain. 2024c. ArMeme: Propagandistic content in arabic memes. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
  8. Walisa Alam, Mehreen Rahman, Shawly Ahsan, and Mohammed Moshiul Hoque. 2025. Cuet_zahra_duo@mahed 2025: Hate and hope speech detection in arabic social media content using transformer. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  9. Nouar AlDahoul and Yasir Zaki. 2025. Nyuad at mahed shared task: Detecting hope, hate, and emotion in arabic textual speech and multi-modal memes using large language models. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  10. R. H. Aljuhani, A. Alshutayri, and H. Alahdal. 2021. Arabic speech emotion recognition from saudi dialect corpus. IEEE Access, 9:127081–127085. ↗
  11. Maha Jarallah Althobaiti. 2023. An open-source dataset for arabic fine-grained emotion recognition of online content amid covid-19 pandemic. Data in Brief, 51:109745. find
  12. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  13. Muhammad Arif, Moein Shahiki Tash, Ainaz Jamshidi, Fida Ullah, Iqra Ameer, Jugal Kalita, Alexander Gelbukh, and Fazlourrahman Balouchzahi. 2024. Analyzing hope speech from psycholinguistic and emotional perspectives. Scientific Reports, 14(1):23548. ↗
  14. Nsrin Ashraf, Mariam Labib, Tarek Elshishtawy, and Hamada Nayel. 2025. Reglat at mahed shared task: A hybrid ensemble-based system for arabic hate speech detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  15. F. Balouchzahi and 1 others. 2025. Urduhope: Analysis of hope and hopelessness in urdu texts. Knowledge-Based Systems, 308:112746. find
  16. Nguyen Minh Bao and Dang Van Thin. 2025. Baoflowin502 at mahed shared task: Text-based hate and hope speech classification. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  17. Boaz M. Ben-David, Nandini Multani, Vered Shakuf, Frank Rudzicz, and Pascal H. H. van Lieshout. 2016. Prosody and semantics are separate but not separable channels in the perception of emotional speech: Test for rating of emotions in speech. Journal of Speech, Language, and Hearing Research, 59(1):72–89. ↗
  18. Kristin M. Brethel-Haurwitz, Maria Stoianova, and Abigail A. Marsh. 2020. Empathic emotion regulation in prosocial behaviour and altruism. Cognition and Emotion, 34(8):1532–1548. ↗
  19. Bharathi Raja Chakravarthi, Vigneshwaran Muralidaran, Ruba Priyadharshini, Subalalitha Chinnaudayar Navaneethakrishnan, John P. McCrae, Miguel Ángel García, Salud María Jiménez-Zafra, Rafael Valencia-García, Prasanna Kumar Kumaresan, Rahul Ponnusamy, Daniel García-Baena, and José Antonio García-Díaz. 2022. Overview of the shared task on hope speech detection for equality, diversity and inclusion. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pages 378–388, Dublin, Ireland. Association for Computational Linguistics. ↗
  20. Md Sagor Chowdhury and Adiba Fairooz Chowdhury. 2025. Quasar at mahed shared task : Decoding emotions and offense in arabic text using llm and transformer-based approaches. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  21. Joy Das, Alamgir Hossain, and Mohammed Moshiul Hoque. 2025. joy_2004114 at mahed shared task : Filtering hate speech from memes using a multimodal fusion-based approach. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  22. Riddhiman Swanan Debnath, Abdul Wadud Shakib, and Md Saiful Islam. 2025. Multiminds at mahed 2025: Multimodal and multitask approaches for detecting emotional, hate, and offensive speech in arabic content. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  23. Ratnajit Dhar and Arpita Mallik. 2025. Cuet-823 at mahed 2025 shared task: Large language model-based framework for emotion, offensive, and hate detection in arabic. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  24. Marc Djandji, Fady Baly, Wissam Antoun, and Hazem Hajj. 2020. Multi-task learning using AraBert for offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 97–101, Marseille, France. European Language Resource Association.
  25. Yasmine El Abed, Mariem Ben Arbia, Saoussen Ben Chaabene, and Omar Trigui. 2025. Anlp-uniso at mahed shared task: Detection of hate and hope speech in arabic social media based on xlm-roberta and logistic regre. In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics.
  26. Yassir El Attar. 2025. Yassirea at mahed 2025: Fusion-based multimodal models for arabic hate meme detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  27. Ahmed Elzainy, Hazem Abdelsalam, Ahmed Samir, and Mohamed Amin. 2025. Aaa at mahed shared task: A systematic encoder evaluation for arabic hope and hate speech classification. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics. find
  28. Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Berta Chulvi, Paolo Rosso, Alyssa Lees, and Jeffrey Sorensen. 2022. SemEval-2022 task 5: Multimedia automatic misogyny identification. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 533–549, Seattle, United States. Association for Computational Linguistics.
  29. Saad Hameed and Ala Al-Fuqaha. 2025. Kalam at mahed shared task 2025: Transformer-based approaches for arabic sentiment classification and stance detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics. Maram Hasanain, Md. Arid Hasan, Fatema Ahmed, Reem Suwaileh, Md. find
  30. Rafiul Biswas, Wajdi Zaghouani, and Firoj Alam. 2024. ArAIEval Shared Task: Propagandistic techniques detection in unimodal and multimodal arabic content. In Proceedings of the Second Arabic Natural Language Processing Conference, Bangkok. ACL.
  31. Enas A. Hakim Khalil and Wafaa S. El-Kassas. 2025. Aranlp at mahed 2025 shared task: Using arabert for text-based hate and hope speech classification. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  32. Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. The hateful memes challenge: detecting hate speech in multimodal memes. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. Curran Associates Inc. ↗
  33. Samira Klaylat, Zeina Osman, and Rached Zantout. 2018. Emotion recognition in arabic speech. Analog Integrated Circuits and Signal Processing, 96(2):185– 198. ↗
  34. Mohamed Meddeb, Rima Malka, and Mohamed Ali Hammami. 2017. Building and analysing emotion corpus of the arabic speech. In 2017 1st International Workshop on Arabic Script Analysis and Recognition (ASAR), pages 18–22. find
  35. Ali Hamid Meftah, Yousef Ajami Alotaibi, and Sid-Ahmed Selouani. 2018. Evaluation of an arabic speech corpus of emotions: A perceptual and statistical analysis. IEEE Access, 6:72845–72861. ↗
  36. Ali Hamid Meftah, Mohammad A. Qamhan, Yasser M. Seddiq, Yousef Ajami Alotaibi, and Sid-Ahmed Selouani. 2021. King saud university emotions corpus: Construction, analysis, evaluation, and comparison. IEEE Access, 9:54201–54219. ↗
  37. Leila Moudjari, Mélissa Hacene Cherkaski, and Farah Benamara. 2025. Descartes_hope at mahed shared task 2025: Integrating pragmatic features for arabic hope and hate speech detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics. find
  38. Hamdy Mubarak, Hend Al-Khalifa, and Abdulmohsen Al-Thubaity. 2022. Overview of OSACT5 shared task on Arabic offensive language and hate speech detection. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 162–166, Marseille, France. European Language Resources Association.
  39. Hamdy Mubarak, Kareem Darwish, Walid Magdy, Tamer Elsayed, and Hend Al-Khalifa. 2020. Overview of OSACT4 Arabic offensive language detection shared task. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 48–52, Marseille, France. European Language Resource Association.
  40. Thien Bao Nguyen and Van Thin Dang. 2025b. Loveheaven at mahed 2025: Text-based hate and hope speech classification using arabert-twitter ensemble. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  41. A. E. Obiadoh, O.J Abiola, T.D. Ogunleye, B.A. Tewodros, and T.O Abiola. 2025. Cic-nlp at mahed 2025 task 1:assessing the role of bigram augmentation in multiclass arabic hate and hope speech classification. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  42. S. S. Poorna and G. J. Nair. 2019. Multistage classification scheme to enhance speech emotion recognition. International Journal of Speech Technology, 22(2):327–340. Md. Abdur Rahman, Md. Sabbir Dewan, Md. Tofael Ahmed Bhuiyan, and Md. ↗
  43. Ashiqur Rahman. 2025. Smollab_seu at mahed shared task: Do arabic-native encoders surpass multilingual models in detecting the nuances of hope, hate, and emotion? In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  44. Ibad-ur-Rehman Rashid and Muhammad Hashir Khalil. 2025. Ayahverse at mahed shared task: Fine-tuning arabicbert with preprocessing for hope and hate detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  45. Itbaan Safwan. 2025. Thinking nodes at mahed: A comparative study of multimodal architectures for arabic hateful meme detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  46. Abdallah Saleh and Mariam Biltawi. 2025. Htu at mahed shared task: Ensemble-based classification of arabic hate and hope speech using pre-trained dialectal arabic models. In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics.
  47. L. Nasr, M. Hani, A. Harkous, Y. Al Khalil, A. Abou Daya, H. Hajj, and R. El-Khoury. 2024. Survey on arabic speech emotion recognition. International Journal of Speech Technology. Online first. ↗
  48. Octa Syakila Salsabila, Sri Melati Rahmayani, and Isti Yuniastuti. 2024. Content analysis of arabic vocabulary in al quran for the improvement of emotional intelligence. LiNGUA: Jurnal Ilmu Bahasa dan Sastra, 19(1):97–108. ↗
  49. Minh Triet Nguyen and Van Thin Dang. 2025a. Nguyentriet at mahed shared task: Ensemble of arabic bert models with hierarchical prediction and soft voting for text-based hope and hate detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  50. Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, and Paul Buitelaar. 2020. Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text. In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, pages 32–41, Marseille, France. European Language Resources Association (ELRA).
  51. Tran Tran, Trinh and Thin Dang, Van. 2025. Trantranuit at mahed shared task: Multilingual transformer ensemble with advanced data augmentation and optuna-based hyperparameter optimization. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  52. Gerben A. van Kleef and Gert-Jan Lelieveld. 2022. Moving the self and others to do good: The emotional underpinnings of prosocial behavior. Current Opinion in Psychology, 44:80–88. Epub 2021-08-31. ↗
  53. Al-Habashi Yasser, Sibaee1 Serry, Nacar Omer, Ammar Adel, and Wadii Boulila1. 2025. Anlpers at mahed shared task: From hate to hope: Boosting arabic text classification. In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics. find
  54. Wajdi Zaghouani and Md Rafiul Biswas. 2025a. An annotated corpus of arabic tweets for hate speech analysis. arXiv preprint arXiv:2505.11969. ↗
  55. Wajdi Zaghouani and Md Rafiul Biswas. 2025b. Emohopespeech: An annotated dataset of emotions and hope speech in english and arabic. arXiv preprint arXiv:2505.11959. Wajdi Zaghouani, Hamdy Mubarak, and Md. ↗
  56. Rafiul Biswas. 2024. So hateful! building a multi-label hate speech annotated Arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055, Torino, Italia. ELRA and ICCL.
  57. Mohamed Zaytoon, Ahmed Salem, Ahmed Sakr, and Hossam Elkordi. 2025. Araminds at mahed 2025: Leveraging vision-language models and contrastive multi-task learning for multimodal hate speech detection. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2025-mahed-shared-task-multimodal-detection/