Elamur Research⌕
☰
home / papers / 2025 / Thinking Nodes at MAHED: A Comparative Study of Multimodal Architectures for Arabic Hateful Meme Detection

Thinking Nodes at MAHED: A Comparative Study of Multimodal Architectures for Arabic Hateful Meme Detection

Paper · 2025 · Itbaan Safwan

Summary

This system paper, written for Task 3 (MAHED) of the Arabic NLP 2025 shared tasks, addresses binary classification of Arabic memes as hateful or not hateful. Hate in memes often comes from how the picture and the caption interact, so the author compares three ways of fusing the two modalities. All three use the same frozen-mostly backbones: CLIP (openai/clip-vit-base-patch32) for images and MARBERT for Arabic text. The variants are bidirectional cross-attention, a stack of 1D convolutions that merges the two embedding sequences step by step, and a two-stage pipeline. That pipeline first aligns modalities contrastively in CLIP style, then trains a small gated classifier. The labels are heavily skewed toward non-hate. To cope, the author pools all labelled splits, re-splits them in a stratified way, and trains with a class-weighted Focal Loss. After retraining on all data and scoring on the released gold test labels, the CNN fusion model did best, with macro F1 0.779. The work gives a clear baseline comparison of fusion strategies for an under-studied language and modality pair.

Key points

  • Backbones are openai/clip-vit-base-patch32 and UBC-NLP/MARBERT, and only the last two layers of each are unfrozen in the end-to-end models.
  • The provided train, dev and earlier-phase labelled test data were merged and re-split 70/30 with stratification, because the original dev set was badly imbalanced.
  • The loss is Focal Loss with γ = 2, and α is set from inverse class frequency, w_c = N / (2·N_c).
  • Training used AdamW with lr 2e-5 for the end-to-end models and 5e-5 for the light classifier, batch size 32, ReduceLROnPlateau and early stopping.
  • On the official test set, models trained on the 70% split scored about 0.72 macro F1 each. Retrained on all data, Cross-Attention and Custom Embedding reached 0.765 and CNN Fusion reached 0.779 (+6.1 points).

Interesting ideas

  • I1. When the official dev split has a skewed label mix, pooling all labelled data and re-splitting it in a stratified way gives steadier validation signals for model selection.
  • I2. Using inverse-frequency class weights together with a focusing term moves the training effort toward rare, hard examples instead of easy majority ones.
  • I3. Separating alignment from decision lets a cheap gated classifier work on precomputed embeddings, and the gate shows how much each input source contributed to a prediction.
  • I4. Reporting both the as-submitted results and a retrain on all data separates the effect of architecture from the effect of data volume, and here the data effect was the larger one.
  • I5. The proposed future step is a large VLM teacher that produces soft probability labels for a small, efficient student model.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova's held-out evaluation sets could be re-stratified by construction type or rare relation, so that negative-control comparisons are not distorted by an unlucky split.
  • I2 (tree): The perceptron parser and gparse could weight updates for rare deprels such as vocative, reparandum or dislocated, which a frequency-dominated loss tends to neglect.
  • I3 (frames): Irony or speech-act decisions in prag could combine lexical, contextual and world-state cues through an explicit gate, and log the gate weights as part of the explanation.
  • I4 (learn): Every new silver-data batch could be evaluated with a fixed-data control run, so that gains are credited correctly to more data or to better rules.
  • I5 (learn): LLM teacher tiers could hand over graded confidence distributions instead of hard labels, so the student parsers and readers also learn from how uncertain the teacher is.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Institute of Business Administration Karachi.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksVision & language · 657 works

Cite

Cite as Itbaan Safwan. 2025. Thinking Nodes at MAHED: A Comparative Study of Multimodal Architectures for Arabic Hateful Meme Detection. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.98

@inproceedings{safwan2025thinking,
  title = {{Thinking Nodes at MAHED: A Comparative Study of Multimodal Architectures for Arabic Hateful Meme Detection}},
  author = {Itbaan Safwan},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.98},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.98}
}

References

14 references, 13 of them with a card of ours. ↗ leads to the work.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Firoj Alam, Md Rafiul Biswas, Uzair Shah, Wajdi Zaghouani, and Georgios Mikros. 2024a. Propaganda to hate: A multimodal analysis of arabic memes with multi-agent llms. In International Conference on Web Information Systems Engineering, pages 380– 390. Springer. Firoj Alam, Abul Hasnat, Fatema Ahmad, Md.
  3. Arid Hasan, and Maram Hasanain. 2024b. ArMeme: Propagandistic content in Arabic memes. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21071–21090, Miami, Florida, USA. Association for Computational Linguistics.
  4. Hao Tan and Mohit Bansal. 2019. Lxmert: Learning cross-modality encoder representations from transformers. Preprint, arXiv:1908.07490. ↗
  5. Amir Zadeh, Minghai Chen, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2017. Tensor fusion network for multimodal sentiment analysis. Preprint, arXiv:1707.07250. ↗
  6. Wajdi Zaghouani, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgio Mikros, Abul Hasnat, and Firoj Alam. 2025. MAHED shared task: Multimodal detection of hope and hate emotions in arabic content. In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics. Wajdi Zaghouani, Hamdy Mubarak, and Md.
  7. Rafiul Biswas. 2024. So hateful! building a multi-label hate speech annotated Arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055, Torino, Italia. ELRA and ICCL.
  8. John Arevalo, Thamar Solorio, Manuel Montes y Gómez, and Fabio A. González. 2017. Gated multimodal units for information fusion. Preprint, arXiv:1702.01992. ↗
  9. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. Preprint, arXiv:1503.02531. ↗
  10. Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2021. The hateful memes challenge: Detecting hate speech in multimodal memes. Preprint, arXiv:2005.04790. ↗
  11. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. Preprint, arXiv:2201.12086. ↗
  12. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2018. Focal loss for dense object detection. Preprint, arXiv:1708.02002. ↗
  13. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. Preprint, arXiv:2103.00020. Uzair Shah, Md. ↗
  14. Rafiul Biswas, Marco Agus, Mowafa Househ, and Wajdi Zaghouani. 2024. MemeMind at ArAIEval shared task: Generative augmentation and feature fusion for multimodal propaganda detection in Arabic memes through advanced language and vision models. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 467–472, Bangkok, Thailand. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/safwan-2025-thinking-nodes-mahed-comparative-study/