Elamur Research⌕
☰
home / papers / 2025 / CUET_Zahra_Duo@Mahed 2025: Hate and Hope Speech Detection in Arabic Social Media Content using Transformer

CUET_Zahra_Duo@Mahed 2025: Hate and Hope Speech Detection in Arabic Social Media Content using Transformer

Paper · 2025 · Walisa Alam, Mehreen Rahman, Shawly Ahsan et al.

Summary

This paper presents a system built for subtask 1 of MAHED 2025, a shared task organised as part of ArabicNLP 2025. The task is three-way classification of Arabic social media posts as hate, hope, or not_applicable. The provided data has about 6.9k training, 1.5k validation and 1.5k test texts, and not_applicable dominates. The authors clean the tweets by stripping noise, removing Tashkeel and normalising informal writing. They enlarge the training set by swapping selected words for substitutes predicted by an Arabic BERT model through nlpaug. They then compare a broad set of systems: TF-IDF with classical learners, Keras-based neural nets (MLP, CNN, BiLSTM, CNN+BiLSTM), and fine-tuned MARBERT, AraBERT-base and AraBERT-large. AraBERT-large was clearly the strongest, with macro-F1 around 0.69–0.70, which placed 8th on the leaderboard. Error analysis shows that hate and hope are rarely mistaken for each other. Most mistakes involve the neutral class, often triggered by sarcasm, offensive vocabulary or dialectal words. The work is a useful reference point for how much model scale and tuning contribute on small, imbalanced Arabic data.

Key points

  • Data split: train 6,890 (1,301 hate / 1,892 hope / 3,697 N/A), validation 1,476, test 1,477. Average text length is roughly 21–23 words.
  • Best macro-F1 per family: Naive Bayes 0.563 (classical), CNN+BiLSTM 0.619 (neural), MARBERT 0.640, AraBERT-base 0.600 and AraBERT-large 0.695. The official leaderboard score was 0.698, ranked 8th.
  • Ablation results for AraBERT-large:
    • On test data, raw text gave 0.67, while preprocessing with learning rate 3.5×10⁻⁶ and max length 256 gave 0.69.
    • On development data, preprocessing plus augmentation reached 0.64, compared with 0.61 for raw text.
  • Confusion matrix on test: only 5 hate→hope and 4 hope→hate errors. In contrast, 89 hate and 138 hope posts fell into not_applicable, and 80 and 127 neutral posts were pulled into hate and hope respectively.
  • Augmentation used contextual word substitution with Arabic BERT (Antoun et al., 2020) via the nlpaug library. Training ran on two Kaggle T4 GPUs.

Interesting ideas

  • I1. Break down confusions by class pair rather than reporting only an aggregate score; here this showed that the two polar classes are well separated and the real difficulty lies at the border with the neutral class.
  • I2. Augment the data with a masked language model that replaces chosen words by in-context substitutes, which keeps the label while varying the wording.
  • I3. The qualitative errors fall into recognisable mechanisms: sarcasm hides hostility, harsh words falsely signal hate, and an upbeat tone falsely signals hope.
  • I4. Run a full ladder of baselines on the same split, from TF-IDF bag-of-words up to large transformers, to measure how much each level of modelling adds.
  • I5. Dialectal and colloquial words appear in the failure cases as a distinct error source.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation reports could list confusions between each label and a "none/other" bucket separately, so we can tell whether a component misreads meaning or simply misses or over-fires.
  • I2 (learn): Same-POS substitutions could generate variant sentences for silver data, and the witness parsers would then check whether the tree stayed the same, giving a cheap robustness test for parsers.
  • I3 (frames): The irony and speech-act modules in prag should test against these exact traps, so that the intent read from a frame does not collapse into word polarity.
  • I4 (learn): A bag-of-words classifier could serve as a standing negative control, so that every task where Mova's structured pipeline claims success must beat it.
  • I5 (tokens): The tagger and lemmatizer could flag out-of-lexicon slang or dialect forms explicitly, so that later stages know their input is uncertain instead of trusting a guessed analysis.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh25 works · 29 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Walisa Alam, Mehreen Rahman, Shawly Ahsan and Mohammed Moshiul Hoque. 2025. CUET_Zahra_Duo@Mahed 2025: Hate and Hope Speech Detection in Arabic Social Media Content using Transformer. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.96

@inproceedings{alam2025cuet,
  title = {{CUET\_Zahra\_Duo@Mahed 2025: Hate and Hope Speech Detection in Arabic Social Media Content using Transformer}},
  author = {Walisa Alam, Mehreen Rahman, Shawly Ahsan and Mohammed Moshiul Hoque},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.96},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.96}
}

References

16 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Nouar AlDahoul and Yasir Zaki. 2025. Detecting hope, hate, and emotion in arabic textual speech and multimodal memes using large language models. Preprint, arXiv:2508.15810. ↗
  3. Seham Alghamdi, Youcef Benkhedda, Basma Alharbi, and Riza Batista-Navarro. 2024. AraTar: A corpus to support the fine-grained detection of hate speech targets in the Arabic language. In Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024, pages 1–12, Torino, Italia. ELRA and ICCL.
  4. Utku Yagci, Egemen Iscan, and Ahmet Kolcak. 2024. ReBERT at HSD-2Lang 2024: Fine-tuning BERT with AdamW for hate speech detection in Arabic and Turkish. In Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024), pages 195–198, St. Julians, Malta. Association for Computational Linguistics.
  5. Yongjin Yang, Joonkee Kim, Yujin Kim, Namgyu Ho, James Thorne, and Se young Yun. 2023. Hare: Explainable hate speech detection with step-by-step reasoning. Preprint, arXiv:2311.00321. ↗
  6. Wajdi Zaghouani and Md Rafiul Biswas. 2025a. An annotated corpus of arabic tweets for hate speech analysis. arXiv preprint arXiv:2505.11969. ↗
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. ↗
  8. Anis Charfi, Mabrouka Bessghaier, Raghda Akasheh, Andria Atalla, and Wajdi Zaghouani. 2024. Hate speech detection with adhar: a multi-dialectal hate speech corpus in arabic. Frontiers in Artificial Intelligence, 7. ↗
  9. Ahmed Elmadany, Wajdi Zaghouani, and Nizar Habash. 2024. A survey on arabic natural language processing: Challenges and applications. ACM Computing Surveys, 57(2):1–40. find
  10. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2019. Spread of hate speech in online social media. In Proceedings of the 10th ACM Conference on Web Science (WebSci ’19), pages 173–182, Boston, MA, USA. ACM. find
  11. Wajdi Zaghouani and Md Rafiul Biswas. 2025b. Emohopespeech: An annotated dataset of emotions and hope speech in english and arabic. arXiv preprint arXiv:2505.11959. ↗
  12. Wajdi Zaghouani, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgios Mikros, Abul Hasnat, and Firoj Alam. 2025. MAHED shared task: Multimodal detection of hope and hate emotions in arabic content. In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics.
  13. Wajdi Zaghouani, Hamdy Mubarak, and Md Rafiul Biswas. 2024. So hateful! building a multi-label hate speech annotated arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055.
  14. Pradeep Roy, Snehaan Bhawal, Abhinav Kumar, and Bharathi Raja Chakravarthi. 2022. IIITSurat@LT-EDI-ACL2022: Hope speech detection using machine learning. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pages 120–126, Dublin, Ireland. Association for Computational Linguistics. find
  15. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020. KUISAIL at SemEval-2020 task 12: BERT-CNN for offensive speech identification in social media. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 2054–2059, Barcelona (online). International Committee for Computational Linguistics.
  16. Muhammad Usman, Muhammad Ahmad, M. Shahiki Tash, Irina Gelbukh, Rolando Quintero Tellez, and Grigori Sidorov. 2025. Multilingual hate speech detection in social media using translation-based approaches with large language models. Preprint, arXiv:2506.08147. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alam-2025-cuet-zahra-duo-mahed-2025/