Elamur Research⌕
☰
home / papers / 2025 / NguyenTriet at MAHED Shared Task: Ensemble of Arabic BERT Models with Hierarchical Prediction and Soft Voting for Text-Based Hope and Hate Detection

NguyenTriet at MAHED Shared Task: Ensemble of Arabic BERT Models with Hierarchical Prediction and Soft Voting for Text-Based Hope and Hate Detection

Paper · 2025 · Nguyen Minh Triet and Thìn Đặng Văn

Summary

The paper describes a system for the MAHED 2025 shared task on detecting hope and hate in Arabic social media content. The authors took part in two text-only subtasks. Subtask 1 is a three-way label (hate, hope, not_applicable). Subtask 2 is a layered problem: assign one of 12 emotions, decide whether a post is offensive, and then decide whether offensive posts are hateful.

Each classifier pairs two Arabic tweet-pretrained encoders, MARBERTv2 and AraBERTv0.2-Twitter, and merges their outputs by summing class probabilities (soft voting). For Subtask 2 the classifiers form a cascade. The emotion label is added to the input of the offensiveness model, and the hate model runs only on posts judged offensive. During training, gold emotions supply that context.

Preprocessing turns emojis into Arabic words, normalizes the script and removes noise. A class-weighted loss addresses the heavy label skew.

The system placed 3rd on Subtask 1 (macro-F1 0.707) and 2nd on Subtask 2 (0.553). In ablations, the cascade beat flat multiclass setups by about 4–5 points.

Key points

  • Data: Subtask 1 has 9,843 instances (not_applicable 53.36%, hope 27.65%, hate 18.97%). Subtask 2 has 8,515 instances, where anger covers 25.94% and fear only 0.91%; 70.79% are non-offensive, and 82.40% of offensive posts are not_hate.
  • Models: MARBERTv2 (MLM on about 1B Arabic tweets) and AraBERTv0.2-Twitter (about 60M tweets). Fine-tuning used lr 3e-5, weight decay 0.1, 2 epochs, AdamW with a cosine schedule and class-weighted CrossEntropyLoss, on a single Kaggle P100.
  • Subtask 1 test macro-F1: MARBERTv2 0.692, AraBERTv0.2-Twitter 0.698, ensemble 0.707 (3rd, behind HTU 0.723 and NYUAD 0.721).
  • Subtask 2 test macro-F1:
    • Flat multiclass setups scored 0.483, 0.490 and 0.510 (ensemble).
    • Hierarchical setups scored 0.538, 0.547 and 0.553 (ensemble; 2nd, behind NYUAD at 0.578).
  • Preprocessing tools: the emoji library, pyarabic for removing diacritics and tatweel, camel_tools for normalizing alef maksura and teh marbuta, and NLTK Arabic stopwords.

Interesting ideas

  • I1. An upstream label (emotion) is attached to the input of downstream classifiers, and gold upstream labels are used while training; the paper reports this cascade outperforms a flat label space.
  • I2. The second-level decision (hate) is only made for items that passed the first gate (offensive), so each model faces a narrower, better-defined question.
  • I3. Two encoders tokenize and predict independently, then their probability vectors are added and the top class is taken, which consistently gained 1–2 points over either model alone.
  • I4. The ablation design reruns identical backbones as flat and as cascaded models, so the gain from structure is separated from the gain from ensembling.
  • I5. Emojis are rewritten as their textual descriptions before tokenization, so models see words rather than opaque symbols.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): speech-act or irony classifiers in prag could take an explicit upstream signal (e.g. sentiment or frame type) as a readable extra input, keeping the dependency between stages visible and explainable.
  • I2 (frames): irony or implicature detection could run only on utterances already flagged as non-literal, which shrinks the search and gives a clear audit trail of why a reading was chosen.
  • I3 (learn): when several witness parsers agree on silver data, summing their arc and label distributions instead of hard-voting could produce graded confidence for selecting training trees.
  • I4 (learn): any new pipeline stage could be evaluated against the same components wired without it, a clean control for claims that the structure itself helps.
  • I5 (tokens): the en tokenizer could map emojis to lexical descriptions, letting the tagger and lemmatizer assign them POS and meaning that later stages can reason about.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Nguyen Minh Triet and Thìn Đặng Văn. 2025. NguyenTriet at MAHED Shared Task: Ensemble of Arabic BERT Models with Hierarchical Prediction and Soft Voting for Text-Based Hope and Hate Detection. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.77

@inproceedings{triet2025nguyentriet,
  title = {{NguyenTriet at MAHED Shared Task: Ensemble of Arabic BERT Models with Hierarchical Prediction and Soft Voting for Text-Based Hope and Hate Detection}},
  author = {Nguyen Minh Triet and Thìn DJặng Văn},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.77},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.77}
}

References

13 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Firoj Alam, Md Rafiul Biswas, Uzair Shah, Wajdi Zaghouani, and Georgios Mikros. 2024a. Propaganda to hate: A multimodal analysis of arabic memes with multi-agent llms. In International Conference on Web Information Systems Engineering, pages 380– 390. Springer. Firoj Alam, Abul Hasnat, Fatema Ahmad, Md.
  3. Arid Hasan, and Maram Hasanain. 2024b. ArMeme: Propagandistic content in Arabic memes. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21071–21090, Miami, Florida, USA. Association for Computational Linguistics.
  4. Wajdi Zaghouani, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgios Mikros, Abul Hasnat, and Firoj Alam. 2025. MAHED shared task: Multimodal detection of hope and hate emotions in arabic content. In Proceedings of the Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics. Wajdi Zaghouani, Hamdy Mubarak, and Md.
  5. Rafiul Biswas. 2024a. So hateful! building a multi-label hate speech annotated Arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055, Torino, Italia. ELRA and ICCL.
  6. Wajdi Zaghouani, Hamdy Mubarak, and Md Rafiul Biswas. 2024b. So hateful! building a multi-label hate speech annotated arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055.
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. Md. ↗
  8. Rafiul Biswas and Wajdi Zaghouani. 2025a. An annotated corpus of arabic tweets for hate speech analysis. CoRR, abs/2505.11969. Md. ↗
  9. Rafiul Biswas and Wajdi Zaghouani. 2025b. Emohopespeech: An annotated dataset of emotions and hope speech in english and arabic. CoRR, abs/2505.11959. ↗
  10. Maram Hasanain, Fatema Ahmad, and Firoj Alam. 2024a. Large language models for propaganda span annotation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14522– 14532, Miami, Florida, USA. Association for Computational Linguistics. find
  11. Maram Hasanain, Fatema Ahmed, and Firoj Alam. 2024b. Can gpt-4 identify propaganda? annotation and detection of propaganda spans in news articles. In Proceedings of the 2024 Joint International Conference On Computational Linguistics, Language Resources And Evaluation, LREC-COLING 2024, Torino, Italy. find
  12. Wajdi Zaghouani and Md Rafiul Biswas. 2025a. An annotated corpus of arabic tweets for hate speech analysis. arXiv preprint arXiv:2505.11969. ↗
  13. Wajdi Zaghouani and Md Rafiul Biswas. 2025b. Emohopespeech: An annotated dataset of emotions and hope speech in english and arabic. arXiv preprint arXiv:2505.11959. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/triet-2025-nguyentriet-mahed-shared-task-ensemble/