Elamur Research⌕
☰
home / papers / 2025 / CUET-823 at MAHED 2025 Shared Task: Large Language Model-Based Framework for Emotion, Offensive, and Hate Detection in Arabic

CUET-823 at MAHED 2025 Shared Task: Large Language Model-Based Framework for Emotion, Offensive, and Hate Detection in Arabic

Paper · 2025 · Ratnajit Dhar and Arpita Mallik

Summary

The paper describes the CUET-823 entry to Subtask-2 of the MAHED 2025 Shared Task. The subtask asks for three labels on short Arabic social media posts, mostly tweets: an emotion from 12 classes, an offensive yes/no decision, and a hate decision that applies only to offensive posts. The authors split the problem into two separately fine-tuned stages, both built on Meta-Llama-3.1-8B. The first stage predicts the emotion. The second gives a two-part answer: offensive or not, then hate, not_hate, or not_applicable. Training used 4-bit quantization and LoRA through Unsloth, with only the official MAHED 2025 data and no class balancing. On the hidden test set the system scored a macro F1 of 0.518, ranking 4th, and had the best precision of all teams. On the validation set, fine-tuning beat zero-shot and few-shot prompting, and LLaMA-3.1-8B came out ahead of several other small open models. The work shows that a modest, cheaply adapted LLM can handle hierarchical, imbalanced, dialect-rich classification with little data.

Key points

  • Data is the MAHED 2025 dataset. Of the 5,960 training posts, 1,744 are offensive and 303 are hate; the 1,277 validation posts include 363 that are offensive and 68 that are hate; a further 1,278 posts without labels form the test split. The emotion classes are very skewed, from anger at 1,551 down to fear at 53.
  • Setup: Meta-Llama-3.1-8B, 4-bit (BitsAndBytes), LoRA rank 16, AdamW at lr 2e-4, 1000 steps for the emotion model and 500 for the offensive/hate model, and a 2048-token cap. Tools were Unsloth, Hugging Face Transformers, and TRL. Decoding used temperature 0.
  • Test results: macro F1 0.518 (4th place) and precision 0.617, the top precision among participants.
  • On validation, fine-tuned LLaMA-3.1-8B reached 0.554 macro F1, against 0.484 zero-shot and 0.477 few-shot. Mistral 7B v0.3, Qwen-2 7B, CodeGemma 7B, Zephyr 7B and Gemma 3 4B all scored lower.
  • The paper claims conditional prompting beats flat multi-label classification, but the text reports no flat baseline numbers.
  • The main errors were offensive posts not escalated to hate when the hate was implicit or culturally coded, and polite thanks labeled neutral instead of love.

Interesting ideas

  • I1. The output has a fixed hierarchical slot format in which the dependent label is written as an explicit not_applicable when its parent is negative, so every answer has the same shape and stays easy to parse.
  • I2. Each stage gets its own prompt and its own adapter rather than one model that predicts everything at once, which lets each stage focus on different cues.
  • I3. The full closed label inventory goes into the prompt, and decoding is deterministic at temperature 0, which makes runs reproducible and keeps answers inside the label set.
  • I4. The same model family is compared in zero-shot, few-shot and fine-tuned modes on one dev set, which separates what adaptation contributes from what the base model contributes.
  • I5. The error analysis separates surface politeness from real affection and explicit hate from implied hate, two places where literal meaning and intended meaning come apart.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): LLM teacher tiers could return gated slot answers, such as "is there a speech act → which one", with explicit not_applicable values, so silver labels never contain contradictory parent and child labels.
  • I2 (frames): in crate prag, irony or offense judgments could run as a separate conditional pass only after the speech-act layer has fired, so that each decision stays isolated and explainable.
  • I3 (learn): teacher queries for VerbNet roles or frame labels could list the permitted inventory and use zero temperature, so silver data can be regenerated and checked against witness parsers.
  • I4 (learn): evaluation of LLM teacher tiers could use this three-way grid to decide whether a tier is worth fine-tuning or works well enough prompted.
  • I5 (frames): these make good test cases for the implicature and speech-act modules, for example a thanking act that should not by itself imply an affection state in the world of the text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh25 works · 29 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Ratnajit Dhar and Arpita Mallik. 2025. CUET-823 at MAHED 2025 Shared Task: Large Language Model-Based Framework for Emotion, Offensive, and Hate Detection in Arabic. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.84

@inproceedings{dhar2025cuet,
  title = {{CUET-823 at MAHED 2025 Shared Task: Large Language Model-Based Framework for Emotion, Offensive, and Hate Detection in Arabic}},
  author = {Ratnajit Dhar and Arpita Mallik},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.84},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.84}
}

References

19 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amr Al-Khatib and Samhaa R El-Beltagy. 2017. Emotional tone detection in arabic tweets. In International Conference on Computational Linguistics and Intelligent Text Processing, pages 105–114. Springer. find
  2. Ali Al-Laith and Mamdouh Alenezi. 2021. Monitoring peoples emotions and symptoms from arabic tweets during the covid-19 pandemic. Information, 12(2):86. ↗
  3. Sallam AL-Sarayreh, Azza Mohamed, and Khaled Shaalan. 2023. Challenges and solutions for arabic natural language processing in social media. In International conference on Variability of the Sun and sun-like stars: from asteroseismology to space weather, pages 293–302. Springer. find
  4. Zakaria H Ali and Hiba J Aleqabie. 2024. Emotion detection in arabic text in social media: A brief survey. Al-Furat Journal of Innovations in Electronics and Computer Engineering, 3(2):412–421. find
  5. Ghadah Alqahtani and Abdulrahman Alothaim. 2022. Emotion analysis of arabic tweets: Language models and available resources. Frontiers in Artificial Intelligence, 5:843038. ↗
  6. Center for Democracy and Technology. 2023. Moderating maghrebi arabic content on social media. https://cdt.org/insights/moderat ing-maghrebi-arabic-content-on-socia l-media/. Accessed: 2025-08-03. ↗
  7. Dieter Galea, Paolo Inglese, Lidia Cammack, Nicole Strittmatter, Monica Rebec, Reza Mirnezami, Ivan Laponogov, James Kinross, Jeremy Nicholson, Zoltan Takats, and 1 others. 2017. Translational utility of a hierarchical classification strategy in biomolecular data analytics. Scientific Reports, 7(1):14981. find
  8. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. ↗
  9. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  10. Khouloud Mnassri, Praboda Rajapaksha, Reza Farahbakhsh, and Noel Crespi. 2023. Hate speech and offensive language detection using an emotion-aware shared encoder. In ICC 2023-IEEE International Conference on Communications, pages 2852–2857. IEEE. find
  11. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. Arabic offensive language on twitter: Analysis and experiments. arXiv preprint arXiv:2004.02192. ↗
  12. Abdelrahim Qaddoumi. 2022. Arabic sentiment ensemble nadi shared task 2. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP 2022). Association for Computational Linguistics. find
  13. Omneya Rabie and Christian Sturm. 2014. Feel the heat: Emotion detection in arabic social media content. In The International Conference on Data Mining, Internet Computing, and Big Data (BigData2014), pages 37–49. Kuala Lumpur Citeseer. find
  14. Statista. United arab emirates: Number of internet users from 2010 to 2025. https://www.statista.com/statistics/1389944/ua e-number-of-internet-users/. Accessed: 2025-08-03. ↗
  15. Youpeng Yang, Qiuhong Zeng, Gaotong Liu, Shiyao Zheng, Tianyang Luo, Yibin Guo, Jia Tang, and Yi Huang. 2023. Hierarchical classification-based pan-cancer methylation analysis to classify primary cancer. BMC bioinformatics, 24(1):465. ↗
  16. Wajdi Zaghouani and Md Rafiul Biswas. 2025a. An annotated corpus of arabic tweets for hate speech analysis. arXiv preprint arXiv:2505.11969. ↗
  17. Wajdi Zaghouani and Md Rafiul Biswas. 2025b. Emohopespeech: An annotated dataset of emotions and hope speech in english and arabic. arXiv preprint arXiv:2505.11959. ↗
  18. Wajdi Zaghouani, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgios Mikros, Abul Hasnat, and Firoj Alam. 2025. MAHED shared task: Multimodal detection of hope and hate emotions in arabic content. In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
  19. Wajdi Zaghouani, Hamdy Mubarak, and Md Rafiul Biswas. 2024. So hateful! building a multi-label hate speech annotated arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dhar-2025-cuet-823-mahed-2025-shared/