Elamur Research⌕
☰
home / papers / 2025 / CIC-NLP at MAHED 2025 TASK 1:Assessing the Role of Bigram Augmentation in Multiclass Arabic Hate and Hope Speech Classification

CIC-NLP at MAHED 2025 TASK 1:Assessing the Role of Bigram Augmentation in Multiclass Arabic Hate and Hope Speech Classification

Paper · 2025 · Tolulope Olalekan Abiola, Oluwatobi Joseph Abiola, Ogunleye Temitope Dasola et al.

Summary

The paper describes a CIC-NLP submission to MAHED 2025 Task 1, a three-way classification of Arabic social media posts as hate, hope or not_applicable. The authors ask whether adding frequent word bigrams to the input text helps a transformer classifier in this imbalanced, dialect-heavy setting. Their pipeline normalises text with the ArabertPreprocessor (AraElectra profile), keeps emojis and fine-tunes MARBERT. They compare three setups: no augmentation, global augmentation (the same corpus-wide top bigrams added to every post) and classwise augmentation (each class's top bigrams added according to its label). At test time, labels are unknown, so class bigrams are chosen by an overlap heuristic. The plain baseline generalised best, reaching 68.25% accuracy and macro-F1 0.6729 on test. Global bigrams brought no gain. Classwise bigrams looked excellent on validation but collapsed on test. The work is a useful negative result: label-dependent augmentation can inflate validation scores by memorising class tokens, so validation alone cannot be trusted to judge it.

Key points

  • Data: the MAHED dataset with the organisers' train/val/test split. Validation support is 238 / 359 / 729 for classes 0 / 1 / 2, so class 2 dominates.
  • Model and training: UBC-NLP/MARBERT trained with AdamW, batch size 16 and maximum length 128. The baseline ran for 10 epochs and the augmented variants for 4. Checkpoints were chosen by validation macro-F1.
  • Baseline results: macro-F1 0.63 on validation. On test, accuracy was 0.6825 and macro-F1 0.6729.
  • Global bigrams: accuracy about 63% and macro-F1 about 0.62. Minority-class recall rose slightly while majority-class precision fell.
  • Classwise bigrams: validation accuracy 93% and macro-F1 0.92. On test, accuracy dropped to 59.65% and macro-F1 to 0.4726, with recall at only 0.466 against precision of 0.68.

Interesting ideas

  • I1. If extra input features are built from gold labels during training but from a heuristic at test time, a near-perfect validation score signals leakage rather than skill. A large gap between validation and test is the cheapest alarm for this.
  • I2. Explicit lexical cues, such as frequent n-grams, can be appended to the text fed to a contextual encoder to make a hybrid lexical–contextual model. The trick is cheap, but it only helps if the cues are equally available at inference.
  • I3. Normalisation that keeps emojis instead of stripping them preserves cues about emotion and stance that matter for evaluative text.
  • I4. Per-class precision and recall show what an aggregate score hides. Here augmentation traded majority-class precision for minority-class recall while macro-F1 barely moved.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): add a negative control that rebuilds every training-time feature through the exact inference path and flags any component whose score drops sharply when gold-derived hints are removed.
  • I2 (tree): gparse could take explicit hint tokens (for example, lexicon-derived tags) as extra input, as long as the same deterministic source produces them in both training and parsing.
  • I3 (tokens): the tokenizer could keep emojis as their own tagged tokens so later stages can still see them.
  • I4 (learn): evaluation reports could break scores down per label (dependency relation, frame role, speech-act type), so a change that just moves errors between classes is visible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Tolulope Olalekan Abiola, Oluwatobi Joseph Abiola, Ogunleye Temitope Dasola, Tewodros Achamaleh and Obiadoh Augustine Ekenedilichukwu. 2025. CIC-NLP at MAHED 2025 TASK 1:Assessing the Role of Bigram Augmentation in Multiclass Arabic Hate and Hope Speech Classification. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.80

@inproceedings{abiola2025cic,
  title = {{CIC-NLP at MAHED 2025 TASK 1:Assessing the Role of Bigram Augmentation in Multiclass Arabic Hate and Hope Speech Classification}},
  author = {Tolulope Olalekan Abiola, Oluwatobi Joseph Abiola, Ogunleye Temitope Dasola, Tewodros Achamaleh and Obiadoh Augustine Ekenedilichukwu},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.80},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.80}
}

References

13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Other factors: Token truncation and domain shift likely reduced augmentation benefits. find
  2. Tolulope O. Abiola, Tewodros A. Bizuneh, Oluwatobi J. Abiola, Temitope O. Oladepo, Olumide E. Ojo, Adebanji O. O., Grigori Sidorov, and Olga Kolesnikova. 2025b. Cic-nlp at genai detection task 1: Leveraging distilbert for detecting machine-generated text in english. In Proceedings of the 31st International Conference on Computational Linguistics (COLING), Abu Dhabi, UAE. Association for Computational Linguistics.
  3. Tolulope O. Abiola, Tewodros A. Bizuneh, Fatima Uroosa, Nida Hafeez, Grigori Sidorov, Olga Kolesnikova, and Olumide E. Ojo. 2025c. Cic-nlp at genai detection task 1: Advancing multilingual machine-generated text detection. In Proceedings of the 31st International Conference on Computational Linguistics (COLING), Abu Dhabi, UAE. Association for Computational Linguistics. find
  4. Tewodros Achamaleh, Tolulope Olalekan Abiola, Lem-lem Eyob Kawo, Mikiyas Mebraihtu, and Grigori Sidorov. 2025. CIC-NLP@DravidianLangTech 2025: Detecting AI-generated product reviews in Dravidian languages. In Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages, pages 502–507, Acoma, The Albuquerque Convention Center, Albuquerque, New Mexico. Association for Computational Linguistics. find
  5. Olaronke Oluwayemisi Adebanji, Irina Gelbukh, Hiram Calvo, and Olumide Ebenezer Ojo. 2022. Sequential models for sentiment analysis: A comparative study. In Advances in Computational Intelligence, 21st Mexican International Conference on Artificial Intelligence, MICAI 2022, Proceedings, Part II, Monterrey, Mexico. Springer. find
  6. Hassan Al-Sukhani, Qusay Bsoul, Abdelrahman H. Elhawary, Ziad M. Nasr, Ahmed E. Mansour, Radwan M. Batyha, Basma S. Alqadi, Jehad Saad Alqurni, Hayat Alfagham, and Magda M. Madbouly. 2025. Multilingual hate speech detection: Innovations in optimized deep learning for english and arabic hate speech detection. SN Computer Science, 6(205). ↗
  7. Sadeem Alrasheed, Suliman Aladhadh, and Abdulatif Alabdulatif. 2025. Protecting intellectual security through hate speech detection using an artificial intelligence approach. Algorithms, 18(4):179. ↗
  8. Eman S. Alshahrani, Mehmet S. Aksoy, and Ahmed Emam. 2025. Detection of hate speech and offensive language in arabic text: A systematic literature review. Applied Computational Intelligence and Soft Computing. First published: 13 April 2025. ↗
  9. Karim Gasmi, Ibtihel Ben Ltaifa, Alameen Eltoum Abdalrahman, Omer Hamid, Mohamed Othman Altaieb, and Shahzad Ali. 2025. Hybrid feature and optimized deep learning model fusion for detecting hateful arabic content. IEEE Access, 13. ↗
  10. O. Kolesnikova and A. Gelbukh. 2020. A study of lexical function detection with word2vec and supervised machine learning. J. Intell. Fuzzy Syst., 39. ↗
  11. Olumide E. Ojo, Olaronke O. Adebanji, Hiram Calvo, Damian O. Dieke, Olumuyiwa E. Ojo, Seye E. Akin-sanya, Tolulope O. Abiola, and Anna Feldman. 2023. Legend at araieval shared task: Persuasion technique detection using a language-agnostic text representation model. Preprint, arXiv:2310.09661. ↗
  12. Temitope Oladepo, Oluwatobi Abiola, Tolulope Abiola, Abdullah , Usman Muhammad, and Babatunde Abiola. 2025. Predicting emotion intensity in text using transformer-based models. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 1677–1682, Vienna, Austria. Association for Computational Linguistics. find
  13. Wajdi Zaghouani, Md Rafiul Biswas, Mabrouka Bessghaier, Shimaa Ibrahim, Georgio Mikros, Abul Hasnat, and Firoj Alam. 2025. Overview of mahed shared task: Multimodal detection of hope and hate emotions in arabic content. In The Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abiola-2025-cic-nlp-mahed-2025-task/