📄 REGLAT at MAHED Shared Task: A Hybrid Ensemble-Based System for Arabic Hate Speech Detection

Summary

The REGLAT team built a system for subtask 1 of the MAHED shared task at ArabicNLP 2025. The subtask asks for social media posts in Arabic dialects and MSA to be sorted into three classes: Hate, Hope, or Not Applicable. The organizers supplied manually annotated data, already divided into training, validation, and test portions. The authors cleaned the text with regex and NLTK, then compared several families of models:

• sparse TF-IDF features fed to SVM and Logistic Regression;
• a CNN-BiLSTM network using either AraVec or GloVe embeddings;
• fine-tuned AraBERTv2 and CAMeLBERT.

Their final system combines the predictions of a TF-IDF + Logistic Regression model and a fine-tuned AraBERTv2 model through majority voting. This hybrid reached a macro F1 of 0.58, clearly above any single model; the best standalone result was AraBERTv2 at 0.46. The main lesson is that a simple linear model over n-grams still adds value next to a transformer. Combining lexical and contextual signals is a cheap way to gain accuracy on imbalanced, dialect-rich classification.

Key points

• The data is MAHED subtask 1, with labels Hate, Hope, and Not Applicable. The organizers split it 70/15/15 into train, validation, and test, and the official metric is macro F1 because the classes are imbalanced.
• Preprocessing removes URLs, mentions, punctuation, symbols, emojis, Latin letters, and digits, and normalizes Arabic letters.
• Single-model macro F1 scores:
  • SVM (TF-IDF): 0.42
  • LR (TF-IDF): 0.40
  • AraBERTv2: 0.46
  • CAMeLBERT: 0.36
  • CNN-BiLSTM with AraVec (300d): 0.31
  • CNN-BiLSTM with GloVe (200d): 0.12
• Hybrid macro F1 scores:
  • LR-AraBERTv2: 0.58 (best)
  • LR-CAMeLBERT: 0.54
  • SVM-AraBERTv2: 0.47
  • SVM-CAMeLBERT: 0.44
• Settings:
  • TF-IDF: 1–3-grams, at most 50,000 features, min_df 5, max_df 0.9, sublinear tf.
  • LR: class_weight balanced.
  • AraBERTv2: AdamW, learning rate 2e-5, batch size 16, 3 epochs.
  • Hardware: a Colab T4 GPU.

Interesting ideas

• I1. Combining a sparse n-gram linear model with a contextual transformer beat both of them (0.58 vs 0.40 and 0.46), so the two models evidently make different mistakes.
• I2. The best partner for the transformer was not the strongest solo model: LR scored below SVM on its own but produced the better hybrid.
• I3. The authors handled a skewed label set, where a catch-all "Not Applicable" class dominates, with balanced class weights and macro F1, so rare classes count fully.
• I4. A language-specific embedding (AraVec) far outperformed a general-purpose one (GloVe) inside the same architecture (0.31 vs 0.12), showing that resource match can matter more than model design.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): when assembling witness parsers for silver data, include a deliberately cheap, feature-based witness next to gparse, so that agreement draws on genuinely different error profiles.
• I2 (learn): choose the set of witnesses or teachers by measuring how the combined ensemble scores on dev data, not by ranking each candidate's individual LAS.
• I3 (frames): speech-act and irony labels are similarly rare, so classifiers for them should use class weighting and be reported with macro F1 rather than accuracy.
• I4 (tokens): before tuning the tagger's architecture, check whether the lexical resources and embeddings actually match the genre being processed, such as school texts or fables.