📄 Baoflowin502 at MAHED Shared Task: Text-based Hate and Hope Speech Classification

Summary

The paper addresses three-way classification of Arabic social media posts into hate speech, hope speech, or not-applicable for the MAHED shared task. The task is hard because Arabic has rich morphology, many dialects and heavy cultural nuance. The authors fine-tune three Arabic-specific encoders (AraBERTv2, CAMeLBERT and MARBERT) on a balanced set of 6,889 posts. Before training, they clean the text: emojis become words, URLs and mentions are removed, and the script is normalized. To enlarge the training data, they add three kinds of new examples:

• paraphrases from a translation round trip through English and French;
• synthetic posts written by GPT-4 from a few examples;
• synonym swaps that leave key terms untouched.

For comparison, they also train a bagged XGBoost and an MLP on AraBERTv2 embeddings. Augmentation improves every encoder, and a majority vote over the three fine-tuned models reaches 0.689 Macro-F1 in 5-fold cross-validation. Classical models score close to the transformers. This suggests the difficulty comes from fuzzy class boundaries and limited data rather than model capacity, which is a useful signal for low-resource moderation work.

Key points

• Data: 6,889 posts split almost evenly: Hate 2,296, Hope 2,301 and Not-applicable 2,292. The reported sources are Twitter, Facebook and regional forums.

• No-augmentation baselines (5-fold CV Macro-F1):

  Model	Macro-F1
  bagged XGBoost	0.626
  MLP	0.616
  AraBERTv2	0.623
  CAMeLBERT	0.647
  MARBERT	0.639

• With augmentation at a 50% ratio:

  Model	Macro-F1
  AraBERTv2	0.665
  MARBERT	0.652
  CAMeLBERT	0.679
  XGBoost	0.656
  hard voting ensemble	0.689

• Augmentation details:

  • Google Translate round trip Arabic→English→French→Arabic, filtered by multilingual embedding similarity.
  • GPT-4 prompts with 3–5 examples per class.
  • Substitution via Arabic WordNet, with POS tagging and NER protecting names and class-bearing words.

• Setup: Stratified K-Fold, Optuna tuning, NVIDIA Tesla P100.

• Caveats:

  • The abstract names CAMeLBERT (0.6868) as best, which disagrees with the tables.
  • There is no ablation separating the individual augmentation methods.

Interesting ideas

• I1. Chaining translation through two pivot languages yields more varied paraphrases than a single hop, and an embedding-similarity filter discards drifted outputs before they pollute training data.
• I2. Synonym substitution that uses POS and NER to shield proper nouns and decisive words creates lexical variety without changing the label.
• I3. Hard majority voting across differently specialized models, with ties broken by the most confident member, beat every single model.
• I4. A classical model on frozen embeddings scoring close to fine-tuned transformers was read as evidence that labels, not capacity, limit accuracy.
• I5. Turning emojis into textual descriptions keeps their emotional signal available to a text-only model.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): English→X→Y→English paraphrases could serve as invariance probes, where the UD tree may change but the extracted frames must not, giving a cheap consistency check on silver data.
• I2 (tree): swapping same-POS lemmas in treebank sentences while keeping the gold arcs could augment parser training and test whether gparse relies on memorized word forms.
• I3 (learn): the same scheme fits the witness-parser agreement step, where a vote decides each arc and the highest-confidence parser settles disagreements.
• I4 (frames): before scaling irony or speech-act classifiers, a simple embedding baseline could show whether the category boundaries themselves are too fuzzy to learn.
• I5 (tokens): mapping emojis to word tokens in the en tokenizer would let the tagger and lemmatizer handle them, so prag can later use them as tone or irony cues.