📄 ASU_OPTO at OSACT4 - Offensive Language Detection for Arabic text

Summary

The paper describes the ASU_OPTO team's entry to the OSACT4 shared task on Arabic tweets. Subtask A asks whether a tweet is offensive in any way. Subtask B asks whether it contains hate speech aimed at a person or group.

The team compared five classifiers on Subtask A:

• a tf-idf character n-gram baseline with logistic regression,
• a CNN over AraVec embeddings,
• a BiLSTM over AraVec embeddings,
• fine-tuned multilingual BERT,
• fine-tuned AraBERT, whose input is segmented with Farasa.

AraBERT performed best. Multilingual BERT came out clearly behind it, which the authors take as a sign that Arabic needs a language-specific model and tokenization.

They then combined AraBERT with a lookup step. Any tweet containing a word from a morphologically expanded profanity list is forced to the offensive label. This hybrid gave a small further gain and placed fourth in Subtask A. The same AraBERT setup, retrained for hate speech, placed third in Subtask B.

The paper closes with an error analysis of annotation problems: inconsistent labels, quoted movie lines, and outright mislabeled tweets. It is useful for anyone weighing monolingual against multilingual transformers, or adding rule-based lexicons on top of neural classifiers.

Key points

• Development macro-F1 on Subtask A:

  Model	Dev macro-F1
  tf-idf + LR	0.746
  CNN + Aravec	0.867
  BiLSTM	0.869
  multilingual BERT	0.826
  AraBERT	0.876
  AraBERT + profanity list	0.880

• Official results: the hybrid scored 0.896 macro-F1 (4th place, Subtask A). The BiLSTM, their second-best system, scored 0.856. AraBERT on Subtask B scored 0.807 (3rd place).

• Profanity lexicon: a seed list of 87 bad words grew to 5497 forms. The expansion used three steps:

  • orthographic swaps (Taa-marbuta→Haa, Zain→Zaal),
  • words found by manually inspecting the training data,
  • prefix/suffix lists by part of speech; one verb alone yielded 113 forms.

• Training setup: AraBERT was fine-tuned on 7000 training tweets for six epochs at learning rate 1e-5, with sequence length 64. Multilingual BERT used length 75 and three epochs.

• Reproducibility: GPU training gave different results across runs despite a fixed seed of 42. The authors' workaround was to save the weights from several runs and keep the best one.

Interesting ideas

• I1. Generating every affix combination, including invalid ones, is harmless for precision, because forms that don't exist in the language never appear in real text.
• I2. A small, high-precision lexicon can override a neural classifier's decision, adding an explicit and inspectable rule on top of the learned model.
• I3. The error analysis looked for the same cue word appearing in tweets with opposite labels, which exposes inconsistent annotation.
• I4. Quoted movie lines can be offensive in literal meaning while not offensive in use, so the meaning depends on whether the speaker is quoting someone else.
• I5. Nondeterminism on GPUs persists even with a fixed seed, so a single run is not a trustworthy measurement.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): lemma lookup tables in crate en could be built by aggressive rule-based inflection without filtering, as long as they are used only for matching and never for generation.
• I2 (frames): insult or threat speech acts in prag could fire from a curated cue lexicon whose matches are logged as the explanation, with the statistical component used only as a fallback.
• I3 (learn): silver data could be audited by grouping items that share an identical lexical or structural cue and flagging conflicting labels before training.
• I4 (frames): quoted or reported speech should be marked so its speech act is attributed to the quoted voice, not to the author, which also helps irony detection.
• I5 (learn): gparse evaluations could train several runs, keep every checkpoint, and report the spread across runs alongside the best result.