📄 Fine-Tuning Approach for Arabic Offensive Language Detection System: BERT-Based Model

Summary

The paper looks at automatic detection of offensive comments in Arabic social-media text. This content is mostly dialectal, and dialect diversity makes it hard for NLP tools. The authors first compared four pretrained transformers on two datasets using identical hyperparameters: XLM-RoBERTa, M-BERT, Arabic-BERT and AraBERT. The Arabic-only models did better than the multilingual ones, and AraBERT came out on top.

They then fine-tuned AraBERT on four public corpora, Aljazeera deleted comments, YouTube comments, L-HSAB and OSACT. All labels were reduced to offensive versus not offensive. Each corpus was tested separately, then across corpora, and finally with all training sets pooled together.

Within a single corpus, scores were strong on OSACT and weakest on Aljazeera. Pooling the data brought no improvement and slightly hurt the Levantine L-HSAB set. Transfer between datasets helped little, especially for heavily dialectal comments.

The practical lesson is that a large pretrained model plus more pooled data will not automatically cover dialect and domain shifts in abuse detection.

Key points

• Four models were compared on OSACT and L-HSAB: XLM-RoBERTa, M-BERT, Arabic-BERT and AraBERT. AraBERT scored best on both, and the monolingual models beat the multilingual ones.
• The same settings were used everywhere:
  • max length 128, batch size 16, 5 epochs, learning rate 2e-5, epsilon 1e-8
  • the classifier is the pooled encoder output with a simple FFNN layer on top
  • no text preprocessing was applied
• Each dataset was merged from any official splits and re-split randomly 80/20. Multi-class labels (hate, abusive, obscene) were collapsed into a single offensive class.
• In-domain macro-F1 / accuracy results:
  • OSACT 0.91 / 0.94
  • L-HSAB 0.87 / 0.88
  • YouTube 0.87 / 0.88
  • Aljazeera 0.74 / 0.85
• Fine-tuning on the concatenated corpus gave no gains over single-corpus training. L-HSAB lost about 3 points of macro-F1, which the authors attribute to its Levantine vocabulary being rare in the other sets.

Interesting ideas

• I1. Training on each corpus and testing on every other one shows how far a model really transfers, which a single in-domain score hides. — For Mova (learn): Run a full treebank-by-treebank train/test grid for the perceptron parser and gparse to see which genres and dialects the parser actually generalizes to.
• I2. Merging every dataset can drag down the most distinctive subset, as happened with the dialect-heavy L-HSAB. Pooled results should therefore always be broken down per source. — For Mova (tree): When gparse is trained on mixed treebanks, report LAS per treebank so that losses on unusual registers are not masked by the overall average.
• I3. The model choice was made with a cheap pilot on two datasets under frozen, shared hyperparameters, so differences come from the models rather than from tuning. — For Mova (learn): Compare candidate witness parsers or LLM teacher tiers on a small fixed slice with one shared configuration before investing in full training runs.
• I4. Mapping several incompatible label schemes onto one coarse shared label makes heterogeneous corpora directly comparable and combinable. — For Mova (tree): Use the same move in UD dialect conversion, collapsing fine treebank-specific relations onto a common core before mixing data or comparing scores.
• I5. Related work cited in the paper finds that errors cluster on insulting words used harmlessly, sarcasm, and quoting someone else's offensive statement. These cases need context rather than keywords. — For Mova (frames): Mark reported or quoted speech and ironic speech acts in prag, so that a quoted insult is attributed to its original speaker rather than asserted by the writer.