Elamur Research⌕
☰
home / papers / 2026 / A Hybrid Confidence-Aware Framework for Arabic Toxicity Detection in Social Media

A Hybrid Confidence-Aware Framework for Arabic Toxicity Detection in Social Media

Paper · 2026 · Fawzia Zaal Alanazi, Asma Mohammed Alamri, Arwa Bin Saleh et al.

Summary

The paper tackles automatic detection of toxic and offensive posts in Saudi-dialect Arabic social media. Rich morphology, dialect variation and noisy writing make neural classifiers unsure on borderline inputs. The authors fine-tune AraBERT on the "X Posts Hate Speech Dataset for the Saudi Dialect" (35,000 posts labelled by native speakers, published by Alhazmi on Mendeley Data in 2024). They add a confidence-aware controller on top of the classifier. Lexical rules, drawn from an automatically induced lexicon, are consulted only when the model's probability falls inside a predefined uncertainty band (the "gray zone"). Confident predictions pass through unchanged, so the neural model stays dominant. The lexicon covers cues such as tribal slurs, sentiment-bearing emojis and dialectal markers. On the held-out test set, the hybrid system improves over the AraBERT baseline, mainly by recovering toxic posts the network missed. The authors report that this raises recall without hurting precision. The work matters because it shows a cheap, interpretable way to make moderation systems more reliable exactly where the network is weakest.

Key points

  • Data: 35,000 Saudi-dialect X posts with binary labels (Toxic/Non-toxic), roughly 47.4% toxic and 52.6% non-toxic, split 70/15/15 with stratification.
  • Preprocessing removes URLs, mentions, hashtags, punctuation and numbers, and normalises diacritics and letter elongation. Emojis are kept, and there is no conversion to MSA. AraBERT tokenisation uses max length 128 and do_lower_case=False.
  • The reported baseline macro-F1 is about 88.10%, with toxic-class recall 0.8870 and precision 0.8640. The paper also reports a macro-F1 of 0.895 alongside 171 false positives and 198 false negatives on the test set.
  • Residual errors fall into three groups:
    • harsh but non-offensive criticism, which gets over-flagged;
    • implicit insults and sarcasm without profanity, which get missed;
    • posts whose toxicity depends on the conversational context.
  • Limitations: only one dialect was tested, and the gray-zone thresholds were set empirically and need recalibration. The lexicon is also sensitive to surface-form and morphological variants, and the hybrid design adds system complexity.

Interesting ideas

  • I1. Symbolic rules are applied only inside a calibrated band of low model confidence, which keeps the strong model in charge while letting transparent cues fix its shakiest calls.
  • I2. The correcting lexicon is induced automatically from labelled data rather than written by hand, giving cheap, inspectable features that explain each override.
  • I3. The error analysis separates "harsh but not targeted" from "polite surface, offensive intent", showing that sentence-level lexical cues fail on sarcasm and indirect disparagement.
  • I4. Emojis are deliberately kept as meaningful tokens instead of being stripped as noise, because they carry sentiment and sarcasm in informal text.
  • I5. The authors warn that empirically chosen confidence thresholds may not transfer across datasets, which argues for re-tuning gates per domain.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Parser decisions or arcs with low margin could trigger hand-checkable repair rules, while confident arcs stay untouched, which limits the damage rules can do.
  • I2 (learn): Disagreements between witness parsers on silver data could be mined into explicit repair rules that fire only when the parsers' agreement is weak.
  • I3 (frames): Irony and implicature detection in prag could use this split as a test category, checking whether the target of an attack is a person or an opinion/situation.
  • I4 (tokens): The en tokenizer could treat emojis as first-class tokens with lemma and sentiment features rather than discarding them.
  • I5 (learn): Any confidence gate in Mova should be re-calibrated on each new corpus and checked against a negative control where the gate is disabled.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Imam Mohammad ibn Saud Islamic UniversityRiyadh · Saudi Arabia11 works · 51 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 3,066 worksBias, fairness & safety · 587 works

Cite

Cite as Fawzia Zaal Alanazi, Asma Mohammed Alamri, Arwa Bin Saleh and Abdullah I. Alharbi. 2026. A Hybrid Confidence-Aware Framework for Arabic Toxicity Detection in Social Media. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.abjadnlp-1.42

@inproceedings{alanazi2026hybrid,
  title = {{A Hybrid Confidence-Aware Framework for Arabic Toxicity Detection in Social Media}},
  author = {Fawzia Zaal Alanazi, Asma Mohammed Alamri, Arwa {Bin Saleh} and Abdullah I. Alharbi},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.42},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.42}
}

References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kathrein Abu Kwaik, Stergios Chatzikyriakidis, Simon Dobnik, Motaz Saad, and Richard Johansson. 2020. An Arabic tweets sentiment analysis dataset (AT-SAD) using distant supervision and self training. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 1–8, Marseille, France. European Language Resource Association. find
  2. Abdulmohsen Al-Thubaity, Qubayl Alqahtani, and Abdulaziz Aljandal. 2018. Sentiment lexicon for sentiment analysis of saudi dialect tweets. Procedia Computer Science, 142:301–307. find
  3. Abdullah I. Alharbi and Mark Lee. 2020. Combining character and word embeddings for the detection of offensive language in Arabic. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, pages 91–96, Marseille, France. European Language Resource Association.
  4. Alberto Barrón-Cedeño and Andrés García-Silva. 2025. Nlp4good: Beto-mp model for spanish toxicity classification. Journal of Computational Linguistics, 51(1):85–101. find
  5. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T. Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R. El-Beltagy, Wassim El-Hajj, Mustafa Jarrar, and Hamdy Mubarak. 2021. A panoramic survey of natural language processing in the arab world. Commun. ACM, 64(4):72–81. ↗
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT.
  7. Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. ACM Computing Surveys, 51(4). ↗
  8. Bert Kamphuis and Sjoerd van der Meer. 2024. Tiny-toxic-detector: A compact transformer for toxicity detection. Transactions of the ACL, 12(3):215–229. find
  9. Lucas Magnossão, Gabriela Cardenas, and José L. Martín. 2022. Upv at arabic hate speech 2022: Multi-dialect transformer ensembles. In Proceedings of the WANLP Shared Task on Hate Speech and Offensive Language in Arabic, pages 66–75. find
  10. Rabia Rachidi, Mohamed Amine Ouassil, Mouaad Errami, Mounir Omari, Bouchaib Cherradi, and Hassan Silkan. 2025. Leveraging BERT models for toxicity detection in moroccan dialect. In Proceedings of the 5th International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET), Fez, Morocco. find
  11. Azzam Radman, Mohammed Atros, and Rehab Duwairi. 2022. Spans detection of toxic phrases in arabic tweets. In Proceedings of the 13th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan. find
  12. Mohammed Alrashidi and Mashael AlGhamdi. 2023. Multi-task learning for fine-grained abusive language detection in arabic. Arabic Language Processing, 10(2):143–157. find
  13. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT). ↗
  14. Abdulaziz Asiri and Mohammed Saleh. 2024. Sod: A corpus for saudi offensive language detection. Computers, 13(1). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alanazi-2026-hybrid-confidence-aware-framework-arabic/