Elamur Research⌕
☰
home / papers / 2022 / Arabic Fake News Detection Based on Deep Contextualized Embedding Models

Arabic Fake News Detection Based on Deep Contextualized Embedding Models

Summary

Arabic fake news detection has had little research compared with English, mostly because labelled Arabic data is scarce. The language also has spellings that change meaning with context and many divergent dialects. The authors attack the data gap from two directions. They assemble their own Arabic corpus by web scraping and by drawing on official Arabic news outlets. They also produce a second corpus by machine-translating an English fake news dataset from Kaggle into Arabic with a Python script. On both corpora they fine-tune transformer classifiers built on eight state-of-the-art Arabic contextualized embedding models, most of which had not been tried on this task before. They score each model with accuracy, precision, recall and F-score and examine its false positives and false negatives. Accuracy passes 98%, and AraBert and QaribBert set a new state of the art. The natively Arabic corpus gives better results than the translated one. The work supplies a sizeable resource for a low-resource task and a broad comparison of Arabic encoders.

Key points

  • Two datasets are used: a newly built native Arabic fake news corpus (scraped data plus official Arabic news) and an Arabic machine translation of an English Kaggle fake news dataset.
  • Eight transformer-based classifiers are compared, each built on a different Arabic contextualized embedding model.
  • The best systems exceed 98% accuracy, and AraBert and QaribBert give the new state-of-the-art results.
  • BERT models for Arabic perform better on the natively Arabic dataset than on the translated one.
  • Evaluation covers accuracy, precision, recall and F-score, plus a discussion of false negatives and false positives. The paper is published in Neural Computing and Applications (Springer).

Interesting ideas

  • I1. Training the same models on a native corpus and on a machine-translated corpus shows how much translation artefacts cost, since the native data wins — For Mova (learn): when building silver data, keep a translated or synthetic subset as a control to measure how far it drifts from natively written English text.
  • I2. Running many pretrained encoders under identical data, setup and metrics turns model choice into an evidence-based decision rather than a habit — For Mova (tree): gparse could benchmark several candidate encoder backbones on the same UD treebanks with LAS before committing to one.
  • I3. Reporting where errors fall, split into false positives and false negatives, adds insight beyond a single accuracy number, which can hide systematic failure modes — For Mova (learn): evaluation reports could break mistakes down by error type so repair rules target the dominant failure.
  • I4. The paper's example of one Arabic spelling read as either "go" or "gold" shows that only sentence context can settle the reading of an ambiguous form — For Mova (tokens): the tagger and lemmatizer should resolve English noun/verb homographs from syntactic context and record which cue decided, keeping the choice explainable.
  • I5. When labelled data is missing, a corpus can be bootstrapped from credible official outlets as one class plus scraped material, or by translating an existing resource — For Mova (answer): QA or test sets for fables and tales could be seeded from trusted sources and translated material, each tagged with its origin for later comparison.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1007/s00521-022-07206-4 · arXiv 2205.03114

@misc{nassif2022arabic,
  title = {{Arabic Fake News Detection Based on Deep Contextualized Embedding Models}},
  author = {Nassif, Ali Bou and Elnagar, Ashraf and Elgendy, Omar and Afadar, Yaman},
  year = {2022},
  doi = {10.1007/s00521-022-07206-4},
  eprint = {2205.03114},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/s00521-022-07206-4}
}

References

36 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jardaneh G, Abdelhaq H, Buzz M, Johnson D (2019) Classifying Arabic tweets based on credibility using content and user features. In: 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology, JEEIT 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., pp 596–601 find
  2. Injadat M, Salo F, Nassif AB (2016) Data mining techniques in social media: A survey. Neurocomputing 214:. https://doi.org/10.1016/j.neucom.2016.06.045 ↗
  3. Mehta D, Dwivedi A, Patra A, Anand Kumar M (2021) A transformer-based architecture for fake news classification. Soc Netw Anal Min 11:39. https://doi.org/10.1007/s13278-021-00738-y ↗
  4. de Souza JV, Gomes Jr. J, Souza Filho FM de, et al (2020) A systematic mapping on automatic classification of fake news in social media. Soc Netw Anal Min 10:48. https://doi.org/10.1007/s13278-020-00659-2 ↗
  5. Injadat M, Moubayed A, Nassif AB, Shami A (2021) Machine learning towards intelligent systems: applications, challenges, and opportunities. Artif Intell Rev 54:3299–3348 find
  6. Nassif AB, Shahin I, Attili I, et al (2019) Speech Recognition Using Deep Neural Networks: A Systematic Review. IEEE Access 7:19143–19165. https://doi.org/10.1109/ACCESS.2019.2896880 ↗
  7. Nassif AB, Shahin I, Bader M, et al (2022) COVID-19 Detection Systems Using Deep-Learning Algorithms Based on Speech and Image Data. Mathematics find
  8. Hijazi H, Abu Talib M, Hasasneh A, et al (2021) Wearable Devices, Smartphones, and Interpretable Artificial Intelligence in Combating COVID-19. Sensors 21:. https://doi.org/10.3390/s21248424 ↗
  9. Nassif AB, Darya AM, Elnagar A (2022) Empirical evaluation of shallow and deep learning classifiers for Arabic sentiment analysis. Trans Asian Low-Resource Lang Inf Process 21:1–25 find our card
  10. Oueslati O, Cambria E, HajHmida M Ben, Ounelli H (2020) A review of sentiment analysis research in Arabic language. Futur Gener Comput Syst 112:408–430. https://doi.org/10.1016/j.future.2020.05.034 ↗ our card
  11. Nassif AB, Elnagar A, Shahin I, Henno S (2020) Deep learning for Arabic subjective sentiment analysis: Challenges and research opportunities. Appl. Soft Comput. 98:106836 find
  12. Boudad N, Faizi R, Thami ROH, Chiheb R (2018) Sentiment analysis in Arabic: A review of the literature. Ain Shams Eng J 9:2479–2490. https://doi.org/https://doi.org/10.1016/j.asej.2017.04.007 ↗
  13. Vilares D, Peng H, Satapathy R, Cambria E (2019) BabelSenticNet: A Commonsense Reasoning Framework for Multilingual Sentiment Analysis. Proc 2018 IEEE Symp Ser Comput Intell SSCI 2018 1292–1298. https://doi.org/10.1109/SSCI.2018.8628718 ↗
  14. Alwaneen TH, Azmi AM, Aboalsamh HA, et al (2021) Arabic question answering system: a survey. Artif Intell Rev. https://doi.org/10.1007/S10462-021-10031-1 ↗
  15. Saadany H, Mohamed E, Or˘ C (2020) Fake or Real? A Study of Arabic Satirical Fake News. Online find
  16. Helwe C, Elbassuoni S, Al Zaatari A, El-Hajj W (2019) Assessing Arabic Weblog Credibility via Deep Co-learning. Association for Computational Linguistics (ACL), pp 130–136 find
  17. Rangel F, Rosso P, Charfi A, Zaghouani W (2019) Detecting deceptive tweets in arabic for cybersecurity. In: 2019 IEEE International Conference on Intelligence and Security Informatics, ISI 2019. Institute of Electrical and Electronics Engineers Inc., pp 86–91 find
  18. El Ballouli R, El-Hajj W, Ghandour A, et al (2017) CAT: Credibility Analysis of Arabic Content on Twitter. Association for Computational Linguistics (ACL), pp 62–71 find
  19. Haouari F, Ali ZS, Elsayed T (2019) bigIR at CLEF 2019: Automatic Verification of Arabic Claims over the Web. undefined find
  20. Sabbeh SF, Baatwah SY (2018) ARABIC NEWS CREDIBILITY ON TWITTER: AN ENHANCED MODEL USING HYBRID FEATURES. J Theor Appl Inf Technol 96: find
  21. Sutanto DH, Ghani MKA (2015) A BENCHMARK OF CLASSIFICATION FRAMEWORK FOR NON-COMMUNICABLE DISEASE PREDICTION : A REVIEW find
  22. Alkhair M, Meftouh K, Smaïli K, Othman N (2019) An Arabic Corpus of Fake News: Collection, Analysis and Classification. Commun Comput Inf Sci 1108:292–302. https://doi.org/10.1007/978-3-030-32959-4_21 ↗
  23. Hadj Ameur MS, Aliane H (2021) AraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News and Hate Speech Detection Dataset. Procedia CIRP 189:232–241. https://doi.org/10.1016/j.procs.2021.05.086 ↗ our card
  24. Al-Yahya M, Al-Khalifa H, Al-Baity H, et al (2021) Arabic Fake News Detection: Comparative Study of Neural Networks and Transformer-Based Approaches. Complexity 2021:. https://doi.org/10.1155/2021/5516945 ↗
  25. Ozbay FA, Alatas B (2020) Fake news detection within online social media using supervised artificial intelligence algorithms. Phys A Stat Mech its Appl 540:123174. https://doi.org/10.1016/j.physa.2019.123174 ↗
  26. Traylor T, Straub J, Gurmeet, Snell N (2019) Classifying Fake News Articles Using Natural Language Processing to Identify In-Article Attribution as a Supervised Learning Estimator. In: Proceedings - 13th IEEE International Conference on Semantic Computing, ICSC 2019. Institute of Electrical and Electronics Engineers Inc., pp 445–449 find
  27. Yang K-C, Niven T, Kao H-Y (2019) Fake News Detection as Natural Language Inference. arXiv find
  28. Kaliyar RK (2018) Fake news detection using a deep neural network. In: 2018 4th International Conference on Computing Communication and Automation, ICCCA 2018. Institute of Electrical and Electronics Engineers Inc. find
  29. Antoun W, Baly F, Achour R, et al (2020) State of the Art Models for Fake News Detection Tasks. In: 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies, ICIoT 2020. Institute of Electrical and Electronics Engineers Inc., pp 519–524 find
  30. Conneau A, Khandelwal K, Goyal N, et al (2019) Unsupervised Cross-lingual Representation Learning at Scale find
  31. Lan W, Chen Y, Xu W, Ritter A (2020) An Empirical Study of Pre-trained Transformers for Arabic Information Extraction find
  32. Antoun W, Baly F, Hajj H (2020) AraBERT: Transformer-based Model for Arabic Language Understanding find our card
  33. Chowdhury SA, Abdelali A, Darwish K, et al (2020) Improving Arabic Text Categorization Using Transformer Training Diversification find
  34. Safaya A, Abdullatif M, Yuret D (2020) KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media. Online find our card
  35. Wang H, Zheng H (2013) True Positive Rate. In: Encyclopedia of Systems Biology. Springer New York, pp 2302–2303 find
  36. Nagoudi EMB, Elmadany A, Abdul-Mageed M, et al (2020) Machine Generation and Detection of Arabic Manipulated and Fake News find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nassif-2022-arabic-fake-news-detection-based/