Elamur Research⌕
☰
home / papers / 2020 / Fake or Real? A Study of Arabic Satirical Fake News

Fake or Real? A Study of Arabic Satirical Fake News

Paper · 2020 · Hadeel Saadany, Constantin Orasan and Emad Mohamed

Summary

This paper asks whether Arabic satirical news, which imitates the style of official outlets and can be mistaken for genuine reporting once shared without its source, can be told apart from real news automatically. The authors built a new corpus. The satirical side has 3185 articles scraped from Al-Hudood and Al-Ahram Al-Mexici. The real side has 3710 articles from BBC-Arabic, CNN-Arabic and Al-Jazeera. Both sides cover Middle Eastern politics. They used inferential statistics to show that satire carries distinctive lexico-grammatical traits. They then trained several classifiers, from count/TF-IDF baselines to neural networks. A CNN fed with pre-trained word embeddings performed best, reaching about 98.6% accuracy, and it recovered satirical items that other models had labelled as real. Error and feature analyses showed that satire imitates credibility markers and adds invented hyperbole, while real reports anchor events in time and place. The work matters because Arabic fake-news research is sparse. It also suggests that, for satire, the text alone can be enough, with no need for source or social-network metadata.

Key points

  • New dataset: 3185 satirical articles (Al-Hudood, Al-Ahram Al-Mexici) and 3710 real articles (BBC-Arabic, CNN-Arabic, Al-Jazeera), all on Middle Eastern political topics.
  • The best model is a CNN with pre-trained word embeddings, with recall 98.61 and precision 98.49. XGBoost came very close and ran much faster.
  • Most errors across all models were false negatives, meaning satire taken for real news. As a result, precision was generally higher than recall.
  • Misclassified satirical pieces were written entirely in MSA, contained no slang, and packed in up to 19 journalistic clichés each. The emotion lexicon that scored how strong their sentiment was failed to pick up their coined hyperboles.
  • Top features from the CountVectorizer/TF-IDF baselines differed by class:
    • Satire: "the source", reporting verbs such as "asserted", strongly polar adjectives, and names like Trump and Al-sisi.
    • Real news: time and place words such as Monday and Gaza, and never an emotive term.

Interesting ideas

  • I1. The hardest satire hides behind a dense layer of stock news phrasing, so a high count of credibility formulas combined with odd evaluative praise is itself a signal; this pattern of register parody plus hyperbole generalises to other irony detection.
  • I2. A fixed emotion lexicon missed the satirist's invented hyperboles, which were built from ordinary nouns and descriptive adjectives; intensity therefore has to be read from how words combine, not looked up word by word.
  • I3. Genuine reports were dominated by time and venue anchors, while satire leaned on emotive and attributional words; a missing grounding of events in time and place is a cheap, explainable genre cue.
  • I4. Reading the per-class weights of a simple linear baseline gave a quick, human-readable picture of what separates the classes before trusting a black-box model.
  • I5. The authors checked their classifier against the very article that caused the Egypt–Mexico incident, a memorable real-world case used as a sanity test.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): the irony module could flag texts where many attribution and assertion frames ("the source said", "asserted") wrap evaluations far stronger than the reporting genre normally allows, and explain the flag by pointing to those frames.
  • I2 (frames): compute evaluative intensity compositionally over the UD tree (amod/nmod chains around a person entity), so novel epithets are scored even when no single word is in a lexicon.
  • I3 (world): when building the world of a text, record how many events get explicit temporal and locative anchoring, and expose a low ratio as an interpretable feature for genre or reliability judgements.
  • I4 (learn): as a negative control, fit a shallow lexical baseline on each new silver dataset and inspect its top features, to catch tasks a model could solve through superficial cues instead of structure.
  • I5 (answer): keep a small named suite of notorious hard cases (famous ironic or misleading texts) as regression tests that readers must handle with an explained answer.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WolverhamptonWolverhampton · United Kingdom35 works · 6,331 citations🎓University of SurreyGuildford · United Kingdom21 works · 677 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Hadeel Saadany, Constantin Orasan and Emad Mohamed. 2020. Fake or Real? A Study of Arabic Satirical Fake News. Proceedings of the 3rd International Workshop on Rumours and Deception in Social Media (RDSM).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{saadany2020fake,
  title = {{Fake or Real? A Study of Arabic Satirical Fake News}},
  author = {Hadeel Saadany, Constantin Orasan and Emad Mohamed},
  booktitle = {Proceedings of the 3rd International Workshop on Rumours and Deception in Social Media (RDSM)},
  year = {2020},
  url = {https://aclanthology.org/2020.rdsm-1.7/}
}

References

35 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rasha M BinSultan Al-Eidan, Hend S Al-Khalifa, and AbdulMalik S Al-Salman. 2010. Measuring the credibility of Arabic text content in Twitter. In 2010 Fifth International Conference on Digital Information Management (ICDIM), pages 285–291. IEEE. find
  2. Dana Al-Ghadhban, Eman Alnkhilan, Lamma Tatwany, and Muna Alrazgan. 2017. Arabic sarcasm detection in twitter. In 2017 International Conference on Engineering & MIS (ICEMIS), pages 1–7. IEEE. find
  3. “Al-Hudood”. 2019. The limits. https://alhudood.net. [Online; accessed 29-June-2019]. ↗
  4. AlAhram-AlMexici. 2020. The mexican ahram. http://alahraam.com/. [Online; accessed 29-June-2020]. ↗
  5. Ali Allaith, Muhammad Shahbaz, and Mohammed Alkoli. 2019. Neural Network Approach for Irony Detection from Arabic Text on Social Media. In FIRE (Working Notes), pages 445–450. find
  6. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. ↗
  7. Brian Brett. 2012. The psychology of sharing: why do people share online. New York Times. find
  8. Clint Burfoot and Timothy Baldwin. 2009. Automatic satire detection: Are you having a laugh? In Proceedings of the ACL-IJCNLP 2009 conference short papers, pages 161–164. find
  9. François Chollet et al. 2018. Keras: The python deep learning library. Astrophysics Source Code Library. find
  10. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of machine learning research, 12(Aug):2493–2537.
  11. Sohan De Sarkar, Fan Yang, and Arjun Mukherjee. 2018. Attending sentences to detect satirical fake news. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3371–3380. find
  12. Mohamed El Kourdi, Amine Bensaid, and Tajje-eddine Rachidi. 2004. Automatic Arabic document categorization based on the Naı̈ve Bayes algorithm. In proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, pages 51–58. find
  13. Kevin Fallon. (2017). Fooled by the onion: 9 most embarrassing fails. [Online; accessed 15-July-2019]. find
  14. Axel Gelfert. 2018. Fake news: A definition. Informal Logic, 38(1):84–117. ↗
  15. Bilal Ghanem, Jihen Karoui, Farah Benamara, Paolo Rosso, and Véronique Moriceau. 2020. Irony detection in a multilingual context. In European Conference on Information Retrieval, pages 141–149. Springer. find
  16. Jennifer Golbeck, Matthew Mauriello, Brooke Auxier, Keval H Bhanushali, Christopher Bonk, Mohamed Amine Bouzaghrane, Cody Buntain, Riya Chanduka, Paul Cheakalos, Jennine B Everett, et al. 2018. Fake news vs satire: A dataset and analysis. In Proceedings of the 10th ACM Conference on Web Science, pages 17–21. ACM. find
  17. H Paul Grice. 1991. Studies in the Way of Words. Harvard University Press. find
  18. Michael AK Halliday. 1988. On the ineffability of grammatical categories. perspective, page 27. John Benjamins. In Linguistics in a systemic Jihen Karoui, Farah Banamara Zitoune, and Veronique Moriceau. 2017. SOUKHRIA: Towards an Irony Detection System for Arabic in Social Media. Procedia Computer Science, 117:161–168. find
  19. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. ↗
  20. Emad Mohamed and Zeeshan Ali Sayyed. 2019. Segmenter and Orthography Standardazier (SOS) for Classical Arabic (CA). https://https://github.com/zeeshansayyed/ArabicSOS. ↗
  21. Saif M. Mohammad, Mohammad Salameh, and Svetlana Kiritchenko. 2016a. How translation alters sentiment. Journal of Artificial Intelligence Research, 55:95–130. ↗
  22. Saif M. Mohammad, Mohammad Salameh, and Svetlana Kiritchenko. 2016b. Sentiment Lexicons for Arabic Social Media. In Proceedings of 10th edition of the the Language Resources and Evaluation Conference (LREC), Portorož, Slovenia.
  23. Didrik Nielsen. 2016. Tree boosting with xgboost-why does xgboost win” every” machine learning competition? Master’s thesis, NTNU. find
  24. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. ↗
  25. James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: Liwc 2001. Mahway: Lawrence Erlbaum Associates, 71(2001):2001. find
  26. Tharindu Ranasinghe, Hadeel Saadany, Alistair Plum, Salim Mandhari, Emad Mohamed, Constantin Orasan, and Ruslan Mitkov. 2019. RGCL at IDAT: deep learning models for irony detection in Arabic language. Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, pages 416–425. find
  27. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2931–2937.
  28. Victoria Rubin, Niall Conroy, Yimin Chen, and Sarah Cornwell. 2016. Fake news or truth? using satirical cues to detect potentially misleading news. In Proceedings of the second workshop on computational approaches to deception detection, pages 7–17. find
  29. Sahar F Sabbeh and Sumaia Y Batwaah. 2018. Arabic news credibility on twitter: An enriched model using hybrid features. Journal of Theoretical & Applied Information Technology, 96(8). find
  30. Mohammad Salameh, Saif Mohammad, and Svetlana Kiritchenko. 2015. Sentiment after translation: A case-study on Arabic social media posts. In Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: Human language technologies, pages 767–777.
  31. Vlad Sandulescu and Mihai Chiru. 2016. Predicting the future relevance of research institutions-the winning solution of the kdd cup 2016. arXiv preprint arXiv:1609.02728. ↗
  32. L Torlay, Marcela Perrone-Bertolotti, Elizabeth Thomas, and Monica Baciu. 2017. Machine learning–xgboost analysis of language networks to classify patients with epilepsy. Brain informatics, 4(3):159–169. ↗
  33. Kristina Toutanova and Christopher D. Manning. 2000. Stanford log-linear part-of-speech tagger 3.9.2. find
  34. Fan Yang, Arjun Mukherjee, and Eduard Dragut. 2017. Satirical news detection and analysis using attention mechanism and linguistic features. arXiv preprint arXiv:1709.01189. ↗
  35. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems, pages 649–657. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/saadany-2020-fake-or-real-study-arabic/