Elamur Research⌕
☰
home / papers / 2021 / Detecting Harmful Memes and Their Targets

Detecting Harmful Memes and Their Targets

Paper · 2021 · Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee et al.

Summary

The paper argues that harmful memes are a broader category than hateful or offensive ones. A meme can damage a person's or group's reputation without using insulting language. The authors also note that the victim of a harmful meme deserves attention of its own. They define two tasks. The first rates a meme's harmfulness on three levels. The second asks whether a harmful meme is aimed at one specific person, at society broadly, at a particular community, or at an organization.

To support both tasks, they built HarMeme, a corpus of 3,544 real COVID-19 memes gathered from the web and labeled in two annotation stages. They then benchmarked ten unimodal (text-only and image-only) and multimodal models. Models that fuse text and image signals did best, with V-BERT the strongest. Even so, every model stayed well below human judges.

A LIME analysis showed that V-BERT partly relies on shortcuts, such as the mere presence of a particular face. This suggests current off-the-shelf models do not really grasp meme semantics. The work matters because it supplies the first benchmark for harm and target detection in memes, which content moderation research needs.

Key points

  • HarMeme contains 3,544 COVID-19 memes. Each has an intensity label (harmless, partially harmful, very harmful), and harmful memes also carry a target label (individual, organization, community, society/general public/other).
  • Unlike the synthetic memes in Facebook's Hateful Memes Challenge, HarMeme uses real memes posted online.
  • Ten unimodal and multimodal baselines were tested. V-BERT (pre-trained on COCO) was best and beat ViLBERT (CC) by about 8% F1 and roughly 0.3 MMAE.
  • Separate expert evaluators reached roughly 86–91% accuracy on the test set, far higher than V-BERT.
  • LIME showed V-BERT over-associating Donald Trump's image with the very harmful class. The dataset and baseline code are available for research use only at github.com/di-dimitrov/harmeme.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hamad bin Khalifa UniversityDoha · Qatar71 works · 12,942 citations🎓Sofia University "St. Kliment Ohridski"Sofia · Bulgaria10 works · 2,341 citations🎓Indraprastha Institute of Information Technology DelhiDelhi · India7 works · 1,233 citations🏢WiproBengaluru · India4 works · 883 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Social Sciences · 744 works Sociology and Political Science · 221 works

🧭 Topics 1 Bias, fairness & safety · 209 works

Cite

Cite as Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov and Tanmoy Chakraborty. 2021. Detecting Harmful Memes and Their Targets. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-acl.246

@inproceedings{pramanick2021detecting,
  title = {{Detecting Harmful Memes and Their Targets}},
  author = {Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov and Tanmoy Chakraborty},
  booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.246},
  url = {https://doi.org/10.18653/v1/2021.findings-acl.246}
}

References

61 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, and Preslav Nakov. 2021. A survey on multimodal disinformation detection. arXiv 2103.12541. find
  2. Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2009. Evaluation Measures for Ordinal Regression. In Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications, ISDA ’09, pages 283–287, Pisa, Italy. find
  3. Victoria Bobicev and Marina Sokolova. 2017. Inter-annotator agreement in sentiment analysis: Machine learning perspective. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP ’17, pages 97–102, Varna, Bulgaria. find
  4. Lisa Bonheme and Marek Grzes. 2020. SESAM at SemEval-2020 task 8: Investigating the relationship between image and text in sentiment analysis of memes. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 804– 816, Barcelona, Spain. find
  5. Darren Crovitz and Clarice Moran. 2020. Analyzing Disruptive Memes in an Age of International Interference. The English Journal, 109(4):62–69. find
  6. Abhishek Das, Japsimar Singh Wahi, and Siyao Li. 2020. Detecting hate speech in multi-modal memes. arXiv 2012.14891. — [arXiv:2012.14891](https://arxiv.org/abs/2012.14891) find
  7. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’09, pages 248–255, Miami, FL, USA. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… our card
  9. Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021a. Detecting propaganda techniques in memes. In Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics… find
  10. Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021b. find
  11. SemEval-2021 Task 6: Detection of persuasion techniques in texts and images. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’21. our card
  12. Marc J. Dupuis and Andrew Williams. 2019. The spread of disinformation on the Web: An examination of memes on social networking. In Proceedings of the 2019 IEEE SmartWorld, Ubiquitous Intelligence Computing, Advanced Trusted Computing, Scalable Computing Communications, Cloud Big Data Computing, Int… find
  13. Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. 2020. Large-scale adversarial training for vision-and-language representation learning. In Proceedings of the 34th Conference on Neural Information Processing Systems, NeurIPS ’20, pages 1–15, Vancouver, Canada. find
  14. Sunil Gundapu and Radhika Mamidi. 2020. Gunda-pusunil at SemEval-2020 task 8: Multimodal Memotion Analysis. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 1112–1119, Barcelona, Spain. find
  15. Yingmei Guo, Jinfa Huang, Yanlong Dong, and Mingxing Xu. 2020. Guoym at SemEval-2020 task 8: Ensemble-based classification of visuo-lingual metaphor in memes. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 1120–1125, Barcelona, Spain. find
  16. Alan Hawkes. 1971. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58:83–90. find
  17. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’16, pages 770–778, Las Vegas, NV, USA. find
  18. Anthony Hu and Seth Flaxman. 2018. Multimodal sentiment analysis to explore the structure of emotions. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACMKDD ’18, pages 350–358, London, UK. find
  19. Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’17, pages 2261–2269, Honolulu, HI, USA. find
  20. Kimmo Karkkainen and Jungseock Joo. 2021. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, WACV ’21, pages 1548–1558. find
  21. Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, and Davide Testuggine. 2019. Supervised multimodal bitransformers for classifying images and text. arXiv 1909.02950. find
  22. Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. The hateful memes challenge: Detecting hate speech in multimodal memes. In Advances in Neural Information Processing Systems, volume 33 of NeurIPS ’20, pages 2611–2624, Vancouver… our card
  23. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR ’14, San Diego, CA, USA. find
  24. Jure Leskovec, Lars Backstrom, and Jon Kleinberg. 2009. Meme-tracking and the dynamics of the news cycle. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’09, page 497–506, New York, USA. find
  25. Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. VisualBERT: A simple and performant baseline for vision and language. arxiv 1908.03557. our card
  26. Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, and Jianfeng Gao. 2020. Oscar: Object-semantics aligned pre-training for vision-language tasks. In Proceedings of the 16th European Computer Vision Conference, ECCV ’20, pa… our card
  27. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In Proceedings of the European Conference on Computer Vision, ECCV ’14, pages 740–755, Zurich, Switzerland. our card
  28. Chen Ling, Ihab AbuHilal, Jeremy Blackburn, Emiliano De Cristofaro, Savvas Zannettou, and Gianluca Stringhini. 2021. Dissecting the meme magic: Understanding indicators of virality in image memes. In Proceedings of the 24th ACM Conference on Computer-Supported Cooperative Work and Social Computing,… find
  29. Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, and Helen Yannakoudakis. 2020. A multimodal framework for the detection of hateful memes. arXiv 2012.12871. — [arXiv:2012.12871](https://arxiv.org/abs/2012.12871) find
  30. Zehao Liu, Emmanuel Osei-Brefo, Siyuan Chen, and Huizhi Liang. 2020. UoR at SemEval-2020 task 8: Gaussian mixture modelling (GMM) based sampling approach for multi-modal memotion analysis. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 1201–1207, Barcelona, Spai… find
  31. Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Proceedings of the Conference on Neural Information Processing Systems, NeurIPS ’19, pages 13–23, Vancouver, Canada. our card
  32. Alexandros Mittos, Savvas Zannettou, Jeremy Blackburn, and Emiliano De Cristofaro. 2020. “And we will fight for our race!” A measurement study of genetic testing conversations on Reddit and 4chan. In Proceedings of the Fourteenth International AAAI Conference on Web and Social Media, ICWSM ’20, page… find
  33. Terufumi Morishita, Gaku Morio, Shota Horiguchi, Hiroaki Ozaki, and Toshinori Miyoshi. 2020. Hitachi at SemEval-2020 task 8: Simple but effective modality ensemble for meme emotion recognition. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 1126–1134, Barcelona,… find
  34. Niklas Muennighoff. 2020. Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes. arxiv 2012.07788. — [arXiv:2012.07788](https://arxiv.org/abs/2012.07788) find
  35. Gretel Liz De la Peña Sarracén, Paolo Rosso, and Anastasia Giachanou. 2020. PRHLT-UPV at SemEval-2020 task 8: Study of multimodal techniques for memes analysis. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 908–915, Barcelona, Spain. find
  36. Shraman Pramanick, Md Shad Akhtar, and Tanmoy Chakraborty. 2021. Exercise? I thought you said ‘extra fries’: Leveraging sentence demarcations and multi-hop attention for meme affect analysis. proceedings of the Fifteenth International AAAI Conference on Web and Social Media. find
  37. Julio C. S. Reis, Philipe Melo, Kiran Garimella, Jussara M. Almeida, Dean Eckles, and Fabrı́cio Benevenuto. 2020. A dataset of fact-checked images shared on WhatsApp during the Brazilian and Indian elections. In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM ’20, pag… find
  38. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 1135–1144, San Francisco, CA, USA. our card
  39. Benet Oriol Sabat, Cristian Canton-Ferrer, and Xavier Giró-i-Nieto. 2019. Hate speech in pixels: Detection of offensive memes towards automatic moderation. arXiv 1910.02334. — [arXiv:1910.02334](https://arxiv.org/abs/1910.02334) find
  40. Vlad Sandulescu. 2020. Detecting hateful memes using a multimodal deep ensemble. arXiv 2012.13235. — [arXiv:2012.13235](https://arxiv.org/abs/2012.13235) find
  41. Chhavi Sharma, Deepesh Bhageria, William Scott, Srinivas PYKL, Amitava Das, Tanmoy Chakraborty, Viswanath Pulabaigari, and Björn Gambäck. 2020a. SemEval-2020 task 8: Memotion analysis- the visuo-lingual metaphor! In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages… our card
  42. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Annual Conference on Neural Information Processing Systems, NeurIPS ’17, pages 5998–6008, Long Beach, CA, USA. our card
  43. Mayukh Sharma, Ilanthenral Kandasamy, and W.b. Vasantha. 2020b. Memebusters at SemEval-2020 task 8: Feature fusion model for sentiment analysis on memes using transfer learning. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 1163–1171, Barcelona, Spain. find
  44. Yuping Wang, Fatemeh Tahmasbi, Jeremy Blackburn, Barry Bradlyn, Emiliano De Cristofaro, David Magerman, Savvas Zannettou, and Gianluca Stringhini. 2020. Understanding the use of fauxtography on social media. arXiv 2009.11792. — [arXiv:2009.11792](https://arxiv.org/abs/2009.11792) find
  45. Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL ’18, pages 2556–2565, Melbourne, Aust… our card
  46. Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations, ICLR ’15, San Diego, CA, USA. our card
  47. Amanpreet Singh, Vedanuj Goswami, and Devi Parikh. 2020. Are we pretraining it right? Digging deeper into visio-linguistic pretraining. arXiv 2004.08744. — [arXiv:2004.08744](https://arxiv.org/abs/2004.08744) find
  48. Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL ’19, pages 3645–3650, Florence, Italy. our card
  49. Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2020. VL-BERT: pre-training of generic visual-linguistic representations. In Proceedings of the 8th International Conference on Learning Representations, ICLR ’20, Addis Ababa, Ethiopia. our card
  50. Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, and Paul Buitelaar. 2020. Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text. In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, LREC-TRAC ’20, pages 32–41, Marseille, Fra… our card
  51. Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’17, pages 1492– 1500, Honolulu, HI, USA. find
  52. Fei Yu, Jiji Tang, Weichong Yin, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2021. ERNIE-ViL: Knowledge enhanced vision-language representations through scene graph. In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI ’21, pages 3208–3216. our card
  53. Li Yuan, Jin Wang, and Xuejie Zhang. 2020. YNU-HPCC at SemEval-2020 task 8: Using a parallel-channel model for memotion analysis. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval ’20, pages 916–921, Barcelona, Spain. find
  54. Savvas Zannettou, Tristan Caulfield, Barry Bradlyn, Emiliano De Cristofaro, Gianluca Stringhini, and Jeremy Blackburn. 2020a. Characterizing the use of images in state-sponsored information warfare operations by Russian trolls on Twitter. In Proceedings of the Fourteenth International AAAI Conferenc… find
  55. Savvas Zannettou, Joel Finkelstein, Barry Bradlyn, and Jeremy Blackburn. 2020b. A quantitative approach to understanding online antisemitism. In Proceedings of the Fourteenth International AAAI Conference on Web and Social Media, ICWSM ’20, pages 786–797, Atlanta, GA, USA. find
  56. Xiayu Zhong. 2020. Classification of multimodal hate speech – the winning solution of hateful memes challenge. arXiv 2012.01002. — [arXiv:2012.01002](https://arxiv.org/abs/2012.01002) find
  57. Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason Corso, and Jianfeng Gao. 2020. Unified vision-language pre-training for image captioning and VQA. Proceedings of the AAAI Conference on Artificial Intelligence, pages 13041–13049. find
  58. Hao Tan and Mohit Bansal. 2019. LXMERT: Learning cross-modality encoder representations from transformers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP ’17, pages 5100… our card
  59. Yi Zhou and Zhenhao Chen. 2020. Multimodal learning for hateful memes detection. arXiv 2011.12870. — [arXiv:2011.12870](https://arxiv.org/abs/2011.12870) find
  60. Chi Thang Duong, Remi Lebret, and Karl Aberer. 2017. Multimodal classification for analysing social media. arXiv 1708.02099. — [arXiv:1708.02099](https://arxiv.org/abs/1708.02099) find
  61. Ron Zhu. 2020. Enhance multimodal transformer with external label and in-domain pretrain: Hateful meme challenge winning solution. arXiv 2012.08290. — [arXiv:2012.08290](https://arxiv.org/abs/2012.08290) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pramanick-2021-detecting-harmful-memes-their-targets/