Elamur Research⌕
☰
home / papers / 2026 / When Words Wear Masks: Detecting Malicious Intents and Hostile Impacts of Online Hate Speech

When Words Wear Masks: Detecting Malicious Intents and Hostile Impacts of Online Hate Speech

Paper · 2026 · Priyansh Singhal and Piyush Joshi

Summary

Most hate speech datasets assign a single label such as hate, offensive or toxic. This hides two different things: the reason a speaker posts hateful content and the damage that content can do. The authors address this with I2-Hate, an English dataset of 3,296 Twitter (X) posts that attack collective identities rather than individuals. Each post is annotated under two separate multi-label schemes: Intent (7 categories, partly based on Weberian action types) and Impact (8 categories of societal harm). Three trained annotators, who are mental health professionals, labelled the data with substantial agreement.

The authors fine-tuned 19 transformer models and prompted four frontier LLMs. Fine-tuned encoders reach about 0.90 macro F1 on Intent and about 0.82 on Impact, so harm is harder to predict than motivation. LLMs perform far worse, even with full label definitions and few-shot examples, mainly because they assign too many labels. The resource matters because it supports graduated, proportionate moderation instead of a simple remove-or-keep decision.

Key points

  • Dataset: I2-Hate contains 3,296 English posts from 2024–2025 with 7 Intent and 8 Impact labels.
    • Intent labels include Affective Aggression, Strategic Incitement and Threat & Intimidation.
    • Impact labels include Psychological Harm, Incitement to Violence and Stigmatization & Dehumanization.
  • Agreement and release: Fleiss' κ is 0.74 for Intent and 0.79 for Impact. The data is on Hugging Face (PS4Research/I2-Hate) with gated access, and the code is on GitHub.
  • Best fine-tuned models:
    • Intent: DeBERTa-v3-base scores 0.903 macro F1, with RoBERTa-base close at 0.9007.
    • Impact: BART-large leads with 0.8253 macro F1 and 0.5539 subset accuracy.
    • Large variants and hate-specific models (HateBERT, ToxicBERT) bring no consistent gains.
  • LLMs: Llama 4 Scout, ChatGPT-5, Gemini 2.5 Pro and Claude Sonnet 4.5 were tested in zero-shot and 3-shot settings.
    • The best result, Claude 3-shot, reaches only 0.6243 macro F1 on Intent and 0.5819 on Impact.
    • Subset accuracy stays between 0 and 0.08.
  • Limitations: The data covers only Twitter and only English. Experiments ran on dual T4 GPUs, which ruled out larger models and multi-task learning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksBias, fairness & safety · 517 works

Cite

Cite as Priyansh Singhal and Piyush Joshi. 2026. When Words Wear Masks: Detecting Malicious Intents and Hostile Impacts of Online Hate Speech. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-short.8

@inproceedings{singhal2026when,
  title = {{When Words Wear Masks: Detecting Malicious Intents and Hostile Impacts of Online Hate Speech}},
  author = {Priyansh Singhal and Piyush Joshi},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-short.8},
  url = {https://doi.org/10.18653/v1/2026.eacl-short.8}
}

References

69 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kwabena Odame Akomeah. 2023. Hate Speech Detection beyond plain Natural Language Processing. In Proceedings of the First Workshop on Foundational Models and Disinformation (FDIA@ESSIR 2023). find
  2. Anthropic. 2025. Introducing claude sonnet 4.5. https://www.anthropic.com/news/ claude-sonnet-4-5. ↗
  3. Francesco Barbieri, Jose Camacho-Collados, Leonardo Neves, and Luis Espinosa-Anke. 2020. Tweeteval: Unified benchmark and comparative evaluation for tweet classification. arXiv preprint arXiv:2010.12421.
  4. Caitlin Ring Carlson. 2020. Hate speech as a structural phenomenon. First Amendment Studies, 54(2):217– 224. find
  5. Matteo Cinelli, Andraž Pelicon, Igor Mozetič, Walter Quattrociocchi, Petra Kralj Novak, and Fabiana Zollo. 2021. Dynamics of online hate and misinformation. Scientific reports, 11(1):22083. find
  6. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555. find
  7. Bryn Alexander Coles and Melanie West. 2016. Trolling the trolls: Online forum users constructions of the nature and properties of trolling. Computers in Human Behavior, 60:233–244. find
  8. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. find
  9. Ismael Cortés. 2021. Hate speech, symbolic violence, and racial discrimination. antigypsyism: what responses for the next decade? Social Sciences, 10(10):360. find
  10. Rob Cover. 2023. Digital hostility, subjectivity and ethics: Theorising the disruption of identity in instances of mass online abuse and hate speech. Convergence, 29(2):308–321. find
  11. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media, volume 11, pages 512–515.
  12. Daman Deep Singh, Ramanuj Bhattacharjee, and Abhijnan Chakraborty. 2025. Rethinking hate speech detection on social media: Can llms replace traditional models? arXiv e-prints, pages arXiv–2506. find
  13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
  14. Arne Dreißigacker, Philipp Müller, Anna Isenhardt, and Jonas Schemmel. 2024. Online hate speech victimization: consequences for victims’ feelings of insecurity. Crime Science, 13(1):4. find
  15. Mai ElSherief, Caleb Ziems, and et al. 2021. Latent hatred: A benchmark for understanding implicit hate speech. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345–363. Association for Computational Linguistics.
  16. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. find
  17. Paula Fortuna, Juan Soler, and Leo Wanner. 2020. Toxic, hateful, offensive or abusive? what are we really classifying? an empirical analysis of hate speech datasets. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6786– 6794.
  18. Shannon Fyfe. 2017. Tracking hate speech acts as incitement to genocide in international criminal law. Leiden Journal of International Law, 30(2):523–548. find
  19. Lei Gao, Alexis Kuppersmith, and Ruihong Huang. 2017. Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach. arXiv preprint arXiv:1710.07394. find
  20. Eric Goldman. 2021. Content moderation remedies. Mich. Tech. L. Rev., 28:1. find
  21. Google DeepMind. 2025. Gemini 2.5 pro model card. Technical report. find
  22. Laura Hanu and Unitary team. 2020. Detoxify. Github. https://github.com/unitaryai/detoxify. ↗
  23. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. Preprint, arXiv:2111.09543.
  24. Tirza Leader, Brian Mullen, and Diana Rice. 2009. Complexity and valence in ethnophaulisms and exclusion of ethnic out-groups: What puts the" hate" into hate speech? Journal of Personality and Social Psychology, 96(1):170. find
  25. Jean Lee, Taejun Lim, and et al. 2022. K-MHaS: A multi-label hate speech detection dataset in Korean online news comment. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3530–3538. International Committee on Computational Linguistics. find
  26. Steven P Lee. 2010. Hate speech in the marketplace of ideas. In Freedom of Expression in a Diverse World, pages 13–25. Springer. find
  27. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. CoRR, abs/1910.13461.
  28. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
  29. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. find
  30. Maria Cristina Hinojosa Lee, Johan Braet, and Johan Springael. 2024. Performance metrics for multilabel emotion classification: comparing micro, macro, and weighted f1-scores. Applied Sciences, 14(21):9863. find
  31. Melanie Judge and Juan A Nel. 2018. Psychology and hate speech: A critical and restorative encounter. find
  32. Prashant Kapil and Asif Ekbal. 2024. A survey on combating hate speech through detection and prevention in english. In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 485–501.
  33. Kosisochukwu Madukwe, Xiaoying Gao, and Bing Xue. 2020. In data we trust: A critical analysis of hate speech detection datasets. In Proceedings of the fourth workshop on online abuse and harms, pages 150–161. find
  34. Swapnil Sanjaykumar Mane, Suman Kundu, and Rajesh Sharma. 2025. A survey on online aggression: Content detection and behavioral analysis on social media. ACM Computing Surveys, 57(7):1–36. find
  35. Jae Yeon Kim and Aniket Kesari. 2021. Misinformation and hate speech: The case of anti-asian hate speech during the covid-19 pandemic. Journal of Online Trust and Safety, 1(1). find
  36. Alexandria Marsters. 2019. When hate speech leads to hateful actions: A corpus and discourse analytic approach to linguistic threat assessment of hate speech. Georgetown University. find
  37. Monika Kopytowska and Fabienne Baider. 2017. From stereotypes and prejudice to verbal and physical violence: Hate speech in context. Lodz Papers in Pragmatics, 13(2):133–152. find
  38. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2020. Hatexplain: A benchmark dataset for explainable hate speech detection. arXiv preprint arXiv:2012.10289. find
  39. Marlene Kunst, Pablo Porten-Cheé, Martin Emmer, and Christiane Eilders. 2021. Do “good citizens” fight hate speech online? effects of solidarity citizenship norms on user responses to hate comments. Journal of Information Technology & Politics, 18(3):258–273. find
  40. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A lite BERT for self-supervised learning of language representations. CoRR, abs/1909.11942.
  41. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 14867–14875. find
  42. Meta AI. 2025. Llama 4 scout: Natively multimodal intelligence. https://ai.meta.com/blog/ llama-4-multimodal-intelligence/. ↗
  43. Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, and Grigorios Tsoumakas. 2022. Ethos: a multi-label hate speech detection dataset. Complex & Intelligent Systems, 8(6):4663–4678. find
  44. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108. find
  45. Jinseok Nam, Eneldo Loza Mencía, Hyunwoo J Kim, and Johannes Fürnkranz. 2017. Maximizing subset accuracy with recurrent neural networks in multi-label classification. Advances in neural information processing systems, 30. find
  46. Morgan Klaus Scheuerman, Jialun Aaron Jiang, Casey Fiesler, and Jed R Brubaker. 2021. A framework of severity for harmful content online. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2):1–33. find
  47. John A Naslund, Ameya Bondre, John Torous, and Kelly A Aschbrenner. 2020. Social media and mental health: benefits, risks, and opportunities for research and practice. Journal of technology in behavioral science, 5(3):245–257. find
  48. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the fifth international workshop on natural language processing for social media, pages 1–10.
  49. Nicolás Benjamín Ocampo, Ekaterina Sviridova, Elena Cabrio, and Serena Villata. 2023. An in-depth analysis of implicit and subtle hate speech messages. In EACL 2023-17th Conference of the European Chapter of the Association for Computational Linguistics, volume 2023, pages 1997–2013. Association for… find
  50. Sam Shleifer. 2020. DistilBART-CNN-6-6: Distilled BART Model for Summarization. https://huggingface.co/sshleifer/ distilbart-cnn-6-6. HuggingFace Model Hub. ↗
  51. Ruby O’Kane. 2021. Meta’s private speech governance and the role of the oversight board: Lessons from the board’s first decisions. Stan. Tech. L. Rev., 25:167. find
  52. Ebuka Okpala and Long Cheng. 2025. Large language model annotation bias in hate speech detection. In Proceedings of the International AAAI Conference on Web and Social Media, volume 19, pages 1389–1418. find
  53. OpenAI. 2025. Gpt-5 system card. Technical report. find
  54. Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019. Multilingual and multi-aspect hate speech analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processin…
  55. Wiktor Soral, Michał Bilewicz, and Mikołaj Winiewski. 2018. Exposure to hate speech increases prejudice through desensitization. Aggressive behavior, 44(2):136–146. find
  56. Kanae Takahashi, Kouji Yamamoto, Aya Kuchiba, and Tatsuki Koyama. 2022. Confidence interval for micro-averaged f 1 and macro-averaged f 1 scores. Applied Intelligence, 52(5):4961–4972. find
  57. Sentence Transformers. all-minilm-l6-v2. https://huggingface.co/sentence-transformers/ all-MiniLM-L6-v2. ↗
  58. Bertie Vidgen and Leon Derczynski. 2020. Directions in abusive language training data, a systematic review: Garbage in, garbage out. Plos one, 15(12):e0243300. find
  59. Luna Wang, Andrew Caines, and Alice Hutchings. 2025a. Web (er) of hate: A survey on how hate speech is typed. arXiv preprint arXiv:2506.16190. find
  60. Argha Ray and Joey George. 2021. Online hate and its routes to aggression: A research agenda. Social Impact and Information Systems. find
  61. Xinyu Wang, Sai Koneru, Pranav Narayanan Venkit, Brett Frischmann, and Sarah Rajtmajer. 2025b. The unappreciated role of intent in algorithmic moderation of abusive content on social media. Harvard Kennedy School Misinformation Review. find
  62. Pratik Sachdeva, Renata Barreto, Geoff Bacon, Alexander Sahn, Claudia Von Vacano, and Chris Kennedy. 2022. The measuring hate speech corpus: Leveraging rasch measurement theory for data perspectivism. In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP@ LREC2022, pages 83–94. find
  63. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93, San Diego, California. Association for Computational Linguistics.
  64. Hamidreza Saffari, Mohammadamin Shafiei, Hezhao Zhang, Lasana Harris, and Nafise Sadat Moosavi. 2024. Beyond hate speech: Nlp’s challenges and opportunities in uncovering dehumanizing language. arXiv preprint arXiv:2402.13818. find
  65. Richard Ashby Wilson and Molly K Land. 2020. Hate speech on social media: Content moderation in context. Conn. L. Rev., 52:1029. find
  66. Koustuv Saha, Eshwar Chandrasekharan, and Munmun De Choudhury. 2019. Prevalence and psychological effects of hateful speech in online college communities. In Proceedings of the 10th ACM conference on web science, pages 255–264. find
  67. Guoqiang Wu and Jun Zhu. 2020. Multi-label classification: do hamming loss and subset accuracy really conflict with each other? Advances in Neural Information Processing Systems, 33:3130–3140. find
  68. YituTech. 2021. Convbert-base. Github. https://huggingface.co/YituTech/conv-bert-base. ↗
  69. Zhilu Zhang and Mert Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/singhal-2026-when-words-wear-masks-detecting/