Elamur Research⌕
☰
home / papers / 2020 / Contextualizing Hate Speech Classifiers with Post-hoc Explanation

Contextualizing Hate Speech Classifiers with Post-hoc Explanation

Summary

Hate speech classifiers trained on skewed data often treat group identifiers such as "black", "gay" or "muslim" as signs of hate on their own. This produces false positives on harmless text that mentions these groups. Kennedy et al. examine this bias in a bag-of-words logistic regression and a fine-tuned BERT model. They use the Gab Hate Corpus (GHC) and a Stormfront sentence corpus. To diagnose the bias, they apply the Sampling and Occlusion (SOC) post-hoc explanation method, which shows that identifiers rank among the most influential features. They then add a training penalty on the SOC importance of 25 curated identity terms, which pushes the model to rely on the surrounding context. They test it on a hate-free set of New York Times articles that contain those identifiers. Compared with plain BERT, the regularized model makes far fewer false positives there. Its in-domain F1 stays roughly the same or improves. The method works inside Transformer fine-tuning and needs no data augmentation, which makes it a practical way to reduce lexical bias.

Key points

  • Data:
    • GHC has 27,655 Gab posts in total. The training split (GHC train) has 24,353 posts, 2,027 of them hate, and the test split (GHC test) has 1,586 posts, 372 of them hate.
    • Stormfront splits are 7,896 train, 979 validation and 1,998 test sentences.
    • The NYT evaluation set has 12,500 documents balanced across 25 identity terms.
  • Diagnosis: Removing identifiers from a bag-of-words model raises NYT accuracy but lowers hate-detection F1. This shows that simply ignoring identifiers is not a solution.
  • GHC results (BERT → BERT+SOC):
    • With α=0.1, F1 rose from 67.91 to 69.52 and NYT accuracy from 77.79 to 83.16.
    • With α=1.0, NYT accuracy reached 90.06 and F1 was 66.67.
  • Comparisons and Stormfront:
    • Word removal (BERT+WR) also cut false positives on the GHC setup, but it lowered recall and F1 (63.44).
    • On Stormfront, NYT accuracy rose from 92.29 to as high as 95.40, with F1 roughly unchanged (about 55).
    • All BERT results are averages over 10 runs.
  • Effect on word importance: After regularization, identity terms lose importance under SOC. Context words like "poisoned" and "blamed" move up in the importance ranking.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

76 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 70 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.483

@inproceedings{kennedy2020contextualizing,
  title = {{Contextualizing Hate Speech Classifiers with Post-hoc Explanation}},
  author = {Kennedy, Brendan and Jin, Xisen and Davani, Aida Mostafazadeh and Dehghani, Morteza and Ren, Xiang},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.483},
  url = {https://doi.org/10.18653/v1/2020.acl-main.483}
}

References

41 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Andrew Anthony. 2016. Inside the hate-filled echo chamber of racism and conspiracy theories. The guardian, 18. find
  2. Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of the 13th International Wor… find our card
  3. Thor Benson. 2016. Inside the twitter for racists: Gab the site where milo yiannopoulos goes to troll now. find
  4. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems, pages 4349–4357. find our card
  5. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Eleventh international AAAI conference on web and social media. find our card
  6. Joseph Hoover, Mohammad Atari, Aida Mostafazadeh Davani, Brendan Kennedy, Gwenyth Portillo-Wightman, Leigh Yeh, Drew Kogon, and Morteza Dehghani. 2019. Bound in hatred: The role of group-based morality in acts of hate. PsyArxiv Preprint 10.31234/osf.io/359me. — [doi:10.31234/osf.io/359me](https://doi.org/10.31234/osf.io/359me) find
  7. Xisen Jin, Zhongyu Wei, Junyi Du, Xiangyang Xue, and Xiang Ren. 2020. Towards hierarchical importance attribution: Explaining compositional semantics for neural sequence models. In International Conference on Learning Representations. find
  8. Brendan Kennedy, Mohammad Atari, Aida M Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs Jr., Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez, Joe Hoover, Aida Azatian, Gabriel Cardenas, Alyzeh Hussain, Austin Lara, Adam Omary, Christina Park, Xin Wang, Clarisa Wijaya, Yong Zhang,… find
  9. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations. find
  10. Fabio Del Vigna12, Andrea Cimino23, Felice DellOrletta, Marinella Petrocchi, and Maurizio Tesconi. 2017. Hate me, hate me not: Hate speech detection on facebook. In Proceedings of the First Italian Conference on Cybersecurity (ITASEC17), pages 86–95. find
  11. Lucas Lima, Julio CS Reis, Philipe Melo, Fabricio Murai, Leandro Araujo, Pantelis Vikatos, and Fabricio Benevenuto. 2018. Inside the right-leaning echo chambers: Characterizing gab, an unmoderated social system. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mi… find
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  13. Sean MacAvaney, Hao-Ren Yao, Eugene Yang, Katina Russell, Nazli Goharian, and Ophir Frieder. 2019. Hate speech detection: Challenges and solutions. PloS one, 14(8). find
  14. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67–73. ACM. find
  15. Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages. In Proceedings of the 11th Forum for Information Retrieval Evaluat… find
  16. Iginio Gagliardone, Danit Gal, Thiago Alves, and Gabriela Martinez. 2015. Countering online hate speech. Unesco Publishing. find
  17. Katharine Gelber and Luke McNamara. 2016. Evidencing the harms of hate speech. Social Identities, 22(3):324–341. find
  18. Ona de Gibert, Naiara Perez, Aitor Garcı́a Pablos, and Montse Cuadros. 2018. Hate speech dataset from a white supremacy forum. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 11–20. find our card
  19. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2019. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):93. find
  20. Priscilla Marie Meddaugh and Jack Kay. 2009. Hate speech or “reasonable racism?” the other in stormfront. Journal of Mass Media Ethics, 24(4):251– 268. find
  21. Shruthi Mohan, Apala Guha, Michael Harris, Fred Popowich, Ashley Schuster, and Chris Priebe. 2017. The impact of toxic language on the health of reddit communities. In Canadian Conference on Artificial Intelligence, pages 51–56. Springer. find
  22. Mainack Mondal, Leandro Araújo Silva, and Fabrı́cio Benevenuto. 2017. A measurement study of hate speech in social media. In Proceedings of the 28th ACM Conference on Hypertext and Social Media, pages 85–94. ACM. find
  23. Aida Mostafazadeh Davani, Mohammad Atari, Brendan Kennedy, Shreya Havaldar, and Morteza Dehghani. 2020. Hatred is in the eye of the annotator: Hate speech classifiers learn human-like social stereotypes (in press). In 31st Annual Conference of the Cognitive Science Society (CogSci). find
  24. W. James Murdoch, Peter J. Liu, and Bin Yu. 2018. Beyond word importance: Contextual decomposition to extract interactions from LSTMs. In International Conference on Learning Representations. find
  25. Timothy Niven and Hung-Yu Kao. 2019. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4658–4664. find
  26. Alexandra Olteanu, Carlos Castillo, Jeremy Boy, and Kush R Varshney. 2018. The effect of extremist violence on hateful speech online. In Twelfth International AAAI Conference on Web and Social Media. find
  27. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799–2804. find our card
  28. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. Open AI Blog. find
  29. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144. ACM. find our card
  30. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1668–1678. find our card
  31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
  32. Jeremy Waldron. 2012. The harm in hate speech. Harvard University Press. find
  33. William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In Proceedings of the second workshop on language in social media, pages 19–26. Association for Computational Linguistics. find our card
  34. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of the First Workshop on Abusive Language Online, pages 78–84. find our card
  35. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88–93. find our card
  36. Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of abusive language: the problem of biased datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short… find our card
  37. Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, pages 1391–1399. International World Wide Web Conferences Steering Committee. find
  38. Savvas Zannettou, Barry Bradlyn, Emiliano De Cristofaro, Haewoon Kwak, Michael Sirivianos, Gianluca Stringini, and Jeremy Blackburn. 2018. What is gab: A bastion of free speech or an alt-right echo chamber. In Companion Proceedings of the The Web Conference 2018, pages 1007–1014. International World… find
  39. Leandro Silva, Mainack Mondal, Denzil Correa, Fabrı́cio Benevenuto, and Ingmar Weber. 2016. Analyzing the targets of hate in online social media. In Tenth International AAAI Conference on Web and Social Media. find
  40. Chandan Singh, W. James Murdoch, and Bin Yu. 2019. Hierarchical interpretations for neural network predictions. In International Conference on Learning Representations. find
  41. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3319–3328. JMLR. org. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kennedy-2020-contextualizing-hate-speech-classifiers-post/