Elamur Research⌕
☰
home / papers / 2019 / A Benchmark Dataset for Learning to Intervene in Online Hate Speech

A Benchmark Dataset for Learning to Intervene in Online Hate Speech

Summary

Most NLP work on online hate speech stops at detection, and existing datasets usually label single posts with no surrounding discussion. Qian et al. propose a new task, generative hate speech intervention: given a conversation that contains hateful content, a system should produce a reply that pushes back. To support the task, they built two conversational datasets, one from Reddit and one from Gab. The authors found candidate hateful posts with hate keywords and then rebuilt the full comment threads around them. Mechanical Turk workers marked which posts were hateful and wrote short intervention replies, so the data also works for classification. The paper benchmarks detection classifiers and three response generators: Seq2Seq, VAE, and RL. All models do better on Gab, which is larger and more balanced. The RL model does not improve automatic scores, yet human judges rate its replies as more effective and more varied. Human-written replies remain clearly superior. The authors argue that BLEU-style metrics capture response quality poorly, which leaves substantial room for future work on counter-speech generation.

Key points

  • The two datasets are roughly 5k Reddit conversations (about 22k posts) and 12k Gab conversations (about 34k posts). Every post carries a hate/non-hate label, and the data is public on GitHub.
  • Reddit threads come from the 200 hottest submissions in each of ten subreddits often described as toxic, such as r/MGTOW and r/The Donald. Gab data covers all posts from October 2018. Threads longer than 20 comments were excluded, and replies were limited to 140 characters.
  • For detection, SVM and LR outperform CNN and RNN models that use randomly initialized embeddings.
  • On Gab, RL won human preference in 41.6% of cases for effectiveness and 40.0% for diversity. VAE tended to repeat stock replies about disability slurs.
  • A single generic reply used for every case still scores 4.2 BLEU, 20.4 ROUGE-L, and 18.2 METEOR on Gab. In comparisons with model outputs, human responses were chosen 70% or more of the time.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

111 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 108 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1482

@inproceedings{qian2019benchmark,
  title = {{A Benchmark Dataset for Learning to Intervene in Online Hate Speech}},
  author = {Qian, Jing and Bethke, Anna and Liu, Yinyin and Belding, Elizabeth and Wang, William Yang},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1482},
  url = {https://doi.org/10.18653/v1/d19-1482}
}

References

33 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. space. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, August 11-12, 2016, pages 10–21. find
  2. Pete Burnap and Matthew L Williams. 2016. Us and Them: Identifying Cyber Hate on Twitter across Multiple Protected Characteristics. EPJ Data Science, 5(1):11. find
  3. Pew Research Center. 2017. Online harassment 2017. find
  4. Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, and Athena Vakali. 2017. Mean birds: Detecting aggression and bullying on twitter. In Proceedings of the 2017 ACM on web science conference, pages 13–22. ACM. find
  5. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Lan… find
  6. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Eleventh International AAAI Conference on Web and Social Media. find our card
  7. Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Yang Wang, and Elizabeth Belding. 2018a. Hate lingo: A target-based linguistic analysis of hate speech in social media. In Twelfth International AAAI Conference on Web and Social Media. find
  8. Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 759–760. International World Wide Web Conferences Steering Committee. find
  9. Mai ElSherief, Shirin Nilizadeh, Dana Nguyen, Giovanni Vigna, and Elizabeth Belding. 2018b. Peer to peer hate: Hate speech instigators and their targets. In Twelfth International AAAI Conference on Web and Social Media. find
  10. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72. find our card
  11. Antigoni Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of twitter abusive behavior. In Twelfth International AAAI Confere… find
  12. Heiner Bielefeldt, Frank La Rue, and Githu Muigai. 2011. Ohchr expert workshops on the prohibition of incitement to national, racial or religious hatred. Expert workshop on the Americas. find
  13. Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. 2016. Generating sentences from a continuous Lei Gao, Alexis Kuppersmith, and Ruihong Huang. 2017. Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach.… find
  14. Jennifer Golbeck, Zahra Ashktorab, Rashad O Banjo, Alexandra Berlinger, Siddharth Bhagwan, Cody Buntain, Paul Cheakalos, Alicia A Geller, Quint Gergory, Rajesh Kumar Gnanasekaran, et al. 2017. A large labeled corpus for online harassment research. In Proceedings of the 2017 ACM on Web Science Confer… find
  15. George W. Kennedy III, Andrew W. McCollough, Edward Dixon, Alexie Bastidas, John Ryan, Chris Loo, and Saurav Sahay. 2017. Technology solutions to combat online harassment. In Proceedings of the first workshop on abusive language online, pages 73–77. find
  16. Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. 2018a. Hierarchical cvae for fine-grained hate speech classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3550–3559. find
  17. Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. 2018b. Leveraging intra-user and inter-user representation learning for automated hate speech detection. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan… find
  18. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1746–1751. find our card
  19. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence level training with recurrent neural networks. In 4th International Conference on Learning Representations, ICLR 2016. find our card
  20. Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114. find our card
  21. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10. find our card
  22. Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1192–1202… find our card
  23. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out. find
  24. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. find
  25. Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web, pages 145–153. International World Wide Web Conferences Steering Committee. find
  26. Abby Ohlheiser. 2016. Banned from twitter? this site promises you can say whatever you want. The Washington Post. find
  27. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. find our card
  28. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems. find
  29. Cynthia Van Hee, Els Lefever, Ben Verhoeven, Julie Mennes, Bart Desmet, Guy De Pauw, Walter Daelemans, and Véronique Hoste. 2015. Detection and Fine-grained Classification of Cyberbullying Events. In RANLP’15: International Conference Recent Advances in Natural Language Processing, pages 672–680. find
  30. William Warner and Julia Hirschberg. 2012. Detecting Hate Speech on the World Wide Web. In ACL’12: Proceedings of the 2nd Workshop on Language in Social Media, pages 19–26. Association for Computational Linguistics. find our card
  31. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88–93. find our card
  32. Haoti Zhong, Hao Li, Anna Cinzia Squicciarini, Sarah Michele Rajtmajer, Christopher Griffin, David J Miller, and Cornelia Caragea. 2016. Content-Driven Detection of Cyberbullying on the Instagram Social Network. In IJCAI’16: Proceedings of the 25th International Joint Conference on Artificial Intell… find
  33. Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In 6th International Conference on Learning Representations, ICLR 2018. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qian-2019-benchmark-dataset-learning-intervene-online/