Elamur Research
☰
home / papers / 2019 / Multilingual and Multi-Aspect Hate Speech Analysis

Multilingual and Multi-Aspect Hate Speech Analysis

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D19-1474.pdf

Summary

Most hate speech research handles one language and treats detection as a single, often binary, decision. This work broadens the problem in two directions. It covers three languages, and it describes each hateful post along several dimensions. Over several rounds of refining their search terms, the authors assembled about 13,000 tweets that might be derogatory, written in English, French, and Arabic. Annotators on Amazon Mechanical Turk then labeled five aspects of each tweet:

  • directness
  • hostility type, including a new "fearful out of ignorance" category
  • the discriminated attribute
  • the specific target group
  • the annotator's own emotional reaction

Each aspect became a prediction task. The authors used the multitask architecture of Ruder et al. (2017) with Babylon cross-lingual embeddings. They compared single- and multi-task training, and mono- and multilingual training, against majority and BOW logistic-regression baselines. Deep models generally beat BOW. Single-task monolingual training stayed best for binary directness. Pooling languages helped the data-sparse target-group task, and some task pairs gave small gains. The dataset and code are public, which supports comparative, culture-aware study of toxic language.

Key points

  • The corpus has roughly 13,000 tweets in English, French, and Arabic (MSA plus dialects). It is released as anonymized text rather than tweet IDs, and code is available at github.com/HKUST-KnowComp/MLMA_hate_speech.
  • The five annotated aspects are directness, hostility type (offensive, disrespectful, hateful, fearful, abusive, normal), target attribute, target group (16 groups), and annotator sentiment. The authors say this is the first dataset to record annotator reactions.
  • On directness, STSL achieved the strongest results, with English macro-F1 of 0.94 and average macro-F1 of 0.86.
  • Multilingual training improved target-group classification in all three languages. Joint training of target attribute and group raised Arabic target-group macro-F1 by 0.25.
  • Full multilingual multitask training (MTML) brought little overall gain, which the authors attribute to class imbalance and multilabel tasks. Pairing target attribute with hostility type added 0.03, 0.05, and 0.01 for English, French, and Arabic respectively.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

132 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 129 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1474

@inproceedings{ousidhoum2019multilingual,
  title = {{Multilingual and Multi-Aspect Hate Speech Analysis}},
  author = {Ousidhoum, Nedjma and Lin, Zizheng and Zhang, Hongming and Song, Yangqiu and Yeung, Dit-Yan},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1474},
  url = {https://doi.org/10.18653/v1/d19-1474}
}

References

32 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Buntain, Paul Cheakalos, Alicia A. Geller, Quint Gergory, Rajesh Kumar Gnanasekaran, Raja Rajan Gunasekaran, Kelly M. Hoffman, Jenny Hottle, Vichita Jienjitlert, Shivika Khare, Ryan Lau, Marianna J. Martindale, Shalmali Naik, Heather L. Nixon, Piyush Ramachandran, Kristine M. Rogers, Lisa Rogers, Me… find
  2. Nuha Albadi, Maram Kurdi, and Shivakant Mishra. 2018. Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere. In Proceedings of ASONAM. IEEE Computer Society, pages 69–76. find
  3. Isabelle Augenstein, Sebastian Ruder, and Anders Søgaard. 2018. Multi-task learning of pairwise sequence classification tasks over disparate label spaces. In Proceedings of NAACL. Association for Computational Linguistics, pages 1896–1906. find
  4. Aditya Bohra, Deepanshu Vijay, Vinay Singh, Syed Sarfaraz Akhtar, and Manish Shrivastava. 2018. A dataset of hindi-english code-mixed social media text for hate speech detection. In Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social… find
  5. Erik Cambria, Andrew Livingstone, and Amir Hussain. 2011. The hourglass of emotions. In COST 2102 Training School. Springer, volume 7403 of Lecture Notes in Computer Science, pages 144–157. find
  6. Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, and Athena Vakali. 2017. Mean birds: Detecting aggression and bullying on twitter. In Proceedings of WebSci ’17. ACM, pages 13–22. find
  7. Kazuma Hashimoto, caiming xiong, Yoshimasa Tsuruoka, and Richard Socher. 2017. A joint many-task model: Growing a neural network for multiple nlp tasks. In Proceedings of EMNLP. Association for Computational Linguistics, pages 1923–1933. find
  8. Cynthia Van Hee, Els Lefever, Ben Verhoeven, Julie Mennes, Bart Desmet, Guy De Pauw, Walter Daelemans, and Véronique Hoste. 2015. Detection and fine-grained classification of cyberbullying events. In Proceedings of RANLP. pages 672–680. find
  9. Filip Klubička and Raquel Fernandez. 2018. Examining a hate speech corpus for hate speech detection and popularity prediction. In Proceedings LREC. find
  10. Nane Kratzke. 2017. The \#BTW17 Twitter Dataset -Recorded Tweets of the Federal Election Campaigns of 2017 for the 19th German Bundestag. Data 2(4). find
  11. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12:2493–2537. find our card
  12. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2017. Unsupervised machine translation using monolingual corpora only. arXiv preprint arXiv:1711.00043 . find our card
  13. Thomas Davidson, Dana Warmsley, Michael W. Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of ICWSM. pages 512–515. find our card
  14. Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. Multi-task sequence to sequence learning. In International Conference on Learning Representations. find our card
  15. Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Y. Wang, and Elizabeth Belding. 2018. Hate lingo: A target-based linguistic analysis of hate speech in social media. In Proceedings of ICWSM. find
  16. Shervin Malmasi and Marcos Zampieri. 2018. Challenges in discriminating profanity from hate speech. J. Exp. Theor. Artif. Intell. 30(2):187–202. find
  17. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of twitter abusive behavior. In Proceedings ICWSM. pages 491–500. find
  18. Ryan Martin, Kelsey Ryan Coyier, Leah Vansistine, and Kelly Schroeder. 2012. Anger on the internet: The perceived value of rant-sites. Cyberpsychology, behavior and social networking 16. find
  19. Thiago Galery, Efstathios Charitos, and Ye Tian. 2018. Aggression identification and multi lingual word embeddings. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying. Association for Computational Linguistics, pages 74–79. find
  20. Jennifer Golbeck, Zahra Ashktorab, Rashad O. Banjo, Alexandra Berlinger, Siddharth Bhagwan, Cody Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan,… find our card
  21. Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of WWW ’16. International World Wide Web Conferences Steering Committee, pages 145–153. find
  22. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL Student Research Workshop. The Association for Computational Linguistics, pages 88–93. find
  23. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of EMNLP. Association for Computational Linguistics, pages 2799–2804. find our card
  24. Hao Peng, Sam Thomson, and Noah A. Smith. 2017. Deep multitask learning for semantic dependency parsing. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, pages 2037–2048. find
  25. Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. 2018. Hierarchical cvae for fine-grained hate speech classification. In Proceedings of EMNLP. Association for Computational Linguistics, pages 3550–3559. find
  26. Björn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, and Michael Wojatzki. 2017. Measuring the reliability of hate speech annotations: The case of the european refugee crisis. CoRR abs/1701.08118. — [arXiv:1701.08118](https://arxiv.org/abs/1701.08118) find
  27. Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, and Anders Søgaard. 2017. Sluice networks: Learning what to share between loosely related tasks. arXiv preprint arXiv:1705.08142 . — [arXiv:1705.08142](https://arxiv.org/abs/1705.08142) find
  28. Manuela Sanguinetti, Fabio Poletto, Cristina Bosco, Viviana Patti, and Marco Stranisci. 2018. An italian twitter corpus of hate speech against immigrants. In LREC. European Language Resources Association (ELRA). find
  29. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media. Association for Computational Linguistics, pages 1–10. find our card
  30. Samuel L. Smith, David H. P. Turban, Steven Hamblin, and Nils Y. Hammerla. 2017. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. CoRR abs/1702.03859. find our card
  31. Wiktor Soral, Micha Bilewicz, and Mikoaj Winiewski. 2017. Exposure to hate speech increases prejudice through desensitization. Aggressive Behavior 44. find
  32. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of the First Workshop on Abusive Language Online. Association for Computational Linguistics, pages 78–84. find our card