Elamur Research⌕
☰
home / papers / 2020 / Toxicity Detection: Does Context Really Matter?

Toxicity Detection: Does Context Really Matter?

Summary

Most toxicity datasets and classifiers judge each comment in isolation, even though a reply's meaning often depends on what it answers. The authors examine this assumption with Wikipedia Talk Pages conversations, where context means only the parent comment and the discussion title. They ask two questions: how often context changes human toxicity judgements, and whether context-aware classifiers detect toxicity better.

For the first question, they ran an AB-style crowdsourcing study on 250 comments. One annotator group saw context and the other did not. Context changed the label for a small but statistically significant share of comments, and it could make a post seem either more or less toxic.

For the second question, they built a larger corpus of 20k comments, half labelled with context and half without. They trained BILSTM and BERT models with several ways of using the parent comment, and also tested PERSPECTIVE and BERT-CCTK. No approach gave a consistent or significant gain from context. The likely reason is that context-sensitive comments are rare. The authors argue for larger datasets annotated in context and release their data and code.

Key points

  • CAT-SMALL (250 comments): 5.2% of comments received opposite labels depending on whether context was shown. Context increased perceived toxicity in about 3.6% of cases and decreased it in about 1.6%.
  • CAT-LARGE (20k comments): CAT-LARGE-N holds 10k comments labelled without context, and CAT-LARGE-C holds 10k labelled with context. All models were tested on held-out in-context labels, using five-fold Monte Carlo cross-validation and ROC AUC.
  • Context-aware variants tested: CA-BILSTM-BILSTM, CA-BILSTM-BERT, CA-SEP-BERT (parent and target joined with [SEP]) and concatenation-based CA-CONC models. Training on in-context labels (@C) versus out-of-context labels (@N) also made no consistent difference.
  • Best scores: The externally trained systems performed best. PERSPECTIVE scored 79.27 and CA-CONC-PERSPECTIVE 81.89. BERT-CCTK scored 78.08 and CA-CONC-BERT-CCTK 81.69. These differences were not statistically significant.
  • Limitations and resources: Context covered only the previous post and the title, and the comments were randomly sampled. Data and code are available at github.com/ipavlopoulos/context_toxicity.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 78 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.396

@inproceedings{pavlopoulos2020toxicity,
  title = {{Toxicity Detection: Does Context Really Matter?}},
  author = {Pavlopoulos, John and Sorensen, Jeffrey and Dixon, Lucas and Thain, Nithum and Androutsopoulos, Ion},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.396},
  url = {https://doi.org/10.18653/v1/2020.acl-main.396}
}

References

36 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. B. van Aken, J. Risch, R. Krestel, and A. Löser. 2018. Challenges for toxic comment classification: An in-depth error analysis. In 2nd Workshop on Abusive Language Online, pages 33–42, Brussels, Belgium. find
  2. D. M. Blei, A. Y Ng, and M. I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research, 3(Jan):993–1022. find our card
  3. D. Borkan, L. Dixon, J. Sorensen, N. Thain, and L. Vasserman. 2019. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion Proceedings of The 2019 World Wide Web Conference, WWW ’19, page 491–500. Association for Computing Machinery. find
  4. The discussion title was used only by the human annotators that examined context, not by context-aware systems, which considered only the parent comment. find
  5. T. Davidson, D. Warmsley, M. Macy, and I. Weber. 2017. Automated hate speech detection and the problem of offensive language. In ICWSM, pages 512–515, Montreal, Canada. find our card
  6. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, page 4171–4186, Minneapolis,MN,USA. find
  7. N. Djuric, J. Zhou, R. Morris, M. Grbovic, V. Radosavljevic, and N. Bhamidipati. 2015. Hate speech detection with comment embeddings. In ICWWW, pages 29–30. find
  8. R. Dror, G. Baumer, S. Shlomov, and R. Reichart. 2018. The hitchhiker’s guide to testing statistical significance in natural language processing. In ACL, Melbourne, Australia. find
  9. M. ElSherief, V. Kulkarni, D. Nguyen, W. Y. Wang, and E. Belding. 2018. Hate lingo: A target-based linguistic analysis of hate speech in social media. arXiv preprint. find
  10. B. Gambäck and U. K. Sikdar. 2017. Using convolutional neural networks to classify hate-speech. In 1st Workshop on Abusive Language Online, pages 85–90, Vancouver, Canada. find our card
  11. L. Gao and R. Huang. 2017. Detecting online hate speech using context aware models. In RANLP, pages 260–266. find
  12. Kyle Gorman and Steven Bedrick. 2019. We need to talk about standard splits. In ACL, pages 2786– 2791. find
  13. S. Hochreiter and J. Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. find
  14. H. Hosseini, S. Kannan, B. Zhang, and R. Poovendran. 2017. Deceiving Google’s perspective api built for detecting toxic comments. arXiv preprint. find
  15. Y. Hua, C. Danescu-Niculescu-Mizil, D. Taraborelli, N. Thain, J. Sorensen, and L. Dixon. 2018. Wikiconv: A corpus of the complete conversational history of a large online collaborative community. arXiv preprint. find
  16. R. Kumar, A. K. Ojha, S. Malmasi, and M. Zampieri. 2018. Benchmarking aggression identification in social media. In TRAC, Santa Fe, USA. find our card
  17. S. Malmasi and M. Zampieri. 2017. Detecting hate speech in social media. In RANLP, pages 467–472. find
  18. T. Mikolov and G. Zweig. 2012. Context dependent recurrent neural network language model. In 2012 IEEE Spoken Language Technology Workshop (SLT), pages 234–239. IEEE. find
  19. H. Mubarak, K. Darwish, and W. Magdy. 2017. Abusive language detection on arabic social media. In 1st Abusive Language Workshop, pages 52–56, Vancouver, Canada. find our card
  20. C. Nobata, J. Tetreault, A. Thomas, Y. Mehdad, and Y. Chang. 2016. Abusive language detection in online user content. In ICWWW, pages 145–153. find
  21. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In ACL, pages 311–318. find our card
  22. J. H. Park and P. Fung. 2017. One-step and two-step classification for abusive language detection on twitter. In 1st Workshop on Abusive Language Online, pages 41–45. find our card
  23. J. Pavlopoulos, P. Malakasiotis, and I. Androutsopoulos. 2017a. Deep learning for user comment moderation. In 1st Workshop on Abusive Language Online, pages 25–35. find
  24. J. Pavlopoulos, P. Malakasiotis, and I. Androutsopoulos. 2017b. Deeper attention to abusive user content moderation. In EMNLP, pages 1125–1135, Copeng-hagen, Denmark. find
  25. L. Posch, A. Bleier, F. Flöck, and M. Strohmaier. 2018. Characterizing the global crowd workforce: A cross-country comparison of crowdworker demographics. arXiv preprint. find
  26. Y. Ren, Y. Zhang, M. Zhang, and D. Ji. 2016. Context-sensitive twitter sentiment classification using neural network. In 30th AAAI Conference on Artificial Intelligence. find
  27. B. Ross, M. Rist, G. Carbonell, B. Cabrera, N. Kurowsky, and M. Wojatzki. 2016. Measuring the reliability of hate speech annotations: The case of the european refugee crisis. In NLP4CMC, Bochum, Germany. find
  28. M. D. Smucker, J. Allan, and B. Carterette. 2007. A comparison of statistical significance tests for information retrieval evaluation. In CIKM, pages 623– 632. ACM. find
  29. A. Sordoni, M. Galley, M. Auli, C. Brockett, Y. Ji, M. Mitchell, J.-Y. Nie, J. Gao, and B. Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. arXiv preprint. find our card
  30. Z. Waseem, T. Davidson, D. Warmsley, and I. Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. In 1st Workshop on Abusive Langauge Online, Vancouver, Canada. find our card
  31. Z. Waseem and D. Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In NAACL SRW, pages 88–93, San Diego, California. find our card
  32. M. Wiegand, M. Siegel, and J. Ruppenhofer. 2018. Overview of the germeval 2018 shared task on the identification of offensive language. In Proceedings of GermEval. find
  33. E. Wulczyn, N. Thain, and L. Dixon. 2017. Ex machina: Personal attacks seen at scale. In ICWWW, pages 1391–1399. find
  34. M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, and R. Kumar. 2019a. Predicting the Type and Target of Offensive Posts in Social Media. In NAACL. find our card
  35. M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, and R. Kumar. 2019b. Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). In SemEval. find our card
  36. Z. Zhang, D. Robinson, and J. Tepper. 2018. Detecting hate speech on twitter using a convolution-gru based deep neural network. In Lecture Notes in Computer Science. Springer Verlag. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pavlopoulos-2020-toxicity-detection-does-context-really/