Elamur Research⌕
☰
home / papers / 2021 / Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detection

Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detection

Summary

The paper asks whether the polarized 2020 US presidential race appears as hostile language among Twitter supporters of Joe Biden and Donald Trump, and whether stance and offensiveness can be studied together. The authors collected 382.210 tweets through the Twitter API v1.1, covering six weeks before the vote, election day and one week after. They used candidate names, campaign hashtags and derogatory nicknames as search terms. From this pool they manually labelled 3000 tweets. Each tweet received a stance toward three targets (Trump, Biden, Kanye West) from an extended label set, plus a binary hateful/offensive (HOF) tag. The HOF guidelines were adapted to count insulting nicknames for political rivals. After several rounds of guideline refinement, annotator agreement reached acceptable levels. Manual analysis showed that tweets opposing Biden carry HOF more often than tweets opposing Trump, although the authors conclude that Trump supporters do not use significantly more hate speech overall. A BERT baseline detects support for a candidate well but performs worse on opposition and especially on HOF. The publicly released corpus supports research on offensive language in relation to stance.

Key points

  • Stance labels: The scheme goes beyond the usual Favor/Against/Neither by adding Mixed (both positive and negative aspects) and Neutral (facts, quotes or an unclear position). This lets annotators mark mentions of a candidate that express no opinion.
  • Annotation procedure: Three computational linguistics students annotated in four iterations. Cohen's κ for HOF rose from −0.02 in the first round to 0.62–0.73 in later rounds. Stance κ ranged from 0.61 to 0.88.
  • HOF distribution: About 11.7% of the annotated tweets are HOF. The rate is 26.7% among tweets against Biden versus 18.5% among tweets against Trump.
  • BERT baseline results:
    • Favor stance: .89 F1 for Trump and .91 F1 for Biden.
    • Against stance: .79 F1 for Trump and .64 F1 for Biden.
    • HOF detection: .53 F1.
    • The authors attribute errors partly to class imbalance and to the world knowledge that hate detection sometimes requires.
  • Availability and future work: The corpus and guidelines are available from IMS Stuttgart (stance_hof_us2020). The authors suggest joint multi-task modelling of stance and offensive language as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

31 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 28 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 2103.01664

@inproceedings{grimminger2021hate,
  title = {{Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detection}},
  author = {Grimminger, Lara and Klinger, Roman},
  booktitle = {Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
  year = {2021},
  eprint = {2103.01664},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.01664}
}

References

27 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmet Aker, Leon Derczynski, and Kalina Bontcheva. 2017. Simple open stance classification for rumour analysis. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 31–39, Varna, Bulgaria. find
  2. Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter. In Proceedings of the 13th International Wor… find
  3. Peter Burnap and Matthew Williams. 2014. Hate speech, machine classification and statistical modelling of information flows on twitter: interpretation and communication for policy decision making. In Internet, Policy and Politics, Oxford, United Kingdom. find
  4. Jessica Clement. 2019. Twitter: monthly active users worldwide. https://www.statista.com/statistics/ 282087/number-of-monthly-active-twitter-users/. ↗
  5. Thomas Davidson, Dana Warmsley, Michael W. Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM, pages 512– 515, Montréal, Québec, Canada. AAAI Press. find our card
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  7. Karthik Dinakar, Birago Jones, Catherine Havasi, Henry Lieberman, and Rosalind Picard. 2012. Common sense reasoning for detection, prevention, and mitigation of cyberbullying. ACM Trans. Interact. Intell. Syst., 2(3). find
  8. Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. ACM Comput. Surv., 51(4). find
  9. Lei Gao and Ruihong Huang. 2017. Detecting online hate speech using context aware models. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 260–266, Varna, Bulgaria. INCOMA Ltd. find
  10. Ona de Gibert, Naiara Perez, Aitor Garcı́a-Pablos, and Montse Cuadros. 2018. Hate speech dataset from a white supremacy forum. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 11–20, Brussels, Belgium. Association for Computational Linguistics. find our card
  11. Dilek Küçük and Fazli Can. 2018. Stance detection on tweets: An SVM-based approach. CoRR, abs/1803.08910. — [arXiv:1803.08910](https://arxiv.org/abs/1803.08910) find
  12. Nikita Lozhnikov, Leon Derczynski, and Manuel Maz-zara. 2020. Stance prediction for russian: Data and analysis. In Proceedings of 6th International Conference in Software Engineering for Defence Applications, pages 176–186, Cham. Springer International Publishing. find
  13. Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the HASOC track at FIRE 2019: Hate speech and offensive content identification in indo-european languages. In Proceedings of the 11th Forum for Information Retrieval Evaluat… find
  14. Miljana Mladenović, Vera Ošmjanski, and Staša Vujičić Stanković. 2021. Cyber-aggression, cyberbullying, and cyber-grooming: A survey and research challenges. ACM Comput. Surv., 54(1). find
  15. William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In Proceedings of the Second Workshop on Language in Social Media, pages 19–26, Montréal, Canada. Association for Computational Linguistics. find
  16. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31– 41, San Diego, California. Association for Computational Ling… find
  17. Saif M. Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. 2017. Stance and sentiment in tweets. ACM Trans. Internet Technol., 17(3). find
  18. John T. Nockelby. 2000. Hate speech. Encyclopedia of the American Constitution, 3:1277–1279. find
  19. Björn Roß, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, and Michael Maximilian Wojatzki. 2016. Measuring the reliability of hate speech annotations: The case of the european refugee crisis. https://duepublico2.uni-due.de/ receive/duepublico mods 00042132. ↗
  20. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10, Valencia, Spain. Association for Computational Linguistics. find our card
  21. Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124, Los Angeles, CA. Association for Computational Linguistics. find
  22. Ellen Spertus. 1997. Smokey: Automatic recognition of hostile messages. In Proceedings of the Fourteenth National Conference on Artificial Intelligence and Ninth Conference on Innovative Applications of Artificial Intelligence, AAAI’97/IAAI’97, page 1058–1065. AAAI Press. find
  23. Galen Stolee and Steve Caton. 2018. Twitter, trump, and the base: A shift to a new form of presidential talk? Signs and Society, 6:147–165. find
  24. Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 327–335, Sydney, Australia. Association for Computational Lin… find
  25. Andranik Tumasjan, Timm Sprenger, Philipp Sandner, and Isabell Welpe. 2010. Predicting elections with twitter: What 140 characters reveal about political Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings of… find
  26. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin L… find our card
  27. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. SemEval-2020 task 12: Multilingual offensive language identification in social media (OffensEval 2020). In Proceedings of the Fourteenth Work… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/grimminger-2021-hate-political-opponent-twitter-corpus/