Elamur Research
☰
home / papers / 2019 / Predicting the Type and Target of Offensive Posts in Social Media

Predicting the Type and Target of Offensive Posts in Social Media

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-1144.pdf

Summary

Earlier work on harmful online language usually focused on a single phenomenon, such as hate speech, cyberbullying or aggression, and rarely asked who the abuse was aimed at. This paper treats offensive content as one problem with a layered structure. The authors propose a three-level annotation scheme: whether a post is offensive (A), whether the offense is targeted or just profanity (B), and whether a targeted offense is directed at an individual, a group or something else (C). They used this scheme to build OLID, a publicly released corpus of English tweets. The tweets were collected through keyword searches, half of them with political terms. Crowdworkers on Figure Eight labelled them, and disagreements were settled by majority vote. The authors then trained an SVM, a BiLSTM and a CNN as baselines for each level. The CNN was strongest on the first two levels. Target identification proved hardest, and the OTH class was never predicted correctly. OLID became the official data for SemEval 2019 OffensEval and offers a single resource covering several related abuse-detection tasks.

Key points

  • OLID contains 14,100 tweets, split into 13,240 for training and 860 for testing. Level A labels are NOT/OFF, level B labels are TIN/UNT, and level C labels are IND/GRP/OTH.
  • Keywords included "MAGA", "antifa", "liberals", "conservatives", "gun control" and replies to BreitBartNews. The authors aimed for roughly 30% offensive content. A 300-tweet expert trial reached a Fleiss' kappa of .83 on level A.
  • The best macro-F1 scores were 0.80 for level A (CNN) and 0.69 for level B (CNN). For level C, the BiLSTM and CNN tied at 0.47, with the SVM close behind at 0.45.
  • All models scored zero F1 on the OTH target class. The authors attribute this to the class being heterogeneous and having few examples (395 training tweets).
  • The data was anonymised: URLs and user mentions were replaced with placeholders. OLID served as the official dataset of SemEval 2019 Task 6 (OffensEval).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

393 papers of our library cite it (of the 140673 whose bibliographies we read); 15 of them have a page here, 378 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1144

@inproceedings{zampieri2019predicting,
  title = {{Predicting the Type and Target of Offensive Posts in Social Media}},
  author = {Zampieri, Marcos and Malmasi, Shervin and Nakov, Preslav and Rosenthal, Sara and Farra, Noura and Kumar, Ritesh},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1144},
  url = {https://doi.org/10.18653/v1/n19-1144}
}

References

18 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  2. Pete Burnap and Matthew L Williams. 2015. Cyber hate speech on twitter: An application of machine classification and statistical modeling for policy and decision making. Policy & Internet, 7(2):223–242. find
  3. Maral Dadvar, Dolf Trieschnigg, Roeland Ordelman, and Franciska de Jong. 2013. Improving Cyberbullying Detection with User Context. In Advances in Information Retrieval, pages 693–696. Springer. find
  4. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of ICWSM. find our card
  5. Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018. Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language. In Proceedings of GermEval. find
  6. Jun-Ming Xu, Kwang-Sung Jun, Xiaojin Zhu, and Amy Bellmore. 2012. Learning from Bullying Traces in Social Media. In Proceedings of NAACL. find
  7. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, Suwon Shon, James Glass, Yves Scherrer, Tanja Samardžić, Nikola Ljubešić, Jörg Tiedemann, et al. 2018. Language Identification and Morphosyntactic Tagging: The Second VarDial Evaluation Campaign. In Proceedings of VarDial. find
  8. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval). In Proceedings of SemEval. find our card
  9. Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate Speech Detection with Comment Embeddings. In Proceedings of WWW. find
  10. Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. Proceedings of EMNLP. find our card
  11. Ritesh Kumar, Atul Kr Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking Aggression Identification in Social Media. In Proceedings of TRAC. find
  12. Irene Kwok and Yuzhou Wang. 2013. Locate the Hate: Detecting Tweets Against Blacks. In Proceedings of AAAI. find
  13. Shervin Malmasi and Marcos Zampieri. 2017. Detecting Hate Speech in Social Media. In Proceedings of RANLP. find
  14. Shervin Malmasi and Marcos Zampieri. 2018. Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence, 30:1 – 16. find
  15. Hamdy Mubarak, Darwish Kareem, and Magdy Walid. 2017. Abusive Language Detection on Arabic Social Media. In Proceedings of ALW. find
  16. Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive Language Detection in Online User Content. In Proceedings of WWW. find
  17. Mohammad Sadegh Rasooli, Noura Farra, Axinia Radeva, Tao Yu, and Kathleen McKeown. 2018. Cross-lingual Sentiment Transfer with Limited Resources. Machine Translation, 32(1-2):143–165. find
  18. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding Abuse: A Typology of Abusive Language Detection Subtasks. Proceedings of ALW. find our card