Elamur Research
☰
home / papers / 2019 / SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)

SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1903.08983

Summary

OffensEval, Task 6 of SemEval-2019, tackled the automatic detection of offensive content on social media. Earlier datasets usually covered one phenomenon, such as hate speech, aggression or cyberbullying. This task instead looked at both what kind of offense a post contains and who it is aimed at. The organizers built a new corpus for this purpose, the Offensive Language Identification Dataset (OLID). It consists of English tweets labelled through crowdsourcing under a three-level hierarchical scheme. Each level became a sub-task: (A) offensive versus not offensive, (B) targeted insult versus untargeted profanity, and (C) whether the target is an individual, a group or something else. Because the classes are heavily skewed, systems were ranked by macro-averaged F1. Participation was very high, making it one of the most popular SemEval-2019 tasks. The strongest entries relied on fine-tuned BERT and ensembles, while classic machine-learning classifiers also remained common. OLID was released publicly and has already been adopted in university NLP courses. This makes it a reusable benchmark linking offense type and target.

Key points

  • OLID has 14,100 English tweets (13,240 for training, 860 for testing), plus a 320-tweet trial set released beforehand. Annotation was done on Figure Eight.
  • Labels are hierarchical: NOT/OFF at level A, TIN/UNT at level B, and IND/GRP/OTH at level C. Roughly two thirds of the tweets (9,460) are NOT.
  • About 800 teams registered and 115 submitted results. Sub-task A drew 104 teams, B drew 71 and C drew 66, but only 50 teams wrote system description papers.
  • NULI won sub-task A with fine-tuned BERT, preprocessing the tweets by splitting hashtags into their component words and replacing emojis. The team vradivchev anikolov came first in sub-task C using BERT, and jhan014 topped sub-task B.
  • Planned future work includes enlarging OLID, reducing class imbalance, expanding the small test sets for B and C, and adding other languages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

408 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 395 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1903.08983 · arXiv 1903.08983

@misc{zampieri2019semeval,
  title = {{SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)}},
  author = {Zampieri, Marcos and Malmasi, Shervin and Nakov, Preslav and Rosenthal, Sara and Farra, Noura and Kumar, Ritesh},
  year = {2019},
  doi = {10.48550/arxiv.1903.08983},
  eprint = {1903.08983},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1903.08983}
}

References

82 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Piush Aggarwal, Tobias Horsmann, Michael Wojatzki, and Torsten Zesch. 2019. LTL-UDE at SemEval-2019 Task 6: BERT and two-vote classification for categorizing offensiveness. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  2. Guy Aglionby, Chris Davis, Pushkar Mishra, Andrew Caines, Helen Yannakoudakis, Marek Rei, Ekaterina Shutova, and Paula Buttery. 2019. CAMsterdam at SemEval-2019 Task 6: Neural and graph-based feature extraction for the identification of offensive tweets. In Proceedings of The 13th International Work… find
  3. Lutfiye Seda Mut Altin, Alex Bravo Serrano, and Horacio Saggion. 2019. LaSTUS/TALN at SemEval-2019 Task 6: Identification and categorization of offensive language in social media with attention-based Bi-LSTM model. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  4. Flor Miriam Plaza-del Arco, Dolores Molina-González, Teresa Martı́n-Valdivia, and Alfonso Ureña-López. 2019. SINAI at SemEval-2019 Task 6: Incorporating lexicon knowledge into SVM learning to identify and categorize offensive language in social media. In Proceedings of The 13th International Work… find
  5. Segun Taofeek Aroyehun and Alexander Gelbukh. 2018. Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC), pages 90–97. find
  6. Himanshu Bansal, Daniel Nagel, and Anita Soloveva. 2019. HAD-Tübingen at SemEval-2019 Task 6: Deep learning analysis of offensive language on Twitter: Identification and categorization. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  7. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. CoRR, abs/1607.04606. find
  8. Pete Burnap and Matthew L Williams. 2015. Cyber hate speech on Twitter: An application of machine classification and statistical modeling for policy and decision making. Policy & Internet, 7(2):223–242. find
  9. Aleix Cambray and Norbert Podsadowski. 2019. Bidirectional recurrent models for offensive tweet classification. arXiv preprint arXiv:1903.08808. — [arXiv:1903.08808](https://arxiv.org/abs/1903.08808) find
  10. Maral Dadvar, Dolf Trieschnigg, Roeland Ordelman, and Franciska de Jong. 2013. Improving cyberbullying detection with user context. In Advances in Information Retrieval, pages 693–696. Springer. find
  11. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the International Conference on Weblogs and Social Media (ICWSM). find our card
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technology (NAACL… find
  13. Karthik Dinakar, Roi Reichart, and Henry Lieberman. 2011. Modeling the detection of textual cyberbullying. In The Social Mobile Web, pages 11–17. find
  14. Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate speech detection with comment embeddings. In Proceedings of the Web Conference (WWW). find
  15. Ehsan Doostmohammadi, Hossein Sameti, and Ali Saffar. 2019. Ghmerti at SemEval-2019 Task 6: A deep word- and character-based approach to offensive language identification. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  16. Zeyad El-Zanaty. 2019. Zeyad at SemEval-2019 Task 6: That’s offensive! An all-out search for an ensemble to identify and categorize offense in tweets. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  17. Paula Fortuna, José Ferreira, Luiz Pires, Guilherme Routar, and Sérgio Nunes. 2018. Merging datasets for aggressive text identification. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC), pages 128–139. find
  18. Paula Fortuna, Juan Soler-Company, and Nunes Srgio. 2019. Stop PropagHate at SemEval-2019 Tasks 5 and 6: Are abusive language classification results reproducible? In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  19. Nicolò Frisiani, Alexis Laignelet, and Batuhan Güler. 2019. Combination of multiple deep learning architectures for offensive language detection in tweets. arXiv preprint arXiv:1903.08734. — [arXiv:1903.08734](https://arxiv.org/abs/1903.08734) find
  20. Avishek Garain and Arpan Basu. 2019. The Titans at SemEval-2019 Task 6: Hate speech and target detection. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  21. Spiros V Georgakopoulos, Sotiris K Tasoulis, Aristidis G Vrahatis, and Vassilis P Plagianakos. 2018. Convolutional neural networks for toxic comment classification. arXiv preprint arXiv:1802.09957. — [arXiv:1802.09957](https://arxiv.org/abs/1802.09957) find
  22. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for Twitter: annotation, features, and experiments. In Proceedings of the Annual Meeting of the Associatio… find
  23. Fréderic Godin, Baptist Vandersmissen, Wesley De Neve, and Rik Van de Walle. 2015. Multimedia Lab @ACL WNUT NER Shared Task: Named entity recognition for Twitter microposts using distributed word representations. In Proceedings of the Workshop on Noisy User-generated Text. find
  24. Bharti Goel and Ravi Sharma. 2019. USF at SemEval-2019 Task 6: Offensive language detection using LSTM with word embeddings. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  25. Mario Graff, Sabino Miranda-Jiménez, Eric S. Tellez, and Daniela Moctezuma. 2019. INGEOTEC at SemEval-2019 Task 5 and Task 6: A genetic programming approach for text classification. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  26. Yaakov HaCohen-Kerner, Ziv Ben-David, Gal Didi, Eli Cahn, Shalom Rochman, and Elyashiv Shayovitz. 2019. JCTICOL at SemEval-2019 Task 6: Classifying offensive language in social media using deep learning methods, word/character n-gram features, and preprocessing methods. In Proceedings of The 13th In… find
  27. Jiahui Han, Xinyu Liu, and Shengtan Wu. 2019. jhan014 at SemEval-2019 Task 6: Identifying and categorizing offensive language in social media. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  28. Vijayasaradhi Indurthi, Bakhtiyar Syed, Manish Shrivastava, Manish Gupta, and Vasudeva Varma. 2019. Fermi at SemEval-2019 Task 6: Identifying and categorizing offensive language in social media using sentence embeddings. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEv… find
  29. Madeeswaran Kannan and Lukas Stein. 2019. TüKaSt at SemEval-2019 Task 6: something old, something neu(ral): Traditional and neural approaches to offensive text classification. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  30. Prashant Kapil, Asif Ekbal, and Dipankar Das. 2019. NLP at SemEval-2019 Task 6: Detecting offensive language using neural networks. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  31. Emad Kebriaei, Samaneh Karimi, Nazanin Sabri, and Azadeh Shakery. 2019. Emad at SemEval-2019 Task 6: Offensive language identification using traditional machine learning and deep learning approaches. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  32. Ritesh Kumar, Guggilla Bhanodai, Rajendra Pamula, and Chennuru Maheshwar Reddy. 2019. bhanodaig at SemEval-2019 Task 6: Categorizing offensive language in social media. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  33. Ritesh Kumar, Atul Kr Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking aggression identification in social media. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC). find
  34. Irene Kwok and Yuzhou Wang. 2013. Locate the hate: Detecting tweets against blacks. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). find
  35. Ping Liu, Wen Li, and Liang Zou. 2019. NULI at SemEval-2019 Task 6: Transfer learning for offensive language detection using bidirectional transformers. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  36. Debanjan Mahata, Haimin Zhang, Karan Uppal, Yaman Kumar, Rajiv Ratn Shah, Simra Shahid, Laiba Mehnaz, and Sarthak Anand. 2019. MIDAS at SemEval-2019 Task 6: Identifying offensive posts and targeted offense from Twitter. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEva… find
  37. Prasenjit Majumder, Thomas Mandl, et al. 2018. Filtering aggression from the multilingual social media feed. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC), pages 199–207. find
  38. Shervin Malmasi and Marcos Zampieri. 2017. Detecting Hate Speech in Social Media. In Proceedings of the Conference on Recent Advances in Natural Language Processing (RANLP). find
  39. Shervin Malmasi and Marcos Zampieri. 2018. Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence, 30:1 – 16. find
  40. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations. find our card
  41. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the International Conference on Neural Information Processing Systems (NIPS). find
  42. Jelena Mitrović, Bastian Birkeneder, and Michael Granitzer. 2019. nlpUP at SemEval-2019 Task 6: a deep neural language model for offensive language detection. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  43. Sandip Modha, Prasenjit Majumder, and Daksh Patel. 2019. DA-LD-Hildesheim at SemEval-2019 Task 6: Tracking offensive content with deep learning model using shallow representation. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  44. Preeti Mukherjee, Mainak Pal, Somnath Banerjee, and Sudip Kumar Naskar. 2019. JU ETCE 17 21 at SemEval-2019 Task 6: Efficient machine learning and neural network approaches for identifying and categorizing offensive language in tweets. In Proceedings of The 13th International Workshop on Semantic Ev… find
  45. Alex Nikolov and Victor Radivchev. 2019. Nikolov-Radivchev at SemEval-2019 Task 6: Offensive tweet classification with BERT and ensembles. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  46. Alexander Oberstrass, Julia Romberg, Anke Stoll, and Stefan Conrad. 2019. HHU at SemEval-2019 Task 6: Context does matter - tackling offensive language identification and categorization with ELMo. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  47. Ryan Ong. 2019. Offensive language analysis using deep learning architecture. arXiv preprint arXiv:1903.05280. — [arXiv:1903.05280](https://arxiv.org/abs/1903.05280) find
  48. Gustavo Henrique Paetzold. 2019. UTFPR at SemEval-2019 Task 6: Relying on compositionality to find offense. In Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval). find
  49. Gabriel Florentin Patras, Diana Florina Lungu, Daniela Gifu, and Diana Trandabat. 2019. Hope at SemEval-2019 Task 6: Mining social media language to discover offensive language. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  50. John Pavlopoulos, Nithum Thain, Lucas Dixon, and Ion Androutsopoulos. 2019. ConvAI at SemEval-2019 Task 6: Offensive language identification and categorization with perspective and BERT. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  51. Gretel Liz De la Pea and Paolo Rosso. 2019. Deep-Analyzer at SemEval-2019 Task 6: A deep learning-based ensemble method for identifying offensive tweets. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  52. Ted Pedersen. 2019. Duluth at SemEval-2019 Task 6: Lexical approaches to identify and categorize offensive tweets. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  53. Andraž Pelicon, Matej Martinc, and Petra Kralj Novak. 2019. Embeddia at SemEval-2019 Task 6: Detecting hate with neural network and transfer learning approaches. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  54. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP). find
  55. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech… find
  56. Andrei-Bogdan Puiu and Andrei-Octavian Brabete. 2019. Towards NLP with deep learning: Convolutional neural networks and recurrent neural networks for offensive language identification in social media. arXiv preprint arXiv:1903.00665. — [arXiv:1903.00665](https://arxiv.org/abs/1903.00665) find
  57. Arun Rajendran, Chiyu Zhang, and Muhammad Abdul-Mageed. 2019. UBC-NLP at SemEval-2019 Task 6: Ensemble learning of offensive content with enhanced training data. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  58. Murugesan Ramakrishnan, Wlodek Zadrozny, and Narges Tabari. 2019. UVA Wahoos at SemEval-2019 Task 6: Hate speech identification using ensemble machine learning. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  59. Priya Rani and Atul Kr. Ojha. 2019. KMIColing at SemEval-2019 Task 6: Exploring n-grams for offensive language detection. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  60. Alon Rozental and Dadi Biton. 2019. Amobee at SemEval-2019 Tasks 5 and 6: Multiple choice CNN over contextual embedding. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  61. Jonathan Rusert and Padmini Srinivasan. 2019. NLP@UIOWA at SemEval-2019 Task 6: Classifying the classless using multi-windowed CNNs. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  62. Angel Deborah S, Rajalakshmi S, Logesh B, Harshini S, Geetika B, Dyaneswaran S, S Milton Rajendram, and Mirnalinee T T. 2019. TECHSSN at SemEval-2019 Task 6: Identifying and categorizing offensive language in tweets using deep neural networks. In Proceedings of The 13th International Workshop on Sem… find
  63. Silvia Sapora, Bogdan Lazarescu, and Christo Lolov. 2019. Absit invidia verbo: Comparing deep learning methods for offensive language. arXiv preprint arXiv:1903.05929. — [arXiv:1903.05929](https://arxiv.org/abs/1903.05929) find
  64. Alessandro Seganti, Helena Sobol, Iryna Orlova, Hannam Kim, Jakub Staniszewski, Tymo-teusz Krumholc, and Krystian Koziel. 2019. NLPR@SRPOL at SemEval-2019 Task 6 and Task 5: Linguistically enhanced deep learning offensive sentence classifier. In Proceedings of The 13th International Workshop on Sema… find
  65. Elena Shushkevich, John Cardiff, and Paolo Rosso. 2019. TUVD team at SemEval-2019 Task 6: Offense target identification. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  66. Pardeep Singh and Satish Chand. 2019. Pardeep at SemEval-2019 Task 6: Identifying and categorizing offensive language in social media using deep learning. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  67. Murali Sridharan and Swapna T. 2019. Amrita School of Engineering - CSE at SemEval-2019 Task 6: Manipulating attention with temporal convolutional neural network for offense identification and classification. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  68. Steve Durairaj Swamy, Anupam Jamatia, Björn Gambäck, and Amitava Das. 2019. NIT Agartala NLP Team at SemEval-2019 Task 6: An ensemble approach to identifying and categorizing offensive language in Twitter social media corpora. In Proceedings of the 13th International Workshop on Semantic Evaluatio… find
  69. Nithum Thain, Lucas Dixon, and Ellery Wulczyn. 2017. Wikipedia Talk Labels: Toxicity. find
  70. D Thenmozhi, Senthil Kumar B, Srinethe Sharavanan, and Aravindan Chandrabose. 2019. SSN NLP at SemEval-2019 Task 6: Offensive language identification in social media using machine learning and deep learning approaches. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval… find
  71. Johnny Torres and Carmen Vaca. 2019. JTML at SemEval-2019 Task 6: Offensive tweets identification using convolutional neural networks. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  72. Harrison Uglow, Martin Zlocha, and Szymon Zmys-lony. 2019. An exploration of state-of-the-art methods for offensive language detection. arXiv preprint arXiv:1903.07445. — [arXiv:1903.07445](https://arxiv.org/abs/1903.07445) find
  73. Bin Wang, Xiaobing Zhou, and Xuejie Zhang. 2019. YNUWB at SemEval-2019 Task 6: K-max pooling cnn with average meta-embedding for identifying offensive language. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  74. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. arXiv preprint arXiv:1705.09899. find our card
  75. Gregor Wiedemann, Eugen Ruppert, and Chris Biemann. 2019. UHH-LT at SemEval-2019 Task 6: Supervised vs. unsupervised transfer learning for offensive language detection. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  76. Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018. Overview of the GermEval 2018 shared task on the identification of offensive language. In Proceedings of the GermEval 2018 Workshop (GermEval). find
  77. Zhenghao Wu, Hao Zheng, Jianming Wang, Weifeng Su, and Jefferson Fong. 2019. BNU-HKBU UIC NLP Team 2 at SemEval-2019 Task 6: Detecting offensive language using BERT model. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  78. Jun-Ming Xu, Kwang-Sung Jun, Xiaojin Zhu, and Amy Bellmore. 2012. Learning from bullying traces in social media. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technology (NAACL-HLT). find
  79. Zhang Yaojie, Xu Bing, and Zhao Tiejun. 2019. CN-HIT-MI.T at SemEval-2019 Task6: Offensive language identification based on BiLSTM with double attention. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  80. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Langu… find our card
  81. Chengjin Zhou, Jin Wang, and Xuejie Zhang. 2019. YNU-HPCC at SemEval-2019 Task 6: Identifying and categorising offensive language on Twitter. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find
  82. Jian Zhu, Zuoyu Tian, and Sandra Kübler. 2019. UM-IU@LING at SemEval-2019 Task 6: Identifying offensive tweets using BERT and SVMs. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval). find